Yes, it's totally normal. Also notice that when you turn on your fogs, for instance, the headlights will slightly dim for a second. That's just a momentary lapse in electric current. Same thing (just opposite) when you step on the gas: sudden surge of power.
If it's REALLY significant, then you may have an alternator problem. But from the sounds of it, totally normal.