If the lights flicker at very low RPM i.e. low idle that is ok (not
completely normal but it happens).

If they stay on at driving speed, most likely your alternator or
voltage regulator (cheaper item inside the alternator) is not working.

On most Vanagons the voltage regulator can be changed without even
removing the alternator.

Remember, always disconnect the battery ground cable before working
on your alternator (and most electrical items for that matter)

>So here I am driving and these lights are coming on - does that mean I'm
>not getting a battery charge and I should stop for the night?