Yes, your millivolt readings should be fluctuating quickly if the engine is
running properly and in closed loop mode. It's when the voltage doesn't
fluctuate that you're in trouble. The fluctuation is due to the constant
readjustment of the ECU/O2 system in trying to find the perfect air/fuel
mixture. Only in open loop mode should the voltage remain stable (at .5).
Ken Lewis has a good deal of information about this on his site.