First, I have to ask why you are doing this at all, are you just a
glutton for punishment?  If your oil pressure is low you have bearing
problems, not an oil pump problem.  A high vol pump may buy you some
time, but you are better off rebuilding the engine, especially if you
are over 150,000 miles.  Been there, tried this.  Got 9000 miles before
the engine spun a rod bearing and threw the rod.  Then you will need to
find a rebuildable core engine to get back on the road.  Adds at least
$500 to the cost.

If for some reason you insist on doing this, you will need to drop the
rear of the engine down, watching that you don't over extend any hoses
or wires (you will have to disconnect the fuel lines, accel cable, and
probably other stuff), remove the rear exhaust pipe and muffler, engine
mounting bracket with heat shield, and then the rear pulley. I'm not
familiar with the 2.1 cooling plumbing, so some of that may also need to
be removed if it is in the way.

Then you can usually GENTLY rock the pump out after removing the cover
by using a couple of screwdrivers on opposite sides.  If it won't come
you can loosen those two case bolts near the pump, and even get an
inexpensive special puller from your local VW parts shop.

New pumps have always come with a cover in my experience, and the high
vol pump's cover on my new engine is aluminum with fins.  This is one
that my rebuilder has found that will clear the pulley, so be sure there
is no interference before you bolt it all back up. Pack the pump with
light multi purpose grease to help it get primed the first time.

Stuart