Yes, the bypass cover is where the fuel pump gets it's impulse. And, yes, you can take the cover off and peak inside.
But aside from that, what's the purpose? I'm more curious than anything else, but I'm now paying attention because one of mine is oozing green-black oily grime past the gasket.
New gaskets are on order...but it has me wondering: what do these "do"? What is the purpose of these covers?