That could be the case. I have one of these in a '55 CD-12 I just put together. I also worked on a '54 Fleetwin that had one of these in it recently also. I believe both of those motors also had the brass stepped washer that holds the lower crankshaft seal gasket against the bottom of the powerhead. They have all been worn but only one was worn as much as yours. I'm sure when the roll pin starts to wear thru the wear on the washer accelerates too. I made sure the seam of the roll pin was not riding against the cup washer when I put them back together.
YouTube handle: Old JohnnyRude