I'm going to toss in a wrench to two, as during the restoration of my no1mk6 1930 I discovered that same problem, the rear sight is too low to allow rotation to its rest.
Its part of the reason I've been trying to find out what the original sight maker was for these. I settled for now on a B marked sight and I radiused the front corner where the sight engages the plunger. The problem is that now it folds but isn't as firmly held in the upright or down locations as a stock no4.
Eventually a light came on; these rifles didn't use the stock no4 set up, they used a ball/spring set up, ala no1 mk5, which would remove the plunger. It does make enough room for rotation.
I'm yet to get around to trying out this new set up. Likewise my rifle shows no sign of being messed around with in the pivot area.
Maybe, your rifle has a long story and who knows why used a ball/spring setup.