No. 4 Mk 2 (F) 7/53 Removing rear sight pin retaining pin

[Steve762]

Having a problem removing the small retaining pin that holds the rear sight pin in place on a No. 4 Mk 2 (F) 7/53.

The small pin has very little movement down and none up. What is the trick for getting the retaining pin out? Is the pin tapered?

Any info on diameter of the pin would be helpful. What pin punch size is best?
Thx, Steve