The issue is smacking it with something soft, of course you don't want to damage it, but the reality is that method requires the gun front sight solid and secure and metal to metal ala hammer to X (or at most some thin thing to cushion it)

A large punch works, spreads out the force. Anything brass or metal or even something made.

If you can get a front site removal tool those work smooth and nice but helps to cushion them as well (depends on whats on the end of the pusher shaft)

You can make a clamp and screw system to do it.

Will see if I can find my front sight adjustment tool ref, works to remove as well.