I've seen a lot of front sights messed up with a punch because these sights are really in there. Use one of these.

This is a Swissicon Armoury issue adjuster. They're expensive and difficult to find. We have a pair of them here.

Top view



Bottom view.



......... but we also discovered something that's just as effective for a few bucks.
A simple bolt cutter with the tip flattened. Any hardware store carries them.