Use a brass drift punch and hammer.

Here's a set of brass drift punches.
https://www.milsurps.com/images/imported/2015/01/_35JPGset_id880000500F-1.jpg

$19.95 with free shipping... not too bad...