There’ll be a mark underneath the adjustable portion of the rear sight (if that makes sense). If it were mine, I’d take a small punch and give it a good whack right where the original stake mark was.
If you didn’t want to go that route and would prefer something a bit more reversible, JB Weld would hold quite well in this application.