'EddieB' - The rear-sight on your Krag purchase is a model 1896 sight. You also have the model 1896 hand-guard. (You are correct the sight slide is on backwards. The 'locking-knob' should be on the right side of the leaf).
Krag hand-guards are sight specific: the 1896 sight takes the hand-guard that you have, the model 1898 and 1902 sights use a different sight opening, and the 1901 sight uses yet a different opening.
If you change rear-sight models, you have to change to the appropriate hand-guard.
What is the serial number of your Krag? What source did you look up the date in? Some on-line material is wrong.
Because there was a lot of 'musical chairs' with Krag rear-sights, during updates, the model 1896 sight can be correct on some model 1898 U.S. Krag rifles.
The tape adhesive should dissolve and remove easy with linseed oil or "Howard's Wax and Feed" on a rag with some hard rubbing. (You can try turpentine or mineral spirits first and finish up with Linseed or Howard's).
Some photos to assist taking your sight apart and identifying various Krag hand-guards:
Attachment 105146Attachment 105147