A total drift of 3/16 and a sight radius of 32" (check that) shifts the POA by about 9" (relative to the barrel center line) at 50 yards. If the POI didn't move, then I agree with hi-revr that you may have been aiming with the foresight protector ear instead of the foresight blade. Especially as the grouping was OK.
Apart from that, if the rifle is not centered on the target with the blade within about 1/16 of the barrel center line, then I would certainly check for barrel distortion, channel contact etc. But since the grouping is fine, the trouble will be something curable.