If you test the resolving ability of the human eye at 100 yards (100 meters is close enough too) you will find that a healty normal human eye can barely resolve alternating 1 inch wide back and white bars.
That means 1" is about the best you can manage to align sights with a target without using a scope sight.
I watched 2 world class women shooters testing Eley ammo lots with Anschutz rifles many years ago. Their 10 shot groups hovered around 1 inch at 100 yards. If you put aperture sights on your Lee Enfield you would be able to hold that type of precision with a quality barrel and quality ammo. Given that your groups are larger at 2", the extra 1 inch dispersion is a combination of your vision, less that the best sights, less than a perfect barrel and less than perfect ammo fired in less than perfect conditions.
I would say compared to most iron sighted rifles yours is very good.
Rather than blame your rifle or ammo I would expect that 2 things would improve your groups. Add a quality set of aperture sights and shoot off of a quality bench rest.
I bought a high quality bench rest and leather bags about 30 years ago. The result was instant accuracy improvement for all of my ammo and all of my rifles.
Expensive yes, but I think mine has paid for itself several times over by the components saved while performing load development.