Yeah... how DID this happen??
I've tried to think through the possibilities... an errant Cavalry unit on joint exercises with the Horse Guards? The U.S. Cavalry presaged Marine Embassy guards??? Along more likely lines, I can only suggest that the Cavalryman is possibly a Brit who had served in the U.S. Cavalry and came home with his uniform, having made off with the trapdoor, or a U.S. trooper who emigrated. A variation on this would be a U.K. collector who had the outfit and had his picture made... or a partygoer in - as you Brits say: "fancy dress..."
One more thought occurs...
Photographers in those days often toured areas, took pictures, and later made up pictures and postcards for sale. Possibly the British photographer toured the States and took some shots of people and places for his library of photo cards for sale.