Safety wise, most milsurps in good condition are pretty safe to shoot provided your not going beyond what was expected of the rifle in the first place. Certain rifles you need to take more care with, such as Gewehr 88s which can have a variety of factors that can make them less safe if you don't know what your doing.
For shooting, common ammo rifles which don't cost a small fortune are things like Israeli K98ks or other M98 Mausers that were converted to 7.62 NATO (tons of countries did this conversion as it just involved replacing the barrel, ex.
Chilean M1912s etc.), Ishapore 2A1 Lee Enfields in 7.62 NATO, Brazilan Mausers in .30-06, Mexican Mausers in .30-06,
Swedish Mausers are a great buy (M38s are better for the shooter because of the lower minimum sight setting) thought the 6.5 ammo is more expensive,
Finnish Mosins can be a lot of fun (the better shooting variants being the M28/30 and M39) and the ammo is fairly cheap, K31s are excellent shooters and if you find the ammo at the right price a steal of a deal the other
Swiss Straight pull designs are also fun but have 300m minimum rear sight settings.
Lee Enfields can be good but the ammo is more expensive and you have to be a bit careful with them (at least in
Canada, we have a cottage industry 'restoring' sporters and many are shoddily done). If you wanted to go the .303 route a P14 is a better choice for being a shooter, and the M1917 is also a great choice as well being in .30-06.
Arisaka Type 99s are also excellent shooters and usually have VG+ bores, very easy to reload for but commercial ammo is difficult. You could also go any number of Mausers in 8mm, but ammo costs are higher.
French Berthiers are also a ton of fun, and if the bore is in VG+ condition exceptionally accurate. They are easy to reload for, but can be expensive if your buying factory ammo.