While you CAN kill just about anything with a .22LR...
I would say .22WMR at a minimum, for most goat shooters, assuming you can get within 60 yards or so, and shoot good.
The problem with the little rimfires on light bodied animals is meat wastage (or a potential lack of penetration) if going for heart/lung shots.
Brain stem shots obviate this...but you have to be able to pull them off, ten times out of ten.
But going up to centrefire gives you more room for error. .357, 30-30, 7.62x39, .22hornet, .222, .223, 6.5x47...
Just have to know your gear, your ranges, your ability.
Is this out Clevedon ways?