I have borrowed a number of rifles and all I feel that I needed to do is a couple of dry fires to get a feel for the trigger
I agree, that's all I need to look at before hunting with a borrowed gun unless shots are long. You don't need to shoot a gun if you're sitting in a tree with 50 yards being a long shot.
Growing up, we had 4-5 people sharing 2 rifles. Heck, 3 of us when antelope hunting at the same time with 1 rifle - never had a problem but we all shot the gun before hand at 100 yards.
For me, there needs to be a trust factor in place though. You need to know the mounts/rings are tight, the right ammunition is being shot, etc. Do I trust Elmer Fudd's word and rifle? No. Do I trust my dad's? Yes