Go for it , then . .30-06 is all my dad uses and he's taken everything from White tail to Indian Shiliguri Gaur bison with it .The recoil isn't something I am worried about. I have a 280ai in a 5.75lbs rifle, a 300wm 6.6lbs rifle and a 300 rum in a 7.7lbs package, they will all get your attention, but I don't shoot them off the bench very much.
I am leaning to the 30-06. If I ever get a second one in a short action, the 284 Rem is his pet chambering, so that would be at the top of the list. That is followed by the 6.5x55 or maybe the 6.5x57 that was mentioned earlier.