Welcome to the club! Many of us have gone through that delicious angst
Based on personal experience, here is my feedback...
1) Skip the .416 ... OR ... make the .416 your only big bore
There is tremendous overlap between a .375 shooting 350 gr slugs and a .416 shooting 400 gr slugs, and there is just as much overlap between a .416 shooting 450 gr slugs and a .458 shooting 500 gr slugs.
I sold my .416 Rigby.
2) Consider the concept of "stopper" rifle
Whether you need one or not is a very personal question, but if you think you do, they start at .45 / 500 gr, period. Anything smaller, including .40 is lacking.
Sure, the modern .458 Win does it all, and just as reliably, but why not take a Lott that gives you an edge and can still shoot .458 Win is you prefer.
Mine is a .458 Lott.
3) Ah! the romance of the double...
Sure, oceans of inks have been spilled talking about this fabled lighting-quick second shot, but keep in mind that shots #3 and 4 are a long time away.
Sure, many like the sweet shooting .450/400, but if you go the double route because of this famed second self-defense shot, it almost HAS to be in a stopper caliber, otherwise, what is the point?
Mine is in .470 because ammo choices are the best/most available.
One more point, anything past 50 yards becomes more complicated with a double, while it is still very, very simple with a scoped rifle. Things have a way to happen in the hunting fields, and that last-day shot at 100 yards, or nothing, is a real possibility. I carried my .470 for days and days after elephant in Zim and lion in the Kalahari, and ended up shooting the elephant with the scoped .375 H&H and the lion with the scoped .458 that I had asked one tracker to carry just for this purpose.
My Krieghoff .470 is for sale.
4) Bigger not necessarily better.
Yep, the various .500 Jeff, .505 Gibbs, .450 Rigby / .460 Weatherby (both share essentially the same case with or without belt and the Rigby can be up loaded), .577 and other shoulder canons are cool, but darn few can shoot them accurately. A 300 gr slug in the right place beats any day a .500 gr (or bigger!) slug in the wrong place, and THERE IS a recoil threshold for each of us.
Mine is at the proven 500 gr @ 2,150 fps (e.g. .458 Lott / .470 NE) in a 11 lbs. double or heavy-barreled & low-power scoped bolt rifle.
5) Think practical
. Unless romance supersedes any other concern - which is quite legitimate, I was on that bandwagon myself for decades
- it is hard to beat a R8 Blaser with a PG barrel (.300); the ubiquitous .375 H&H barrel (cats, eland, one-rifle mix bag safari); and a .458 Lott barrel (buff, hippo on land, elephant).
Truth be told, the .375 will suffice, but the .458 sure hits them a lot harder if they do not get the memo at the first shot and come to inquire. This is why I use one... just in case...