IMO any wild animal can be a challange under certain circumstances and to pick one animal would be folly...
I have hunted Bushbuck, Vaal Rhebok, and Kudu, and thought each of them was surely the most challanging animal in the world, Then the next time I hunted them we drove out, walked a 100 yards or so and shot them broadside as they stood and looked at us...Happens all the time.
A Mule Deer buck over the 30 inch spread can be almost a phantom, but the ones that are shot usually walked out in front of some kid...
I have to say, I have no answer to this post, as any wild animal can be elusive as hell on any given day or dumber than a box of rocks on another day.....