When those sequels are FIFA, AC, CoD etc.. I'm not too worried. The Wii expanded audience is gone, but I don't think the core are going anywhere. Who knows, the XBone might find itself another blue ocean.
Handhelds are definitely on a steep decline. Consoles might be, but we just don't know yet.
Well, it's definitely feasible that consoles could face the same struggles. I just don't find it as likely. The crossover with tablets and smartphones just isn't as large.
Thing is, as somewhat capable boxes like Amazon's come out, some of the more casual couch gamers might skip more traditional consoles.
As someone else said, smartphone games scratch the gaming itch enough for many people that they don't need a dedicated handheld anymore. The same might be said in a couple of years time about Amazon box etc.