I think he has a valid reason to be worried, as I have heard many Gamers complain that companies are wasting time trying to appeal to different people--not a bad idea for a business mind you--and are leaving their core base out in the cold. I'm not entirely behind this statement, but I've heard it enough to believe that it must have some small truth to it.
Oh, and let's not forget that Jeff Brown pretty much backed up exactly what Iwata-san said. "Social gaming as a whole aggregates into a business that is undeniably big money. When it's that big you are forced to pay attention." So, in order to pay attention, you need to cut funding from other departments, other projects (you know, the ones that have proven to make you money from your core supporters), and dump all the money into "paying attention" to this new market. Once you think you've figured it out, you buy up all the small studios you can, lay off half their work force to save costs, then force the other half to do just asmuch work, then yell at them when, for some reason, the games they make under your company name aren't nearly as good when they worked alone. You then proceed to fire them and move on to the next trend, motion controls.
Hm...maybe Iwata-san knows what he's talking about after all...