I found the marketing to be cute and consistent with the types of games they were putting out; the marketing depicted something that was family-friendly and good in groups of people, and that's just the type of game it had.
I also don't think any amount of marketing could have made the console more popular with the M-rated games developers. The Wii's claim to fame was motion controls, which I can't really see being that great for first or third person shooters, where rather than sharing the remote around between a group of people you are using it alone for extended periods of time (maybe it was just me, but those controllers were bloody uncomfortable). There's also the significant technical limitations imposed by the hardware, which would require a lot of creativity to overcome, were it even possible. Of course, being creative is also risky, which isn't always desirable.
I think that if Nintendo wants games that attract the "core" gamers, it needs to do one of two things: make a console just like everyone else's, or make some successful games for that market. Making a console that is easy to port to/from with respect to the other systems is the easy way out; so long as your system is as or more powerful than the others, it becomes trivial to get the major franchises on your system. Better would be for them to show developers that a "core"-style game can be successful on their platform, so developers have something to base their own games off of. Nintendo would then be taking on the big innovation risk, and allowing third parties to start making variations on what they have shown can be successful.