I'd say the biggest reason the Vita is in trouble is one most people have already said: A lack of 'Must Own' games. The only game the Vita has that I would want is Persona 4 Golden, but I'm not going to shell out for a new handheld when I've already got Persona 4 on the PS2; yeah, there are new features in Golden, but not enough to warrant the costs. And there isn't really too much hype about upcoming games for the Vita that I'm aware of; the gaming sites I frequent have news stories about upcoming 3DS games, but not much about the Vita.
Compare that to the 3DS, which has Donkey Kong Country 3D coming out next week, Animal Crossing: New Leaf next month, along with stuff like Mario & Luigi: Dream Team, Pokemon X & Y later this year, and A Link to the Past 2 or whatever they end up naming it. There's also Shin Megami Tensei IV coming out, and Ace Attorney 5 on the eShop in the fall.
Not to mention you get shafted by Sony on memory card prices, since they are proprietary tech used by Sony, while the 3DS can use any SD card, to my knowledge.