There are two points of confusion on this issue:
1. Otakudom's western perception as a shunned niche, has been exaggerated by the backlash against weeabooism. At first some people exaggerated how Japan is the Promised Land of Otaku, and appropriated the phrase as a badge of honor. To mock their enthusiasm, others have gleefully rubbed in any anti-otaku or anti-anime news from Japan that they could find, about how Japan actually views them as pathetic immature neckbeards. In reality, otaku roughly means "geek" in Japan, both with it's history as an insult, and it's recently hip usage for all pop culture fans.
2. Anime as a medium *IS* a hardcore otaku field in Japan, but that only applies to the series, not to the art style or to games, manga, light novels, and other moe designs, which are more popular.
For random cultural reasons, it happened that while millions would read moe manga in public, very few would buy it's late night anime adaptation.
this is similar to how comic books are a hardcore geek niche in the west, while their characters, their superpower ideas, and aesthetics are "mainstream geek".
