In my experience 2 types of nerds got bullied; the genuinely neurodiverse who got singled out because they were "weird" (which funking sucks) and cocky assholes who thought they were smarter and better than everyone. Who thought that everyone better looking, more athleticly gifted and just generally more charismatic than them was actually just really stupid and beneath them.
My charitable reading goes something like this.
Many boys are unintentionally taught from a very young age that social motivation is optional. While girls are constantly pushed towards being sociable and defining themselves through their friendships and relationships, boys are allowed and in some cases encouraged to define themselves in terms of interests like sports or games. The general pattern of male friendships tends to assume that boys will bond over these shared interests and thus don't need to develop social motivation outside of them.
But the result of this is that boys have to go through a bit of tonal whiplash during adolescence when they suddenly realize they want to interact with girls but don't have the social skills to do so. Some boys are able to get over this and quickly develop those social skills, or find that they are attractive enough that girls will do most of the work for them, but others never quite manage to grow out of being focused on interests rather than people.
There is generally nothing wrong with these people. If they just had a bit more confidence and threw themselves into more social situations, they would eventually learn the social skills they needed, but instead the message they get is that there is something just fundamentally wrong with them and their interests. After all, they can't get over the idea that socialization is purely a matter of shared interests, so it never crosses their mind that someone might like them or enjoy spending time with them without also liking the same movies or video games or sports (and yes, this does happen with sports too.. sports nerds are still nerds).
I think there's a lot of other stuff involved. I think you're absolutely right in that there is a tendency for extremely mediocre boys to sometimes get the impression they're really smart and deep and interesting and to have a very warped sense of what they deserve. But fundamentally, I think boys aren't pushed hard enough to develop social motivation, and by the time they realize they need it a lot of them will have already internalized (incorrectly) that they're just bad at it.
I was a massive nerd in school, and also very obviously neurodiverse, but I never found myself in that weird culture of social pessimism a lot of nerds seem to fall into, and I credit that to the fact that I wasn't particularly homosocial.
Basically, I feel like a lot of young men learn the lesson of "girls don't like me because of my hobbies" rather than "(most) girls aren't interested in my hobbies and I don't know how to talk about anything else".