They're backup weapons. Not only that, they can be extremely useful.
Scouts bat can often be the difference between life and death if your shotgun is empty,
The soldier's pickaxe gives a speed boost at low health, and can kill most wounded enemies too when you rush them,
Pyro's axetinguisher does a lot more damage than continual fire, and combined with the airblast switching and using it is a lot more effective as fire is delayed after airblasting. In the right situation you should always use the axe.
Demo has the eyelander, which combined with the charge is a one hit kill on most classes, and increases your health and speed. Not only that, but using the bottle in extreme close range is preferable to avoid self damage.
If you surprise and enemy with the heavy the free crits can tear down half the other team
The medic can use it as a risky option to build his charge faster,
the sniper can get out of many a close range situation with his. Probably the most useful to him than any other as he has no close range capabilities otherwise,
The spy's entire playstyle is centred around his melee.
So I don't see what you're saying. At all. Just learn to switch weapons faster. Turn on quickswitch if you haven't already.