People have weird views when it comes to matchmaking ages. Generational gaps are pretty clearly delineated when it comes to pairing with other people - he's literally more than twice your age and easily old enough to be your father. So I can sort of see that aspect, though.. well, to be fair, I'm the sort of person who views age as less important than compatibility to put it mildly. But y'know what? Screw those people. Age is a number, not a definitive aspect of one's mentality.
As for the teacher aspect, well.. that's where it becomes more sharply inappropriate. Most businesses have firm rules for their employees that prevent those sorts of close relationships coming into being between people of different hierarchical positions, say a manager and an office drone. The simple reasoning lays in three words - conflict of interest. If a manager has to decide to lay off (and in this we mean fire, sack, terminate, or end one's employment, not.. whatever else you could misconstrue "lay off" as, you filthy minded degenerate. The same goes for sack, for that matter.) an employee, do they choose their significant other or someone they don't know all too well? You see it in the military as well, and in fact in every kind of place or situation where one individual has direct power over another - fraternizing with those under your command is difficult to handle at best because you can't really keep neutral.
In this very specific instance, who's to say that the guy never ended up marking your assignments a little higher? Or treating you with a little more attention than other students? Did he possibly neglect some poor kid who absolutely needed help in a subject because he was assisting you at the time? I don't mean this as an accusation, merely the simple fact that it's a possibility is the problem.
What it all comes down to as of now, though, is if you're both happy. If you are? Then once more, screw what other people think. They're in no position to judge, nor are you in a position to be judged if you're no longer his student. Accept that some people are going to give you strange looks, and some are going to vocally disapprove - just like anything that falls outside of normal socially constructed boundaries. And hell, if naught else works, take to carrying around a Bible, that thing is loaded with significantly overaged men married to exceptionally underaged girls.
I really hesitate to condemn you to the purgatory that is TvTropes, but they have an article on the subject and its related themes.. so, try not to get too lost in the virtual woods, hm? http://tvtropes.org/pmwiki/pmwiki.php/Main/TeacherStudentRomance
For that matter, take care not to get lost in the woods in an even more metaphorical sense as well; you're moving a bit quick for someone so young, and risking quite a phenomenal amount. Take it from someone who's dealt with love frequently enough to understand that it's a very possibly fleeting thing if you're too rushed going into it. Keep your head relatively clear, and try not to let yourself get put into a situation that you don't want to be in - if you're completely dependent on this man, what will happen in twenty years when he spies another pretty teenager? It's a bit of a cliche I suppose, but things don't stay wondrous forever unless you get phenomenally lucky, and have a significant amount of effort put in on both sides.
I mean, I'm only three years older than you and you still come across that way to me. If half of what you say is true, you're in for one hell of a bumpy ride with a sad ending. Good luck.
