Well, lucky for you I guess, to land a job in this economy... problem is that might be what a lot of other people are thinking if they find out. IE lucky to be born with the right set of chromosomes to (unwittingly) take advantage of a short sighted HR policy which favoured that characteristic over actual suitability for the job.
Or, who knows? Did you do any kind of aptitude tests, etc? Maybe your performance in that, or your specific set of qualifications and experience, actually meant you were the more suitable candidate despite the length of the other applicant's CV? A long work history is no guarantee of skill or knowledge. They could just be having a dumb joke with you that "of course we hired you... SOMEONE had to even up this sausage party a bit".
There may be a touch of regular discrimination at work, too. As teching has quite a pastoral, interpersonal, customer service element to it, they maybe thought that hiring someone with better "inherent" skills in that regard (to get into their mindset here; for the moment, I'm not espousing my own personally held views) would be quite valuable to improve the client experience as well as, yknow, making them look better. Because women can talk and emphasise better than guys, right?
I dunno, I wouldn't chuck the job away over it, guilt or otherwise. Unless you see the guy destitute in a couple years time when there were other jobs you were also being interviewed for with a good chance of success (or having even got offers since taking this one)... It's their decision, which you took advantage of without knowing the full implications. Fortune has rolled in your favour, let the ones who made the call live with it. Hopefully you'll do an excellent job and it'll prove to be the best piece of quasi sexism they ever indulged in.