This is how most of psychiatry works though. There are very few "scientific" or "concrete" ways to determine a psychiatric diagnosis except listening to a person and watching their behavior. For the most part those diagnosis work however, because a lot of professionals with a lot of experience in psychiatric care have talked to and watched a lot of people and realized that some things are recurring or very similar.
Do you feel the same way about depression? Anxiety? Schizophrenia? Psychosis? Bipolar disorder? Personality Disorders? All of these are firmly diagnosed by a combination of personal history, next of kin's experiences and the observations of the diagnosing professional. We can't take a blood sample to determine depression. Shit, even the prevailing theory about serotonine and dopamine deficiencies in the synapses is still mostly a theory because there's no surefire way to test the actual signal substances levels in a living persons brain (at least not that remains ethical and safe). But psychiatric care professionals have seen enough depressed people to know that it is "a thing". What we call depression might be one thing or it might be several that are actually not all that related, but since we can't reliably check signal substances or hormon levels we diagnose based on subjective symptoms.
Gender Dysphoria and Transsexualism both became established because a lot of people expressed these things. Many of them seeking or being led to psychiatric care for their woes. Some who seek for these symptoms end up getting another diagnosis (autism is a common co-morbidity for gender dysphoria and an important differential diagnosis, for example) but for many it is a serious and obvious psychological problem. It is not "up for a person to decide" and if anyone comes in and goes like "I felt like a guy last year but now I feel like a woman and before that I was non-binary" that's a serious warning sign. Gender Dysphoria is characterized by long term problems stemming from the psychological distress of not fitting in with one's assigned gender.
If you want to argue it isn't real go right ahead, but then I'll also ask you to argue that borderline personality disorder and generalized anxiety disorders aren't real too, because they are diagnosed in much the same way.
I think this sounds closer to depression than schizophrenia because it's a blanket thing that we don't really understand due to the variance of symptoms ascribed that label and is kinda like a half scientific half layman's term diagnosis, one which people often make for themselves with no experts involved. Whereas schizophrenia is people thinking there is spying devices in their walls and mind controlling sugar in their coffee, stuff like that, it's way more narrow so I have an easier time seeing that as credible.
But yeah I don't think it isn't real, I think it's something, I don't see much to go on to believe it must be what people say it is, but it's definitely not nothing. Maybe it's just another form of autism, where someone just fixates on this being true but it's more them being autistic about it than anything else? If it's a co-morbidity that'd at least make some sense and it has just about as much concrete evidence behind it as the other hypotheses I see.