So was race science though.
It's why you ended up with "research" showing how clearly the genitalia of white prostitutes resembled that of black women, or medical literature describing how masturbation would cause people to literally devolve into lesser forms of human life and pass their inferior traits onto their children. You could argue (and I would) that scientific racism actually necessitates some kind of intra-racial hierarchy to cover for its own bullshit. If you've decided that white people are the superior race, you then need some explanation for why aren't all the perfect supermen you're claiming they are. Every system of racism has had to produce such an explanation.
Race science, certainly at its high point, was a universal system for measuring human intelligence, inclination to criminality and general worth by reference to their physical features. That's not to say phrenology is the same thing as race science, but the two are so entangled as to be pretty inseparable in historical context. After all, until quite recently almost everyone believed in a scientific racial hierarchy. Not doing so was incredibly difficult, because it was simply part of how the world worked. I seriously doubt there were many phrenologists who did not accept the existence of a racial hierarchy as an indisputable truth of the world.
Scientific racism today is a little different to the scientific racism of history, because in order to survive racists have had to rationalize it and strip it of all the things which just sound stupid and comical to a modern audience. One way of rationalizing racism is by playing down the idea of intra-racial differences and positing it as purely a comparison between racial groups. Of course, let's be real, scientific racists today still believe in the theory of hereditary degeneracy. They still believe that there are superior and inferior people within racial groups who have different racial characteristics despite technically being the same "race". They need to, because otherwise the theory doesn't work. Race is not actually a real thing, it has no scientific basis. To give it a scientific basis, you need to talk about things which are real instead of talking about race, and then pretend that one actually equals the other.