The problem is not the Average Intelligence of an American Citizen, which many of American Escapists seem to believe is nonexistent, but rather the problem of implementation.
While the American Government, as well as American Scientists, Engineers, Mathematicians, Physicists, Chemists and the like have naturally adopted metric, the same cannot be said for colloquial use or in the case of our many road and highways. The real obstacles to introducing metric into these facets of American life are the sheer geographical size of the country as well as the problem of the states.
First of all, the country itself just far too large to rapidly introduce metric for use on road. The effort required to switch them all over would require large scale interstate cooperation and spending that just doesn't seem likely in this current situation. You would also have to rapidly change all of the testing and licensing requirements at every DMV in the country in a way that wouldn't prevent already capable drivers from qualifying or cause too much slowdown. The problem is that most DMV's are run by the individual county/city/state, rather than by a single national office. Even now, DMV's have a very bad reputation for being slow and inefficient in some parts of the country. Changing the official system would only worsen the problem. Even then, Imperial has been established as colloquial/customary unit measurement throughout the United States.
Outside of the impression given by our cultural exports, ie Hollywood, there is no single or solid "American Culture". There are a number of common tropes and idioms, but it's not enough to establish a traditionally defied national identity, which is also my theory on why we just don't get into the world cup.
In the end, metric has been adopted where it counts, and it's not going to hurt anyone if people use imperial at home or daily speech.
Although it would really help math and science scores if we started teaching it at the elementary school level, rather than in High School or University.