No, it wasn't the wrong thing to do - this is a stupid argument, and it rests heavily on all this wishy-washy moral garbage that people really need to let go. About 250,000 people were killed in Hiroshima and Nagasaki when the atomic bombs were dropped; according to military estimates, this is how many Americans would have died had they invaded the Japanese homeland - this is not taking into account the number of Japanese who would have been killed, not to mention those who would have been seriously wounded on either side (Military experts estimated that just the American casualties would have been around one million).
People also seem to never take into account the after effects of an invasion of the Japanese homeland - lets say that America actually did take that route; once they'd successfully defeated the Japanese, American troops would have had to remain there for another 10 to 15 years in order to stabilise the country politically, at which point political dissidents would have undoubtedly caused even more American (and Japanese) deaths.
Thirdly, the American government had spent almost 2 billion dollars on the Manhatten project (and this was in the 1940s) - if the Truman administration had decided not to use this weapon, and instead put American lives at risk again (keep in mind they've been embroiled in bloody conflict with the Japanese for almost 3 years now), there would have been an enormous public outcry (rightly so).
There's also the issue that in order for an invasion of Japan to be carried out, Truman would have had to request military assistance from Stalin and the USSR; and really, at this point in time, tensions were running extremely high between the two leaders, and that was the last thing Truman wanted to do. And finally, everyone should keep in mind that it was because of Hiroshima and Nagasaki that the Atomic bomb was never used again - if it hadn't been used in Japan, it would have undoubtedly been used later on, and we would have seen just as terrible a tragedy take place.