Actually, the shock and impact of the bullet puncturing the body and severing nerves and blood vessels sustaining the ecosystem that keeps the subject alive kills them, not the bullet. The bullet may instigate this phenomena, but it needs to be explosively discharged, usually through a hollowed out tube, to perform this task. This tube is mounted on a frame which houses a mechanism to discharge the bullet through the tube, which when properly aimed, is directed towards the target and causes said rupturing of blood vessels and severs the aforementioned nerves on impact. Whether it kills the subject is highly situational and dependent on where the bullet enters: the puncturing of a place like the skull will more likely kill the subject than the puncturing of an arm. It's possible to survive either if treated correctly: most of the time however, the subject will suffer from a hemorrhage and die before treatment can be provided in the headshot case.
One more thing: I'm not one of those hippie types either: I own a personal sidearm by necessity due to the fact that I live in a bad neighborhood. Knowing how to own, operate, and store such things responsibly is key to preventing incidents like the Connecticut case from happening again.