The first one was great, no question, but it felt like the second one took everything that was great about the first and built upon them while leaving behind most of the crappy parts.
Things I feel that were improved by the second:
-The Hammerhead. Well, okay, it's an improvement by default.
-Ammo. I know it seems weird, but having infinite ammo in the first made me really impatient because it's hard to quantify damage-per-shot when you've got no reason to keep track of how many you've fired, and discouraged me from trying new weapons because my assault rifle was a firehose of death with a bottomless reservoir. Plus, scavenging for clips and reloading made combat feel more varied and tense.
-Space ships. In the first Mass Effect, the only ships you saw for any significant amount of time were the Destiny Ascension, Sovereign and the Normandy, and everything else was more-or-less left for you to imagine based on a small picture in the codex. It really took away from how rewarding the space battle in the end could have been, because without the ability to tell a Turian ship from a human one, I honestly had no idea what the hell was going on. The only real exceptions to this were the Geth ships, but if you put them side-by-side with a Covenant vessel from the Halo universe and I'd have to give it serious thought. It was kind of like a city without cars. The second game adds the derelict reaper, the Shadow Broker's jaw-droppingly awesome ship, the Collector vessel, and shows your team interacting with the citadel transports. It's not a huge improvement, but it's enough to remind you that you are in the future, and not just being ferried from place to place by a space ship so you can have conventional firefights under a funny-looking sky.
-Characters. Well, this one's simple: I don't know whether there were more characters to interact with, but with how memorable most of them are in the second, it certainly feels like there were more. Who made a bigger impact, Fai Dan or the Asari on Illium that wanted to perform experiments on the colony? Keep in mind that Fai Dan was a mission-critical character (if you don't remember him, he's the colonist on Feros who shot himself in the head), whereas the Asari was just an opportunity to score Paragon/Renegade points. Stronger, more distinct characters helped to add a very personal touch, and it really contributed to the overall experience in a "this is what you're fighting for" kind of way... to say nothing of your crew.
-Art. The first game had a highly washed-out color scheme, to the point that the prothean flashbacks were shockingly vibrant. The plausible-future look served its purpose, but it was never particularly striking. The sequel, likely because it was free to build on a pre-established setting, was much more stylized and attractive, looking very much like a populated future instead of a bunch of industrial sites. The colors were more saturated, the architecture had character, and the technologies went from utilitarian to stylish-but-functional. Now, with that said, it may just be personal preference on my part, but I feel like if Feros was designed within the same parameters as Illium, it would've made for a much more memorable experience (those sky bridges were a missed opportunity to show off a gorgeous vista to establish the uniqueness of the setting).
And that's just a couple of things off the top of my head.