I personally really enjoyed Bioshock Infinite, and I'm currently playing through it a second time on a hard mode whilst finding all of the voxophones, infusions, and sightseers.
I haven't really played the first Bioshock in a long time, so comparing that and Infinite is kinda difficult for the most part.
However, I know that Infinite did two things a lot better than the first game:
1. The main character. Now, this is mostly opinion based since I'm sure some people would prefer playing as a silent protagonist they can project themselves onto. Me though, I prefer playing as someone with an established character and voice. Booker Dewitt was a very interesting guy and I enjoyed playing as him.
2. The ending. Say what you want about it, but to me, it was so much more complete and satisfying than the ones we got in the first game. The twist at the end was mind blowing, and while it was pretty confusing at first, it sorta made some amount of sense when you really think about it.
And of course, there's Elizabeth, who is pretty much the heart and soul of the game.
As for whether or not I like Rapture or Columbia better? That's a tough question. I guess I sorta like Rapture a bit better, probably because I like the underwater city theme a bit more. But Columbia was also a really cool place, it was vibrant, colorful, and has a ton of detail to it.
Overall, I feel that it's a great game. Definitely worth buying.