When you start with Lovecraft, it is important to know witch stories to start with. Some are just more easily accessible then others.
My advice: start with the Rats in the Walls, the Colour out of Space or the Music of Erich Zann, as many others also recommended. Just pick one, witch ever title you like most.
Rats in the Walls is about a man who returns to his ancestral home and finds a few skeletons in the closet. Near the end you will understand why this story is so great, and then it sucker punches you with a horrific ending.
The Colour out of Space is about a farmer and his family who are struck by a horrible plague after a meteor fell on their land. It sounds simple, but Lovecraft spends so much time setting the mood for the story, especially emphasising the weird and otherworldly nature of what is going on. This story will have your heart pounding from beginning to end, trust me.
The Music of Erich Zann is about a young man who takes a horrible crappy apartment in France, where he meets a mysterious musician who makes absolutely the weirdest and most beautiful music the protagonist ever heard. Again, this is story is about 90 percent atmosphere, and works wonderful. You won't be done with this story once you finish it, I can guarantee you that.
If you liked these stories and didn't find them too bookish or purple, you can basicly go anywhere.
Fan favourites like the Call of Cthulhu is still a very strong read, as is at the Mountains of Madness, although that is one of his more difficult ones.
An overlooked story that is a big favourite of mine is called the Thing on the Doorstep, quite frightening to me, and the ending disturbed me greatly.
Have fun (???) with Lovecraft, and let us know how it goes!