You mentioned "serious readers" of his work... that's opening a dangeous door, my friend....
Personally, I think it best to read HPL from the beginning (at least of his fiction and his fantastic poetry, which at times ties into his fiction in various ways); That way you get to see his Lovecraft Mythos (to be distinguished from the Cthulhu Mythos) develop and spread its tentacles. But for a modern newcomer that might not be the optimum choice.
However, I first became really interested in him by reading "The Call of Cthulhu", which has also caught the imaginations of a goodly number of people (Neil Gaiman to the contrary notwithstanding). This is a good introduction to his themes and is itself the first "solid" step into developing the myth-figures which feature in so much of his later fiction.
As for where to go for them... the Penguin editions are quite good, despite an occasional typo; but I believe (ningauble can correct me on this) that there's another edition which may be even better, as it collects together all the fiction save revisions.
In any event, happy hunting, and I hope you enjoy!