Since his stuff is in the public domain, I'd do what I did and read *everything* he's done, chronologically. You'll get to see him evolve. You've got the clunky, purple horror stories, then the Dream Saga, and then the Cthlulhu mythos.
Or you can skip that and just read his big hits, but I found it was more satisfying to watch Lovecraft stumble toward finally writing a solid story, and when he hit it, it was cool.
I read In the Mountains of Madness on the bullet train in Japan a few visits back, and it was very good. But it was all the more good because of the lead-up.
In the main, I'd agree with you -- I think that the best way to read HPL is to do so chronologically -- that is, according to when a piece was written, not published (which can be radically different, as some of his works were written nearly 20 years -- or more! -- before they saw publication). I don't know if I'd strictly include juvenilia such as "The Beast in the Cave" or "The Alchemist" (written when he was 14-15 and 18, respectively), but from the point of "The Tomb" (1917) onward, I think it is helpful, as you not only see him developing his craft, but the development of his ideas and the linkage between his various works.
The one place I'd differ with you is that I don't think it is a linear progression. For example, "The Tomb" is really quite an accomplished piece; often very subtle and understated, and extremely well structured; the foreshadowing comes in as early as the epigram, and yet all this does is enhance the effect (and the ambiguity). On the other hand, "The Hound" (1922) is overwritten (quite possibly deliberately and self-parodically) and strained, the supernatural incidents aren't given enough connection to make a coherent pattern, etc. Even in his later work, "The Dreams in the Witch House" (a personal favorite of mine, by the way) is less accomplished than "The Colour Out of space", which he wrote five years before. And, of course, the "Dream Cycle" or, more properly "Dunsanian" tales (as only one or two of these are actually set in a dream world originally,*) are scattered throughout the years 1919 and 1921 or 1922 (if one wishes to count "Hypnos" as such), with a few late additions such as "The Silver Key", "The Strange High House in the Mist" and
The Dream-Quest of Unknown Kadath coming in 1926-27 (and all, incidentally, written one after the other, though he began
Kadath before writing "The Silver Key" and finished it after).
A lot of this can be attributed to the fact that Lovecraft was often experimenting with different ways of storytelling, as well as different types of weird tale (or non-weird tale, even comic tales, such as "A Reminiscence of Dr. Samuel Johnson" or "Old Bugs" or "Ibid"), and therefore was at times less successful than at others.
Aside from these few more or less pedantic points, I'd agree with you. I would also add, incidentally, that one should read his fantastic poetry at the same time, as some of it plays into the prose fictional work; and even several of the revisions he did, as he ties his own work in with these at various points (usually by references within the revision tales rather than in his "own" work, with the notable exception of "The Whisperer in Darkness", which does make reference to the so-called "revision mythos").
Just to make it easy for any interested parties, this link:
http://www.hplovecraft.com/writings/fiction/chrono.aspx