Yes, I think he left soon after he began. Nonetheless, the question -- shorn of its pejorative formulation -- is a worthy one for discussion; and, as it is one -- at least the latter part --which has been given various answers over the years (at least since Addison's dealing with it in the Spectator nearly 300 years ago), yet has yet to be resolved, I think such a discussion could be very interesting.
Several of the people here have given some very good answers, nearly all of which I would agree with to some degree. I do think HPL is a much better writer, and as for Zadok Allen's shout... unfortunately, that was a convention of the time, and yes, I agree that it can be quite risible, depending on how it hits the individual reader. (It took me a long time to get past that very response when I hit the "titan elbow" in "The Shunned House", in other respects one of my very favorite among his tales.) I don't quite find the cannibal recluse in "The Picture in the House" as over-the-top, but that may be because I see him as being as much symbolic as literal: To me, he represents the past devouring its offspring, much as Chronos is depicted as doing... an image or concept which resonates throughout a great deal of HPL's work. This seems, to me, to fit very well with the opening paragraphs of the story, where he is creating such a complex reaction to these remnants of the past: respect, fascination, repulsion, distrust, fear, horror, wonder, a sense of the sublime, a feeling of indebtedness, and many other emotions, all touched on with just the right amount of each one to evoke that feeling of the other which is nonetheless terrifyingly familiar... somewhat like Freud's argument on the topic of the uncanny.
Ultimately, I think for me that is the appeal of Lovecraft. Yes, there are the layers of the familiar thrill and other emotional responses I have had from reading his works in the past; but he is such a multilayered writer that I find each time I visit one of his works, this is overlayed with something new; a new insight, a new facet which I had not previously experienced; just as with all great literature (and yes, I would put HPL in that category; if not among the highest, he is still within that august company). Ramsey Campbell (in the documentary Fear of the Unknown) makes the comment that, in reading M. R. James -- a fine writer in many ways, and certainly one of the most important figures in the history of the weird tale -- one experiences the same thing over again, perhaps slightly diluted; but in reading HPL one often finds something quite different wioth each reading, so that each time a story is read it is a different experience, and thus has a different impact as a whole.
"Horror",
per se, is only a very, very tiny part of it; and even then, the horror in HPL is frequently a symbol for something much greater and less easy to define. "Fear" may be closer to the mark, but "unease", a feeling of reality shifting just slightly from all our accepted concepts, is closest, I think. That feeling that all we accept as reality may be nothing more than comforting illusions we've built for ourselves, which may at any moment crack open and reveal the abyss which lies within as well as without. That can be a truly terrifying experience, and for those who genuinely encounter it, it tends to make them question everything forever after. Nothing is certain any longer.
Then there is the feeling of tragic isolation of his protagonists: so many of them encounter the culminating terror alone, and there is so seldom any solid evidence of its existence for others to find, thus leaving that person more alone than were they isolated on a remote island away from all human contact... because they are alone in the midst of teeming millions. They have truly become "the Outsider", "a stranger in this century and among those who are still men", still genuinely members of the human community. (This is why, despite its many flaws, that tale is one of the keystones of Lovecraft's work.)
There is also love of beauty: beauty of the natural landscape, beauty of language, the beauty of darkness and dream, of the fragility of all we hold dear (and whose very fragility and evanescence makes it all the more precious to us). After all, Lovecraft himself was a great lover of beauty, especially that of Nature and art; and his own prose style is actually much closer to the ideal of the poetic vision than to the flat, pedestrian prose to which we of the last century have become accustomed.
These things, too, are among those which what is mistakenly called "horror" so often offer. One has only to read, say, Gautier, or Baudelaire, or de la Mare, or Machen, or much of Poe or Hawthorne, or Mrs. Radcliffe, or even Spenser or Marlowe or Shakespeare, to realize that. Try, for instance, reading Thomas De Quincey's excellent essay, "On the Knocking of the Gate in 'Macbeth'", or Burke's A Philosophical Inquiry into the Supernatural in Poetry", or Anna Laetitia Barbauld's "On the Pleasure Derived from Objects of Terror", or Sir Devendra P. Varma's The Gothic Flame, among just a few, to get a better understanding of what it is which appeals to readers of "horror" stories... at least, those who respond to something other than fake gore and "stinger" moments (and after all, generally speaking, what is less effective than verbal buckets of blood
on a page of writing -- unless it actually symbolizes something more?). As Lovecraft himself noted early on in Supernatural Horror in Literature:
There is here involved a psychological pattern or tradition as real and as deeply grounded in mental experience as any other pattern or tradition of mankind; coeval with the religious feeling and closely related to many aspects of it[...]
and just as prone to a sense of wonder, awe, mystery, and majesty, as the best religious or mystical art.
That, in brief, is the appeal of Lovecraft, and "horror" in general.....