Oh, this one's going to be a hot potato!!!
It's a thorny problem, and far too many confuse what entertains them, what they like, etc. with something that has "literary merit", which I take to mean lasting value, something not simply for the generation it was written in, but which still speaks to those long after, as well. And that, I think, is the crux of the matter: it has to speak to deep human feelings/longings/emo- tional needs on some level in order to have true literary merit. This does not mean it has to have truly universal appeal (that is, it does not have to appeal to everyone across the board) -- else nothing would earn that term. But it does have to reach a fairly representative number of (at least fairly well) educated and discriminating people over a period of time. Which is why, for example, the majority of the early Gothic writers of the late eighteenth and early nineteenth centuries don't apply (though I have a fondness for them myself); they were often artificial, contrived, and paid little attention to real human thoughts and emotions (even in the limited realm of fear/terror they often used artificial standards). This is also why so few young writers have produced such, because it generally takes a certain amount of life to garner the experience to judge what is and what isn't genuinely affecting. There are exceptions, but they are relatively few. It is also why so few professional writers quite fit the bill (though, again, there are exceptions); because often -- especially during the pulp era -- the motivation was simply to put bread on the table by entertaining rather than probing the human condition in any meaningful way (which, incidentally, has as a prerequisite that it be intrinsically interesting, as it touches chords the reader can identify with on multiple levels). This is not to knock the pulp writers, many of whom I highly respect and enjoy reading, and would hate to see their work go out of print; simply that the necessity of selling to an artificially standardized magazine often limited the scope of what a writer could say or how they could say it. It also has to be informed with the writer's Weltenschauung in such a way as to convey an idea of the world from a perspective no one else has done, in an intelligible way that gives at least a haunting glimpse of the world through someone else's eyes and mind (and, if I'm allowed the phrase, heart). It has to have more to it than simply entertaining (laudable though that be, and I consider it a worthy enough goal in itself). It has to open up the reader to an inner experience he/she has not had before, and in such a way that it adds to their understanding of life, the world, and the universe around us. It often leaves us with a sense of awe, an almost humbling experience, because truly meritorious work contains a beauty that catches the breath and leaves one with something precious, even if dark, unpleasant, horrific, etc.; the best of it is also in language that is itself beautiful, no matter what it conveys. And -- at least in dealing with human emotions/reactions, etc. -- it must be firmly based in realism, no matter how fantastic the tale itself may be. (Which is why so much horror fails on this score; the people within the tale simply don't react the way someone confronted with such a situation is likely to act or feel in reality. Few people confronted with a genuine violation of natural law would show heroics. They're more likely to be left trying to simply keep their mind from breaking. I'm not talking here about slashers, etc. I'm talking a truly supernatural event -- an impossibility in reality, of course, as anything which happens happens in accord with natural processes.)
A longer answer than I intended, but -- for the moment -- it's the best I can do.
Next victim? (Fools rush in......)