An essential thing would be a strong sense of place. That remains a strength of Lovecraft's own writing when some of its other aspects are not necessarily compelling. A problem for the would-be Lovecraftian author is that few people have anything like his imaginative engagement with any place.
That is a direction that the Lovecraftian author might be nudged to go. Imitation of his style or his plot habits is dubious; I think that Lovecraft himself may have been trying to extricate himself from some of these things in his last few years. (Unlike some of his admirers, I see Lovecraft as a developing author, much of whose work labors under deficiencies that he himself was trying to outgrow; so it is ironic if someone tries to imitate stories that the man himself might well have regarded as apprentice work.)
The engagement with place has two aspects. The more familiar one is the creation of imaginary landscapes, as in his rather triumphant
Mountains of Madness. Here is a combination of sense of place that he derived from his reading plus attention to wholly imaginary scenes. Lots of authors do this sort of thing. In sf/fantasy circles, they talk about "world-building."
The second aspect is that about which I was thinking in my third sentence above. Here Lovecraft has some interesting kindred spirits. We have not heard very much yet about Lovecraft and, say, Joseph Mitchell (New York --
Up in the Old Hotel) or even today's author Ian Frazier. In his interest in the local, the small town rather than the global economy, his appreciation for lifeways that have grown up in close interaction with the particular creeks, fields, weather, etc. of a specific area, Lovecraft has affinities with essayist Wendell Berry and the Front Porch Republic bloggers.
Lovecraft had a strong element of a strain of conservatism. Many Chronsfolk probably have almost no idea of what I am talking about. For them, "conservatism" means "right-wing" militarism, unbridled free market consumerism, etc. In fairness to these ignorant folk, there seems to me, on the national level at least, almost no political activity that reflects the kind of conservatism of which I'm speaking, so how can they be blamed for not being aware of it?
Anyway, much of HPL's writing does show much interest in a particular region. Ask yourself, if you are a Lovecraft fan who aspires to write "Lovecraftian" fiction: Do you love (≠ idealize) the place where you live, or some place where you used to live, and do you know it well or at least want to to know well? Do your shelves contain books about the place where you live or where you used to live? If you think to write a "Lovecraftian" story, would it be rooted in a particular place (whether the real names where used or not), or would you be writing based on a recycling of imagery from stories you've read, movies you've seen, etc.?
You have probably been educated and socialized for just the other kind of sentiment, i.e. mobility is good, see the world, "multiculturalism," "diversity," "global economy," etc. College teachers are generally far indeed from the "Lovecraftian" sentiments I've tried to suggest:
http://www.frontporchrepublic.com/2015/06/rootless-professors-reboot/
So the more "educated" you are perhaps the less you are interested in the small, the local, etc., the less rooted you are. But Lovecraft at least believes (you can believe he was wrong) that he drew a great deal of strength from his roots.
I haven't tried to write a Lovecraftian story in about 40 years, but I wrote two back in the day, and both were set in my region (e.g. Kalmiopsis wilderness area).
This doesn't make any aspect of the stories any good! But when I aspired to write a "Lovecraftian" story, I did tune in to this "sense of place" aspect that I'm haranguing you about.
My one and a half cents.