I am extremely fascinated by HPL's letters, even more so than his weird literature although I wouldn't be able to explain why.
I have the same reaction. I tended to ignore his fiction for some reason (lack of access being one), but recently read some of his letters as well as his essays (on horror fiction and cats) and was very amused.
Really interesting stuff, and he wrote in such a way as to explain his views so clearly and passionately.
"I, in my own senescent mellowness, confess to an inordinate and wholly unphilosophic predilection for tiny coal-black kittens with large yellow eyes, and could no more pass one without petting him than Dr. Johnson could pass a sidewalk post without striking it." Cats And Dogs, H. P. Lovecraft