A few reasons writers and readers might like this approach that immediately occur:
1. A lot of these books still contain humour, moments of heroism, glimpses of a better world, which may shine all the more brightly amongst the filth.
True. But hitting the same down beat over and over and over is simply dull from a writing perspective, you have to give some variety otherwise the readers leave. Besides, the rest of the milieu is still unremittingly dark. So overall the effect is the same.
2. You might come back from your crappy job, open your book and think, phew, someone has it way worse than me, I feel better about life.
Again, that people are like this in general creeps me out. "Huh. At least I don't have it as bad as
that guy." Your life's okay as long as someone else's is worse. That's a great way to be.
3. Dark and dangerous worlds can give a visceral excitement of the kind you get from a tough action or horror film.
Right. That's titillating the audience. You're giving them excitement for excitement's sake. In this case blood, gore, violence, death, rape, unremitting glorification of nasty and dark things. Again, that explicitly creeps me out that people get off on that kind of stuff.
4. A book might present a horrible world and a horrible outcome in order to say, 'the real world is awful in many ways, how could it be better, or how might we stop it being worse?' A la 1984.
That's laughable at best. We've already got that well and truly covered. I accept that writers want to elevate their craft to art, hell even high art, and everything. But there's nothing Orwellian about this grimdark stuff. The cautionary tale of what me might become if certain courses are taken instead of others is all well and good, but of Orwell's nine books, only 1984 was a dystopia. His entire bibliography isn't one bleak future after another. Sounding the same hollow note again and again and again.
5. A really powerful depiction of horrible behaviour might make people consider their own behaviour and that of others. Which is to say, some people might read in order to think about our world rather than escape from it.
I read SF to think about the world, I read fantasy to escape it. To each there own. But you're excluding another relevant part of the equation here. The people who read the grimdark fantasy because they really just enjoy the blood, gore, filth, and darkness. They actually really like that stuff. Not as an admonition to rethink their ways, but to simply smile when another character is brutalized. Hence the creeps.
But what about a bit of honesty though. Really Joe and Mark. Would you guys have continued to write in this subgenre even if it didn't sell? Would you continue to hammer away at these kinds of stories and settings if it didn't get you published or keep you published? Your first books were both written in this subgenre, they got you published, they got you noticed, they got you paid. You have no real incentive to go elsewhere, do you? I'm not saying you're selling out or sold out or any of that naive pap, no, but you're writers, you want to write and get published. If the subgenre dries up, you'll move on, right? So there is some clear financial aspect to this, however small it may be.
Not many writers pick a fairly narrow subgenre and stick with it come hell or high water. Most are simply to eclectic themselves to stay put, while others are happy to bounce from subgenre to subgenre or genre to genre just telling the stories they like.
This stuff about publishing slots doesn't really represent the industry as I see it from the inside, by the way. An editor doesn't have x number of slots to fill and if they haven't by year end they just publish the next six books through the door. Nor if a fantastic commercial manuscript arrives do they fling up their hands and say, dash it all, it's the best book I've ever read but I have no free slots left this year. There are general concerns about costs and profits, of course, and editors' freedom varies greatly from place to place and according to their success. But generally they're looking at each manuscript on its merits. Whether they like it and whether they think it'll make money. And competition isn't just between fantasy books - more gritty, less shiny, it's at a much more general level - we publish more fantasy this year, fewer thrillers. You're getting more gritty books not because they're stealing slots from shiny books, but because gritty books are selling. Still plenty of more traditional stuff being published, and lots of the old stuff still in print.
Maybe the word "slots" is misleading here, but the concept exactly mirrors my experience in the industry. Houses rely on the small but steady flow of income from the backlist and use the sales of one new release to fund the production costs of the next, basically. Granted, my experience is not from big houses, so they may have unlimited editors and designers to put onto projects on a whim, but smaller houses have to schedule their time, their employees' time, their freelancers' time, and their money.
No, a marvelous book won't be rejected because there's no immediate slot available, but if there's no decent spot in the next few years or so, they probably will pass. Typically houses won't pull editors and designers from other projects to rush something through. Sure, it happens, but it's expensive and causes delays elsewhere. So you have houses with slots, basically. They can produce only so many books a year due to available funds, staff, etc and if a great ms. comes in over the transom they'll consider it for the schedule in the next few years, but are unlikely to throw money away shuffling things around frantically to publish it now. And less likely to sit on a ms. for years on end because they like it. Sure, it happens, but it's the exception rather than the rule.
Not to mention solicitations, retail release dates, review copies, blurbs, arcs, collateral, scheduling printing time, shipping time, and all the other time commitments publishers have to make well in advance of a book actually being released. My experience is clearly different than yours, but I've never seen a house drop everything they've worked on for a year or more to rush some new book out the door. It's always been more of a slow churn of one project to the next, or overlapping projects, all with relatively tight schedules stretching out months, if not years into the future.
Except for proofs. Damn I hate proofs.