And that's what most modern stories seem to ONLY be focused on. What genre people will like and then just have some characters they can relate too. Therefore there's no need to put much effort in the world because this story is nothing more than entertainment toward an audience. It has no identity. Its bound to whatever the audience wants.
Have you thought that maybe the marketability of a book affects how likely you are to come across it. The less a book conforms to a genre, the harder it is for a publisher to market it and therefore the harder it is to find. Even algorithms are geared up to recommending something similar to something you already like, rather than something
novel.
The types of innovative fiction that tends to get through is often literary works such as Murakami, Cixin Liu, etc. You need to find the spaces and marketing outlets that push literary fiction if that's your interest.
When I watched Dune this month, I was thinking: "Wow! I haven't watch a movie like that in a long time." That is because Dune was more than just a story that picked a random genre that the mass audience would like. It chose to be sci-fi because that would give it its identity. That would help its world to be more immersive. Lord of the Rings would not have worked if it was Sci-fi. And that's okay. Those kinds of stories stand on their own with their own identities. It seems the mass audience would rather see a jumbled up genre like the superhero movies rather than something that doesn't appeal to them because of its chosen identity. When a story chooses a personality of an audience to represent, it is bound to that audiences entrainment. When a story choses a world for its identity, its more free to be itself and be a more immersive experience.
Which leads lastly to why I respectfully disagree with your statement: "Personality IS variety." Think about it, if there were two books/movies that both had sci-fi and fantasy elements, they would basically be the same story. But if the two movies were a fantasy and sci-fi story. There are two different movies unlike that seemingly collective hivemind of clones of sci-fi/fantasy stories.
Identity is variety.
Why is personality, variety? Look at the dictionary definition: "the combination of characteristics or qualities that form an individual's distinctive character" - keyword is
distinctive. We often say things or people that conform to mass-market corporate standards have no personality. With personality we take individual traits in combination and these add up to a unique entity.
Identity isn't variety, rather it does the opposite of personality. Identity is about
belonging and
identifying - i.e. reducing a unique combination of traits into categories so that we can identify them and make assumptions about them. Assigning something an identity is reducing and limiting it to something that is easily definable. Identities help other people to make decisions about the things grouped together. You see this in marketing when marketers talk about
identifying certain audience groups. If we identify someone we reference surface characteristics - tall, wide, thin, short, black, white, blonde, brown etc. When we talk about our identity, we are often talking about fitting in with a group of some kind, even if those identity markers are combined to form an
identity matrix - they are saying "
where do I belong" rather than "
who am I?"
You seem to be saying that world building is more interesting than plot or character. That's fine if that's your thing.
I have a friend who used to write fighting fantasy books, and he decries world building as
mere geekery. These are articles he's posted against worldbuilding as the bench mark of fiction:
Would-be writers of fantasy novels see the example of Tolkien and imagine writing a book is a lot like playing “The Sims” — you just build a world, drop some people in it, and see what happens. That just doesn’t work. You might object, “But some authors don’t know how their plot will end!” That’s true, but such authors create rich characters first. You can have a plot-driven story or you can have a character-driven story, but you can’t have a world-driven story. Look at fantasy authors who aren’t Tolkien, and one thing you’ll notice is that maps tend to be drawn by fans, or retconned after the fact. First, comes the story, then they map the world around the story.
"Worldbuilding!" You read it all the time in reviews. "A triumph of worldbuilding." "A masterpiece of worldbuilding." "…
www.gamesindustry.biz
In the article Fallen London creator Alexis Kennedy quotes Tolkien's own opinions on world building, which are character-first, rather than world first.
The wider point is why, nowadays, are there not more all encompassing world creations, possibly because creating them to the level of Tolkien takes decades, requires the linguistic skill of an oxford Don and most writers don't have lucrative jobs that give them the economic support to write 500,000 word trilogies. Today's market cannot support this level of indulgence.