Only book that really disturbed me I recall was a single instance in the stephen King short story The Jaunt. The story itself was a good horror use of a familiar sci-fi trope, as a family are taking a trip via teleportation to Mars, and the father tells the youngest son about the discovery of teleportation technology, omitting bits that he doesn't want him to hear. Without spoilers, the father recalls as he tells the story that at a midway point in development of the technology, a drunk and troubled scientist ties up his cheating ex-wife and forces her into the teleporter, screaming at what she knows is going to happen.
He then sabotages the device's process and teleports her. What happens to her is maybe not what you'd expect. The story is not exactly hard sci-fi in tone, and I have no idea as a layman in theoretical quantum physics whether something like the result is possible, but physics is a pretty abstract field, for all I know it could be. And the very idea of what happens I found pretty horrifying, not least as it is just a background detail in the story, and is not dwelled upon, but in the story would be continuously ongoing, indefinitely.
I don't generally find supernatural stuff scary, even in movies, but that really did it for me, and I thought of what H.P. Lovecraft apparently said in a letter, that he wrote intending to create fiction that would frighten people of scientific perspective (as opposed to traditional horror based upon superstition or the supernatural).
