Hey Gary,
The nuclear sub's achilles heels are it thermal footprint - the reactor generates significant heat that it just can't hide, and it's noiser than a convential sub.
But after a bit of head scratching, I couldn't think of any way of a satellite to take advantage of either as the ocean is an effective blanket.
I see that modern Bathymetry is done by airbourne systems and use a LIDAR (or LADAR) system, bouncing wavelengths up and back again like a sonar system - but again may be again a step too far to contemplate for a satellite system. Plus you'd have to be constantly scanning a big area to see if something is moving real time and changing the effective depth/response of the system.
Finally satellites do their own bathymetric by measuring sea-level changes caused by the gravitational pull of masses under the sea...but to spot a sub and the tiny, tiny effect it would have...nah!
----
No, the only "sensible" SF option I can think of is to install a very sensitive neutrino detector in your satellite that can pick up the large flux of neutrinos pouring out of the reactor and flying unimpeded through shielding and the water.
Actually we're in the realm of near-future possiblity as I read somewhere that someone has built the first communication system that uses neutrinos as the carrier. Ok putting it into a satellite and making it scan the ocean, tough.
Actually, hell
, why even put the detection system on a satellite? you can have it in a big network on the ground - easier to maintain and you can make it as big and complex as you desire, without the costly expense of sending something up into space. duh!