That's happened to me sometimes, too! If I liked the movie very much, that's not a problem, it may even be a good thing, but if the movie was a disappointment, I hate that it changes the way I hear the characters.
It's even worse if the image you have of a character is obliterated by an actor who is nothing like the character you read. No matter how good the adaptation that character seems to develop a spilt personality in my head when I'm reading next time.