The bolded was my primary inkling. I assumed it was so and it explained some actions but, I never really considered it a big deal in the books. I actually didn't connect the Rainbow Guard bit until...just now
R+L=J leaped out at me about 1/4 of the way into my first AGoT re-read which took place after reading through AFFC. I don't know how anyone could be expected to pick up on that one in their first read. The show is pushing Jon's mom as a much bigger issue right now and I
still expect most people who only have exposure through the show to miss it.
edit: I also don't think there's very much concerning Renly/Loras in the first book. It's not something you get many hints towards until book 2. In book 1 it's mainly just descriptions of how pretty both of them are and how well Renly dresses.