I don't find there to be a very strong magical element in Martin's "A Song of Ice and Fire." There definitely is one, but it takes a back seat to the characters and the story, and quite often I forget it's even part of the mythology.
So my first thought would be that, if it suits the story, there doesn't necessarily need to be a magical element.
This, and I should add that "magic" is really just an otherwise unexplainable phenomenon that, as stated above, seems to violate that which is accepted as the laws of physics and nature. Old men in robes throwing lightning bolts at each other, nine shadowy figures chanting a prayer of resurrection for dead god on a lightless night, the eerie lights that fill the sky every 16 days at midnight, or even something as mundane as the crops in a small town always yielding enormous, bountiful harvests with fruits and vegetables of unsurpassed quality, regardless of the year's weather patterns; All of these could constitute magic in some form or another.As has been argued in other threads, all a fantasy needs is a touch of the numinous, something that hints of a violation of natural law, either beneficent or malevolent (or indifferent); something set in a "fantasy realm" (if I'm understanding what you mean by that -- a more-or-less typical pre-Industrial Revolution sort of milieu) needs some hint of that to become a true fantasy, generally speaking, though there are some exceptions, such as Peake's Titus books; but magic per se certainly isn't required, as can be seen by any number of older fantasy works....