Personally, I think it's important to set out, as close to the start as you can, what sort of book it's going to be (if it's going to be gritty and violent, frex, then you probably don't want to start with three chapters of fluffy kittens. Unless they are mauling crocodiles and guts are flying everywher). This is your promise to your reader. Now, if the focus isn't on the fantasy (it's only a small element) then you can get away with later on. But then, is it a fantasy? If fantasy is going to loom large in your plot, then I think you at least need to hint quite strongly that we aren't in Kansas any more, Toto.
There's more than one way to bring to magic to the front of the book (In Who Fears Death for example, the story is of a girl who only later finds out she has magical powers, so the story starts with the weird things that happen that are the first inkling, and then back tracks, setting up how she got there)
It doesn't have to be as overt as that, of course, but I think you need to give the flavour of the book quite soon. Be clear what sort of book it is. IMO, yada yada, it it works, do it etc etc