We've written about the Hollow Face Illusion before. It will get you every time. But in order to be fooled by something that shouldn't be, you need to know how things should be. There is a class of people who don't ever fall for the Hollow Face Illusion.
Small babies don't understand how the shapes they see translate into physical concepts, so they can't be fooled. This intrigued a group of scientists. They showed small babies, from 5-8 months, virtual representations of the Hollow Face Illusion. The children, as well as a group of adults, watched blocks with either a concave (hollow) face or a convex face spinning around on a screen. Later, the scientists allowed the groups to examine static images of both blocks, knowing that people tend to stare at strange phenomena that they can't figure out. Adults spent their time staring at the concave face, wondering how they'd been fooled. Small babies showed no preference whatsoever. They didn't even know what they were seeing. But 7-8-month-olds tended to look at the convex face on the block. They weren't interested in the hollow face.
Puzzled, the scientists did follow-up experiments. They knew that infants at this age prefer to stare at faces, and that they are beginning to perceive shading and depth. Targeting infants at seven months, they tried habituating the children to the concave or convex blocks, and seeing if children at that age simply preferred a convex face to a concave one. They found that while there was a little preference for a convex face, there wasn't any real indication that the children couldn't see the concave face as a face. The babies got used to both blocks equally, indicating they didn't see anything unusual about a block with a face carved into it - or think that they block was doing anything odd.
The tests are not conclusive, but they are interesting. Perhaps there was, roughly, about a month during which you were immune to this illusion. Once you learn the world a bit better, you are consistently fooled.