There’s nothing that can really prepare you for the rigors of being a new parent, like the crippling fatigue, but Caleb Olson has hacked together a clever way to avoid at least one challenge of having a newborn—the crying when hungry. In a move that’s sure to buy some extra ‘Zs, his new AI-powered webcam detects the signs of hunger before the tears begin to flow.
The last time we featured Olson’s work, he had come up with a way to leverage a backyard security camera to both detect when his dog had pooped, and then guide him to every deposit using a laser pointer attached to a robotic arm. It was over the top yet surprisingly effective, but since that time, Olson’s family has added another mouth to feed. And this mouth starts to loudly cry if their feedings are late. The solution, once again, was to use an AI model capable of recognizing specific facial and body poses so Olson could stay ahead of the tears.
Crying is actually considered a late-stage sign that a baby is hungry, and unfortunately when a newborn gets to the point that they’re physically upset as a result, it can make feedings more of a challenge. It’s better to feed a hungry infant before they cry, but it’s also nearly impossible to maintain a non-stop vigil for other signs of hunger when you’re struggling to balance all the chores that come with a newborn, plus exhaustion.
But most new parents also keep a camera pointed at a baby that livestreams video to devices like a smartphone, which Olson cleverly upgraded to autonomously watch for telltale signs of hunger. These include lip smacking (a repeated opening and closing of the mouth), rejecting a pacifier, rooting and moving the head back and forth in order to find a food source, and bringing fists towards the mouth.
Olson used Google’s MediaPipe machine learning pipeline to monitor the camera’s livestream and look for these movements and behaviors, but on their own, they don’t necessarily mean a newborn is ready for a meal. So Olson assigned these behaviors different levels of importance, and as they’re detected, they contribute to an overall score. Once the score passes a certain threshold, Olson’s smartphone gets a notification indicating that his baby is ready to be fed. Sometimes a pacifier accidentally falls out of a baby’s mouth, or a yawn may look like the beginning of lip smacking when it’s just a yawn.
The system isn’t 100% foolproof, and in the video, Olson reassures us that his family’s still prioritizing their pediatrician’s advice when it comes to feeding their newborn. But even if it doesn’t prevent crying all the time (there are still lots of reasons a baby will wail), the few times it can stave off the tears so Olson and his partner can get a few extra minutes of sleep makes it completely worth the effort.