A food-tracking AI system designed to combat malnutrition and improve overall health in long-term care homes automatically records and monitors how much food residents consume.
It is estimated that more than half of residents of long-term care homes are either malnourished or at risk of malnutrition. Now, using artificial intelligence software to analyze photos of plates of food after residents have eaten, a smart system can examine colour, depth, and other photo features to estimate how much of each kind of food has been consumed and calculate its nutritional value. It uses sophisticated software developed by researchers working at the University of Waterloo, Schlegel-UW Research Institute for Aging and the University Health Network, is just one of many new initiatives coming from the group.
Who really knows?
“Right now, there is no way to tell whether a resident ate only their protein or only their carbohydrates,” said Kaylen Pfisterer, who co-led the research while earning a PhD in systems design engineering at Waterloo. “Our system is linked to recipes at the long-term care home and, using artificial intelligence, keeps track of how much of each food was eaten to make sure residents are meeting their specific nutrient requirements.”
Food intake is now primarily monitored by staff who manually record estimates of consumption by looking at plates once residents have finished eating. Robert Amelard, a Waterloo alumnus and postdoctoral fellow at University Health Network, said studies show the subjectivity of that process results in an error rate of 50 per cent or more. By comparison, the automated system is accurate to within five per cent, “providing fine-grained information on consumption patterns.”
Collaboration is key
Researchers collaborated with personal support workers, dietitians and other long-term care workers to develop the system, which saves time as well as improves accuracy and would ideally be added to tablet computers already used by front-line staff to keep electronic records. “My vision would be to monitor and leverage any changes in food intake trends as yellow or red flags for the health status of residents more generally and for monitoring infection control,” said Pfisterer, now a scientific associate at the University Health Network Centre for Global eHealth Innovation.
The research team also included Heather Keller, a professor of kinesiology and health sciences, Alexander Wong, a systems design engineering professor, and students Audrey Chung, Braeden Syrnyk and Alexander MacLean.
Depth refined semantic segmentation
A paper on the teams work, “Automated food intake tracking requires depth-refined semantic segmentation to rectify visual-volume discordance in long-term care homes”, appears in the journal Scientific Reports.
Visit uwaterloo.ca.
