AUTHOR=Kok Esther , Chauhan Aneesh , Tufano Michele , Feskens Edith , Camps Guido 

TITLE=The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches fillings

JOURNAL=Frontiers in Nutrition

VOLUME=Volume 11 - 2024

YEAR=2025

URL=https://www.frontiersin.org/journals/nutrition/articles/10.3389/fnut.2024.1520674

DOI=10.3389/fnut.2024.1520674

ISSN=2296-861X

ABSTRACT=IntroductionAccurate measurement of dietary intake without interfering in natural eating habits is a long-standing problem in nutritional epidemiology. We explored the applicability of hyperspectral imaging and machine learning for dietary assessment of home-prepared meals, by building a proof-of-concept, which automatically detects food ingredients inside closed sandwiches.MethodsIndividual spectra were selected from 24 hyperspectral images of assembled closed sandwiches, measured in a spectral range of 1116.14 nm to 1670.62 nm over 108 bands, pre-processed with Standard Normal Variate filtering, derivatives, and subsampling, and fed into multiple algorithms, among which PLS-DA, multiple classifiers, and a simple neural network.ResultsThe resulting best performing models had an accuracy score of ~80% for predicting type of bread, ~60% for butter, and ~ 28% for filling type. We see that the main struggle in predicting the fillings lies with the spreadable fillings, meaning the model may be focusing on structural aspects and not nutritional composition.DiscussionFurther analysis on non-homogeneous mixed food items, using computer vision techniques, will contribute toward a generalizable system. While there are still significant technical challenges to overcome before such a system can be routinely implemented in studies of free-living subjects, we believe it holds promise as a future tool for nutrition research and population intake monitoring.