Leveraging Automatic Personalised Nutrition: Food Image Recognition Benchmark and Dataset based on Nutrition Taxonomy

TL;DR

The paper tackles the need for personalized nutrition by introducing AI4Food-NutritionDB, a nutrition taxonomy-enabled image database that fuses seven existing datasets into 558,676+ images across 6 nutritional levels, 19 categories, 73 subcategories, and 893 final products. It provides a standardized benchmark for category, subcategory, and final-product recognition using two CNN architectures (Xception and EfficientNetV2), with pre-trained models released for community use. Across intra- and inter-database evaluations, AI4Food-NutritionDB pre-training improves generalization, notably achieving strong category-level performance and substantial cross-d database transfer to VireoFood-251. This work lays a foundation for nutrition-aware food computing and personalized dietary guidance by enabling more accurate interpretation of food images and supporting interventions tailored to individual needs.

Abstract

Maintaining a healthy lifestyle has become increasingly challenging in today's sedentary society marked by poor eating habits. To address this issue, both national and international organisations have made numerous efforts to promote healthier diets and increased physical activity. However, implementing these recommendations in daily life can be difficult, as they are often generic and not tailored to individuals. This study presents the AI4Food-NutritionDB database, the first nutrition database that incorporates food images and a nutrition taxonomy based on recommendations by national and international health authorities. The database offers a multi-level categorisation, comprising 6 nutritional levels, 19 main categories (e.g., "Meat"), 73 subcategories (e.g., "White Meat"), and 893 specific food products (e.g., "Chicken"). The AI4Food-NutritionDB opens the doors to new food computing approaches in terms of food intake frequency, quality, and categorisation. Also, we present a standardised experimental protocol and benchmark including three tasks based on the nutrition taxonomy (i.e., category, subcategory, and final product recognition). These resources are available to the research community, including our deep learning models trained on AI4Food-NutritionDB, which can serve as pre-trained models, achieving accurate recognition results for challenging food image databases.

Figures (4)

• Figure 1: AI4Food nutritional food pyramid. Top levels (levels 1, 2, and 3) mean lower food intake frequency, whereas bottom levels (levels 4, 5, and 6) imply higher food intake frequency.
• Figure 2: Graphical diagram of the proposed study. First, we graphically show the AI4Food-NutritionDB database generation and the proposed nutrition taxonomy (i.e., nutritional level, category, subcategory, and food product). Second, the AI4Food-NutritionDB benchmark is presented, analysing the proposed nutrition taxonomy and scenarios (i.e., intra- and inter-database).
• Figure 3: Description of the AI4Food-NutritionDB food image database framework and taxonomy. This database is generated using food images from seven different state-of-the-art databases, i.e., UECFood-256 UECFood256, Food-101 Food101, Food-11 EPFL, FruitVeg-81 fruitveg81, MAFood-121 mafood, ISIA Food-500 ISIA, and VIPER-FoodNet viper. AI4Food-NutritionDB database comprises 6 nutritional levels, 19 main categories, 73 subcategories, and 893 final products with over 500K food images.
• Figure 4: Graphical representation of the categories and subcategories within the AI4Food-NutritionDB food image database. Note that the placement of the categories has been attempted to align with the pyramid in Fig. \ref{['fig:pyramid']}. As can be observed, this positioning generates ambiguities and discrepancies (e.g., for mixed and cooked food) that were resolved as described in Sec. \ref{['sub:3b']}.