FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination

TL;DR

FoodSky is a Chinese food-oriented LLM that addresses the gap of domain-specific culinary and dietary reasoning. It builds a large-scale Chinese food corpus (FoodEarth) and employs a Topic-based Selective State Space Model (TS3M) together with Hierarchical Topic Retrieval Augmented Generation (HTRAG) to capture fine-grained food semantics and provide context-aware text. Through extensive experiments on chef and dietetic exams (CDE) and food QA benchmarks, FoodSky-13B achieves state-of-the-art zero-shot and few-shot performance, outperforming general-purpose LLMs and demonstrating strong domain understanding and generation. The work highlights the importance of data quality, topic-aware modeling, and retrieval-augmented generation for practical food intelligence, with potential for future reinforcement learning and multimodal extensions.

Abstract

Food is foundational to human life, serving not only as a source of nourishment but also as a cornerstone of cultural identity and social interaction. As the complexity of global dietary needs and preferences grows, food intelligence is needed to enable food perception and reasoning for various tasks, ranging from recipe generation and dietary recommendation to diet-disease correlation discovery and understanding. Towards this goal, for powerful capabilities across various domains and tasks in Large Language Models (LLMs), we introduce Food-oriented LLM FoodSky to comprehend food data through perception and reasoning. Considering the complexity and typicality of Chinese cuisine, we first construct one comprehensive Chinese food corpus FoodEarth from various authoritative sources, which can be leveraged by FoodSky to achieve deep understanding of food-related data. We then propose Topic-based Selective State Space Model (TS3M) and the Hierarchical Topic Retrieval Augmented Generation (HTRAG) mechanism to enhance FoodSky in capturing fine-grained food semantics and generating context-aware food-relevant text, respectively. Our extensive evaluations demonstrate that FoodSky significantly outperforms general-purpose LLMs in both chef and dietetic examinations, with an accuracy of 67.2% and 66.4% on the Chinese National Chef Exam and the National Dietetic Exam, respectively. FoodSky not only promises to enhance culinary creativity and promote healthier eating patterns, but also sets a new standard for domain-specific LLMs that address complex real-world issues in the food domain. An online demonstration of FoodSky is available at http://222.92.101.211:8200.

Figures (19)

• Figure 1: The potential applications of the proposed food-oriented LLM FoodSky in different scenarios.
• Figure 2: The pipeline of establishing FoodEarth includes data source sorting, data annotation and similarity-based data screening. The semi-automated data filtering and annotation are the main procedures in data annotation. The AHC refers to the Agglomerative Hierarchical Clustering approach.
• Figure 3: The illustration of authoritative data sources of our FoodEarth.
• Figure 4: The prompt used to automatically filter raw data based on ChatGPT in our semi-automated data filtering procedures.
• Figure 5: Hierarchical structure of topics in our FoodEarth dataset.