A Social Robot with Inner Speech for Dietary Guidance

TL;DR

The paper addresses the challenge of trust and explainability in social robots providing dietary guidance in healthcare. It proposes a cognitive architecture that uses inner speech generated by LLMs, both for internal planning and external explanation, integrated with a knowledge graph and symbolic reasoning within a Sense-Plan-Act loop. It implements a Pepper-based prototype with a Neo4j knowledge graph, Prolog/Clingo-based reasoning, and few-shot prompting, and reports latency analyses and a small user study showing improved interpretability when inner speech is exposed. The work demonstrates a practical path toward explainable, trustworthy robotic dietary advisors and identifies avenues to reduce latency and enhance social responsiveness for real-world healthcare HRI deployment.

Abstract

We explore the use of inner speech as a mechanism to enhance transparency and trust in social robots for dietary advice. In humans, inner speech structures thought processes and decision-making; in robotics, it improves explainability by making reasoning explicit. This is crucial in healthcare scenarios, where trust in robotic assistants depends on both accurate recommendations and human-like dialogue, which make interactions more natural and engaging. Building on this, we developed a social robot that provides dietary advice, and we provided the architecture with inner speech capabilities to validate user input, refine reasoning, and generate clear justifications. The system integrates large language models for natural language understanding and a knowledge graph for structured dietary information. By making decisions more transparent, our approach strengthens trust and improves human-robot interaction in healthcare. We validated this by measuring the computational efficiency of our architecture and conducting a small user study, which assessed the reliability of inner speech in explaining the robot's behavior.

Figures (4)

• Figure 1: Overview of the proposed cognitive architecture. The Perception Modules process sensory input and provide information to the Inner Speech Module, which enables self-reflection and contextual reasoning while maintaining a Short-Term Memory for temporary information retention. The Reasoning Modules integrate symbolic and stochastic reasoning to support decision-making. A separate Long-Term Memory stores persistent knowledge, which the reasoning process can access. Finally, the Outer Speech Module externalizes selected reasoning steps to enhance transparency, while the Robot Trajectory Controller executes physical actions.
• Figure 2: Overview of the proposed implementation. The Sense layer captures user input via text or a speech-to-text module, while the Plan layer coordinates dialogue management (Inner Speech), intent recognition, query generation, and logical reasoning through the knowledge graph and logical solver. Finally, the Act layer delivers responses to the user through Outer Speech and explanatory modules, closing the interaction loop.
• Figure 3: Time analysis for different system components and operations. The boxplot illustrates the time required (in seconds) for key processes, including intent recognition, inner speech, query generation, Clingo-based reasoning with varying number of items, and explainability mechanisms. The final three categories represent the full interaction time for dish information retrieval, user insertion, and meal preparation.
• Figure 4: The plot shows the average scores for each question of the questionnaires, reflecting participants' understanding of i) the robot's intent, ii) the required parameters, and iii) the completeness of the initial prompt. Questions 3, 13, and 20 were excluded from the analysis because metrics ii and iii cannot be applied to out-of-scope samples. The final column represents the overall average score.