Factored Reasoning with Inner Speech and Persistent Memory for Evidence-Grounded Human-Robot Interaction

TL;DR

JANUS tackles long-horizon, evidence-grounded human-robot interaction by introducing a factored, memory-augmented cognitive architecture. It separates internal control (inner speech) from external communication (outer speech) and models dialogue as a POMDP with a dedicated memory agent that maintains bounded working contexts and archival knowledge, enabling auditable grounding and containment of memory growth. The architecture is realized via a modular ROS 2 pipeline with a domain customization layer and adapters for diverse backends, and is validated through extensive module-level unit tests in a dietary-assistance domain, demonstrating robust intermediate-variable accuracy and favorable latency profiles. The work highlights the importance of explicit sufficiency and grounding gates, controlled memory consolidation, and separation of decision discourse from human-facing explanations for scalable, trustworthy robot assistance over extended interactions.

Abstract

Dialogue-based human-robot interaction requires robot cognitive assistants to maintain persistent user context, recover from underspecified requests, and ground responses in external evidence, while keeping intermediate decisions verifiable. In this paper we introduce JANUS, a cognitive architecture for assistive robots that models interaction as a partially observable Markov decision process and realizes control as a factored controller with typed interfaces. To this aim, Janus (i) decomposes the overall behavior into specialized modules, related to scope detection, intent recognition, memory, inner speech, query generation, and outer speech, and (ii) exposes explicit policies for information sufficiency, execution readiness, and tool grounding. A dedicated memory agent maintains a bounded recent-history buffer, a compact core memory, and an archival store with semantic retrieval, coupled through controlled consolidation and revision policies. Models inspired by the notion of inner speech in cognitive theories provide a control-oriented internal textual flow that validates parameter completeness and triggers clarification before grounding, while a faithfulness constraint ties robot-to-human claims to an evidence bundle combining working context and retrieved tool outputs. We evaluate JANUS through module-level unit tests in a dietary assistance domain grounded on a knowledge graph, reporting high agreement with curated references and practical latency profiles. These results support factored reasoning as a promising path to scalable, auditable, and evidence-grounded robot assistance over extended interaction horizons.

Figures (8)

• Figure 1: Overview of JANUS as a classical Sense-Plan-Act pipeline. The Sense layer acquires and processes input from the human via a series of perception modules. The Plan layer integrates reasoning modules, inner speech, domain databases for retrieval and query grounding, as well as a dedicated memory agent that maintains a persistent interaction context. The Act layer externalizes decisions through outer speech for communications with the human, and through the robot controller for action execution.
• Figure 2: Overview of JANUS as a factored reasoning and control architecture. A Perception module forwards the input collected by the human to a set of Reasoning modules. Among them, Intent Recognition and Intent Post-processing generate a structured task representation from the user intent, whereas Scope Detection estimates the active interaction domain. If needed, Query Generation and database adapters acquire external information. This process is supervised by Inner Speech, which performs execution-readiness checks. The Memory agent provides persistent context and a bounded working context for the current turn-taking step, while Outer Speech generates responses for the human, and may interface with the robot controller for the execution of embodied actions. A runtime Customization Layer module binds domain-specific task schemas, tools, plugins, and prompt assets without modifying the core pipeline.
• Figure 3: A view of the Memory agent as a controlled retrieval-update subsystem. At each turn, the agent (on the left hand side) constructs a bounded working context $W_t$ from the memory state $S_t = \langle H_t, C_t, A_t \rangle$ via the information-sufficiency gate $s_t$ of Equation \ref{['eq:suffgate']}, and the optional archival retrieval. The resulting $W_t$ is consumed by upstream modules, for example, Inner Speech and Query Generation, and contributes to the evidence bundle $B_t = (W_t, E_t)$. At the end of the turn (on the right hand side), the agent applies a managed update operator $\mathcal{U}$, as described in Equation \ref{['eq:memupdate']}, inserting new traces into $H_t$ and triggering capacity-based consolidation and revision into $C_t$ and $A_t$.
• Figure 4: A complete, example interaction trace for a single turn in the Advisor domain. The human asks "How many carbohydrates are in tiramisu?". Intent Recognition maps the utterance to the DishInfo schema $\tau_t$ and extracts a typed parameter assignment $\theta_t$ containing the dish identifier, that is, tiramisu. Inner Speech evaluates execution readiness, and issues a control decision $c_t = \textsc{Proceed}$, together with the tool-grounding decision $\rho_t = 1$. Query Generation produces an executable query specification $q_t$, which is executed against the domain backend to obtain evidence $E_t$ about the dish's macronutrient composition. Then, Outer Speech transforms the evidence bundle $B_t = (W_t, E_t)$ into an evidence-grounded natural-language reply communicating the retrieved carbohydrates value to the human.
• Figure 5: Inter-domain semantic similarity used in the Scope Detection study. The heatmap reports the pairwise cosine similarity between the textual domain descriptions in Table \ref{['tab:scope_domains']}, including the OutOfScope fallback. Darker cells indicate higher similarity, that is, greater semantic overlap, whereas lighter cells indicate lower similarity. The matrix is shown in triangular form for the sake of readability.