Medication counseling with large language models: balancing flexibility and rigidity

TL;DR

The paper addresses safe, reliable medication counseling using LLMs by balancing flexibility and rigidity in dialog; it targets long, focused counseling for emergency contraceptive pills. It proposes a modular multi-agent system with three ReAct-enabled agents (Conversationalist, Medicine Interpreter, Symptom Assessor) coordinated via a graph-based architecture, a YAML-driven specification, and a retrieval-augmented data flow. Key contributions include a proof-of-concept that increases determinism while preserving conversational dynamics, explicit term-relabeling and similarity tooling for contraindications, and an evaluation framework centered on Human Interaction Evaluations plus ethical considerations. The work offers design patterns for reliable, human-in-the-loop AI in pharmacy contexts and provides guidance for applying these insights to other medications and longer, high-stakes conversations.

Abstract

The introduction of large language models (LLMs) has greatly enhanced the capabilities of software agents. Instead of relying on rule-based interactions, agents can now interact in flexible ways akin to humans. However, this flexibility quickly becomes a problem in fields where errors can be disastrous, such as in a pharmacy context, but the opposite also holds true; a system that is too inflexible will also lead to errors, as it can become too rigid to handle situations that are not accounted for. Work using LLMs in a pharmacy context have adopted a wide scope, accounting for many different medications in brief interactions -- our strategy is the opposite: focus on a more narrow and long task. This not only enables a greater understanding of the task at hand, but also provides insight into what challenges are present in an interaction of longer nature. The main challenge, however, remains the same for a narrow and wide system: it needs to strike a balance between adherence to conversational requirements and flexibility. In an effort to strike such a balance, we present a prototype system meant to provide medication counseling while juggling these two extremes. We also cover our design in constructing such a system, with a focus on methods aiming to fulfill conversation requirements, reduce hallucinations and promote high-quality responses. The methods used have the potential to increase the determinism of the system, while simultaneously not removing the dynamic conversational abilities granted by the usage of LLMs. However, a great deal of work remains ahead, and the development of this kind of system needs to involve continuous testing and a human-in-the-loop. It should also be evaluated outside of commonly used benchmarks for LLMs, as these do not adequately capture the complexities of this kind of conversational system.

Figures (2)

• Figure 1: Graph representation of the system. Optional paths are dashed lines. Mandatory paths are solid lines. Agents are ellipse-shaped, and tools are box-shaped.
• Figure 2: Structure of the conversation specification steps