Optimizing Life Sciences Agents in Real-Time using Reinforcement Learning
Nihir Chadderwala
TL;DR
This paper tackles adaptive decision-making for AI agents in life sciences by formulating agent optimization as a contextual bandit problem and solving it with Thompson Sampling. It optimizes three interrelated decisions—generation strategy, tool selection, and domain routing—using user feedback as the sole signal, implemented within AWS Strands Agents. The approach yields 15-30% gains in user satisfaction over random baselines with rapid convergence (20-30 queries) and operates without ground-truth labels, addressing non-stationarity and multi-objective trade-offs. The work provides practical guidance, an open-source implementation, and a framework applicable to other high-stakes, knowledge-intensive domains.
Abstract
Generative AI agents in life sciences face a critical challenge: determining the optimal approach for diverse queries ranging from simple factoid questions to complex mechanistic reasoning. Traditional methods rely on fixed rules or expensive labeled training data, neither of which adapts to changing conditions or user preferences. We present a novel framework that combines AWS Strands Agents with Thompson Sampling contextual bandits to enable AI agents to learn optimal decision-making strategies from user feedback alone. Our system optimizes three key dimensions: generation strategy selection (direct vs. chain-of-thought), tool selection (literature search, drug databases, etc.), and domain routing (pharmacology, molecular biology, clinical specialists). Through empirical evaluation on life science queries, we demonstrate 15-30\% improvement in user satisfaction compared to random baselines, with clear learning patterns emerging after 20-30 queries. Our approach requires no ground truth labels, adapts continuously to user preferences, and provides a principled solution to the exploration-exploitation dilemma in agentic AI systems.