Collaborative Intelligence in Sequential Experiments: A Human-in-the-Loop Framework for Drug Discovery

TL;DR

This work tackles the challenge of discovering target molecules in drug discovery under limited experimental budgets by introducing a human-in-the-loop framework that combines transformer-based sequence embeddings, Bayesian neural networks with uncertainty quantification, and expert oversight. The method generates two streams of recommendations per round—one to improve the model (uncertainty-driven) and one to identify promising candidates (search-driven)—with human experts ultimately selecting $B$ molecules to test and update the model. Empirical results on a real therapeutic peptide dataset show the approach consistently outperforms baselines, achieving substantially higher final hit rates and recall early in the search, demonstrating strong complementarity between human knowledge and AI inference. The findings underscore the value of meta-knowledge and task delegation strategies, suggesting practical path toward accelerated vaccine and drug development through human-AI collaboration under realistic constraints.

Abstract

Drug discovery is a complex process that involves sequentially screening and examining a vast array of molecules to identify those with the target properties. This process, also referred to as sequential experimentation, faces challenges due to the vast search space, the rarity of target molecules, and constraints imposed by limited data and experimental budgets. To address these challenges, we introduce a human-in-the-loop framework for sequential experiments in drug discovery. This collaborative approach combines human expert knowledge with deep learning algorithms, enhancing the discovery of target molecules within a specified experimental budget. The proposed algorithm processes experimental data to recommend both promising molecules and those that could improve its performance to human experts. Human experts retain the final decision-making authority based on these recommendations and their domain expertise, including the ability to override algorithmic recommendations. We applied our method to drug discovery tasks using real-world data and found that it consistently outperforms all baseline methods, including those which rely solely on human or algorithmic input. This demonstrates the complementarity between human experts and the algorithm. Our results provide key insights into the levels of humans' domain knowledge, the importance of meta-knowledge, and effective work delegation strategies. Our findings suggest that such a framework can significantly accelerate the development of new vaccines and drugs by leveraging the best of both human and artificial intelligence.

Figures (9)

• Figure 1: Framework of Human-in-the-Loop Sequential Experiment in Drug Discovery.
• Figure 2: Diagram of the ESM2 Transformer Architecture.
• Figure 3: Encoding Peptide Molecules with ESM2 and Predicting with Bayesian Neural Networks.
• Figure 4: Model Uncertainty and Data Uncertainty.
• Figure 5: Results of Different Policies Searching for Target Drug Molecules.

Theorems & Definitions (1)

• Example 1