A Conceptual Framework for Human-AI Collaborative Genome Annotation

TL;DR

This paper addresses the limited scalability and accuracy of automated genome annotation (GA) when used in isolation, highlighting the crucial role of manual curation. It proposes HAICoGA, a conceptual framework that codifies a sustained human-AI collaborative workflow for GA, integrating seven key elements (humans, AI tools, data, goals, interfaces, environment, collaboration) and a bi-directional feedback loop to continuously improve annotations. Through a survey of LLM-based AI assistants in biology, the authors illustrate a path toward multi-agent systems that can manage GA tasks, with a vision for a manager-critiqued, hybrid workflow that includes specialized manual-curation agents. They also discuss significant challenges—architectural design, novel ML methods, multi-dimensional evaluation, and user-centered interfaces—providing a roadmap for developing scalable, accurate GA in real-world settings.

Abstract

Genome annotation is essential for understanding the functional elements within genomes. While automated methods are indispensable for processing large-scale genomic data, they often face challenges in accurately predicting gene structures and functions. Consequently, manual curation by domain experts remains crucial for validating and refining these predictions. These combined outcomes from automated tools and manual curation highlight the importance of integrating human expertise with AI capabilities to improve both the accuracy and efficiency of genome annotation. However, the manual curation process is inherently labor-intensive and time-consuming, making it difficult to scale for large datasets. To address these challenges, we propose a conceptual framework, Human-AI Collaborative Genome Annotation (HAICoGA), which leverages the synergistic partnership between humans and artificial intelligence to enhance human capabilities and accelerate the genome annotation process. Additionally, we explore the potential of integrating Large Language Models (LLMs) into this framework to support and augment specific tasks. Finally, we discuss emerging challenges and outline open research questions to guide further exploration in this area.

Figures (3)

• Figure 1: Key elements in human-AI collaborative genome annotation. Humans and AI work together as a team to perceive the environment in which they operate. To achieve the shared high-level goal, they decompose the task into sub-tasks and objectives. Through the human-machine interface, humans and AI utilize the available data to carry out tasks and transition to a new state within the environment. This state may lead to updates in the list of tasks and goals or the addition of new data until the final goal is achieved. The collaboration between humans and AI is dynamic, allowing them to perform individual tasks independently while collaborating on shared tasks when necessary.
• Figure 2: Key competencies for fostering effective for human-AI collaboration. SA: situational awareness.
• Figure 3: (A) Overall multi-agent system design for human-AI collaborative genome annotation. Users submit a genome annotation query through an interactive user interface (UI). The UI requests the manager agent to analyze the task, decompose it into subtasks, and assign them to appropriate agents. While assisting with a subtask, an agent may request additional input from the user to complete the task successfully. The critique agent provides feedback on the outcomes, guiding the system's next steps. The manager agent monitors the global conversation history and intermediate results, updating the task plan as needed or finalizing the task and delivering the results to the user. (B) The top synergy layer of the multi-agent system designed for HAICoGA. Following the practical GA workflow biology9090295, the multi-agent system consists of a user, a manager agent, an automated GA agent, multiple manual curation agents, and a critique agent. (C) Workflow of multi-agent collaboration in automated GA phase. The manager agent delegates the automated GA task to the automated GA agent, which manages a customized pipeline (or an AI model) using genome data to perform specific tasks. The critique agent analyzes the results, evaluates their quality, and suggests the next steps to the manager agent. This process can be repeated iteratively until the desired outcome is achieved. (D) Workflow of multi-agent collaboration in manual curation phase. A manual annotation process follows the automated GA phase. Due to the complexity of manual curation, the system includes several specialized agents performing distinct roles. The sequence search agent identifies homologous genes for a target gene, for example, by running BLAST against genome sequence data. The database agent retrieves gene function annotations from various databases. The literature search agent identifies relevant scientific papers for further analysis, while the document summarization agent extracts key information from these papers. The synthesis agent compiles all relevant data and submits it to the critique agent, which reviews the information and provides suggestions, such as whether the data is sufficient to address the user's query. Finally, the manager agent either updates the task plan or generates the final response.