Towards a Science Exocortex

Abstract

Artificial intelligence (AI) methods are poised to revolutionize intellectual work, with generative AI enabling automation of text analysis, text generation, and simple decision making or reasoning. The impact to science is only just beginning, but the opportunity is significant since scientific research relies fundamentally on extended chains of cognitive work. Here, we review the state of the art in agentic AI systems, and discuss how these methods could be extended to have even greater impact on science. We propose the development of an exocortex, a synthetic extension of a person's cognition. A science exocortex could be designed as a swarm of AI agents, with each agent individually streamlining specific researcher tasks, and whose inter-communication leads to emergent behavior that greatly extend the researcher's cognition and volition.

Figures (5)

• Figure 1: An exocortex seeks to augment human intelligence by connecting computation systems to a person. A science exocortex could be implemented as a swarm of specialized AI agents, operating on behalf of the human researcher, including agents for controlling experimental systems, for exploring data and synthesizing it into knowledge, and for exploring literature and ideation. The AI agents would connect to science components (instruments, databases, software, etc.) and streamline access. Crucially, the AI agents communicate with one another, working on tasks on behalf of the user and only surfacing the most important decisions and outputs for human consideration. If successful, such a system would allow researchers to handle the enormity of modern scientific knowledge, and accelerate discovery and dissemination of new science.
• Figure 2: Diagram of a large language model (LLM) acting as a kernel (image based on social media post by Andrej Karpathy). While LLMs perform text generation, Karpathy has proposed to view them instead as kernels---orchestration agents---of a new kind of operating system.karpathy2023llmoskarpathy2023llmos2 In this paradigm, the LLM is responsible for accessing resources (e.g. documents) or triggering actions (calculations, web browsing, etc.), and feeding results to a desired interface (e.g. chatbot dialog). The ability of LLMs to perform (rudimentary) decision-making can thus be exploited to coordinate more complex activity in response to relatively vague commands (which may come from a human or another LLM system).
• Figure 3: Knowledge mapping is an attempt to align and aggregate a variety of data sources about a particular scientific problem into a single model. One architecture for accomplishing this is shown. Available data is organized into signals of interest (such as physical measurables, material properties, or functional metrics). One typically has a variety of estimates or observations for a given signal, arising from different experiments, calculations, or theories. In principle these observations already map into a common space; in practice there are complex and often unknown disparities between the observations, owing to measurement errors, disparate definitions, or different assumptions. Thus, some non-linear transformation (e.g. accomplished using neural networks) is required to combine them into a single predictive model. Models for distinct signals can be cross-correlated to identify inter-relations; this can effectively combine the models into a single multi-modal model.
• Figure 4: Autonomous ideation aims for the AI agent to develop new scientific ideas (novel research directions, testable hypotheses, actionable research plans). One possible system design is to treat the task simialr to an autonomous experimentation loop, wherein one is exploring a multi-dimensional parameter space. In ideation, one can define the space of ideas using embedding vectors to position each idea. Each idea can be scored using an LLM ranking procedure. The loop consists of selecting a region for exploration (e.g. based on some combination of local sparsity, model error, and quality-maximization), generating ideas in that region (e.g. using an LLM provided with documents/ideas from the local neighborhood), and ranking the resultant ideas. As the loop proceeds, the space becomes populated with ideas. The top generations can eventually be presented to the human for consideration.
• Figure 5: A possible architecture for AI agents aggregating access to scientific facilities. Each researcher's exocortex could negotiate control by dialoging with a set of AI agents provided by the facilities. That layer of agents would be optimized to launch tasks using traditional software APIs. The underlying resources (measurement instruments, compute resources, databases) would be triggered and queried, with the outputs integrated first by the facility AI agents, and then by the researcher's exocortex.