The Agentic Leash: Extracting Causal Feedback Fuzzy Cognitive Maps with LLMs

TL;DR

This work addresses extracting causal, dynamical knowledge from text by coupling LLM agents with fuzzy cognitive maps (FCMs) in an agentic framework. It introduces a three-step, guided extraction pipeline that yields nodes, refining nouns and nouns phrases from text into FCM variables, then derives weighted causal edges, with evidence quotes enabling transparent reasoning. The approach is demonstrated on Kissinger et al.'s AI article, showing FCMs that converge to equilibria similar to human-generated models, and further extended to mixed FCMs from multiple LLM agents, producing novel equilibria that better approximate underlying dynamics. The results suggest a scalable pathway for building explainable, text-derived causal knowledge networks whose global dynamics—via limit cycles and equilibria—inform what-if scenario analysis and potential policy or governance insights, while the mixing framework supports larger, more robust causal models.</p>

Abstract

We design a large-language-model (LLM) agent that extracts causal feedback fuzzy cognitive maps (FCMs) from raw text. The causal learning or extraction process is agentic both because of the LLM's semi-autonomy and because ultimately the FCM dynamical system's equilibria drive the LLM agents to fetch and process causal text. The fetched text can in principle modify the adaptive FCM causal structure and so modify the source of its quasi-autonomy--its equilibrium limit cycles and fixed-point attractors. This bidirectional process endows the evolving FCM dynamical system with a degree of autonomy while still staying on its agentic leash. We show in particular that a sequence of three finely tuned system instructions guide an LLM agent as it systematically extracts key nouns and noun phrases from text, as it extracts FCM concept nodes from among those nouns and noun phrases, and then as it extracts or infers partial or fuzzy causal edges between those FCM nodes. We test this FCM generation on a recent essay about the promise of AI from the late diplomat and political theorist Henry Kissinger and his colleagues. This three-step process produced FCM dynamical systems that converged to the same equilibrium limit cycles as did the human-generated FCMs even though the human-generated FCM differed in the number of nodes and edges. A final FCM mixed generated FCMs from separate Gemini and ChatGPT LLM agents. The mixed FCM absorbed the equilibria of its dominant mixture component but also created new equilibria of its own to better approximate the underlying causal dynamical system.

Figures (15)

• Figure 1: A Large Language Model (LLM) extracts causal variables and their casual relationships out of a Wall Street Journal article from Henry Kissinger and colleagues about the promise of AI and then creates a Fuzzy Cognitive Map (FCM). The figure shows only 5 out of the 15 AI-extracted nodes and the directed weighted edges that connect them. The positive edges are in blue and the negative edges are in red. The figure highlights one of many feedback loops in the FCM in green. In this case: Growth of Human Cognition increases Human-AI Interactions but an increase in Human-AI Interactions decreases Human Cognition.
• Figure 2: A 15-node FCM extracted by the LLM from the WSJ article titled "ChatGPT Heralds an Intellectual Revolution" by Henry Kissinger et al. This FCM converges to a 4-step limit cycle. The limit cycle predicts that the growth of generative AI comes in waves or cycles. In the $1^{\text{st}}$ step generative AI gets widely used, human-AI interactions rise, and human knowledge grows. This helps leaders govern ethically in the $2^{\text{nd}}$ step but it also comes with risks and dangers. In the $3^{\text{rd}}$ step: People trust the mysterious and uncertain AI, misinformation and falsehoods spread, and society changes. But AI improves education at the same time and leads to scientific discoveries. Generative AI is used widely again in the $4^{\text{th}}$ step but this time without ethical leadership. Society changes again but this time without generative AI.
• Figure 3: LLM-based FCM-extraction from text: An LLM agent extracts FCM nodes and edges from a text input through a three-step systematic process. These steps use the same LLM agent with guiding system instructions. The LLM extracts all the nouns and noun phrases used in the text during Step 1. It extracts the FCM nodes from the list of nouns and noun phrases in Step 2. This involves refining the noun list from Step 1 based on associated qualitative or quantitative measures as well as the existence of causal links. The LLM extracts fuzzy-weighted edges from a list of node-pairs to complete the FCM in Step 3.
• Figure 4: Noun Extraction with an agentic LLM: LLM takes a paragraph of text as input and identifies the nouns, noun phrases, and pronouns present in the text. The figure on the right highlights the extracted nouns and noun phrases in red.
• Figure 5: FCM-node extraction with an agentic LLM: The LLM takes the list of nouns and noun phrases as input and filters those that are associated with some kind of qualitative or quantitative measure. This gives a list of 5 FCM nodes.