Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sensorium Arc: AI Agent System for Oceanic Data Exploration and Interactive Eco-Art

Noah Bissell, Ethan Paley, Joshua Harrison, Juliano Calil, Myungin Lee

TL;DR

The paper tackles the challenge of making ocean climate data accessible and emotionally resonant to the public. It introduces Sensorium Arc, a multimodal AI agent system that personifies the ocean through a modular multi-agent LLM pipeline grounded in retrieval-augmented generation and embodied via a Nautilus-inspired interface. Key contributions include the architecture (Visualization Decider, Query Rewriter, Responder), integration of NASA EarthData visuals and the Harrison archive, and an end-to-end pipeline that yields synchronized audiovisual outputs. This work proposes a new paradigm for human–machine–ecosystem interaction, enabling affective inquiry and co-authorship with non-human agents to foster ecological understanding and imagination.

Abstract

Sensorium Arc (AI reflects on climate) is a real-time multimodal interactive AI agent system that personifies the ocean as a poetic speaker and guides users through immersive explorations of complex marine data. Built on a modular multi-agent system and retrieval-augmented large language model (LLM) framework, Sensorium enables natural spoken conversations with AI agents that embodies the ocean's perspective, generating responses that blend scientific insight with ecological poetics. Through keyword detection and semantic parsing, the system dynamically triggers data visualizations and audiovisual playback based on time, location, and thematic cues drawn from the dialogue. Developed in collaboration with the Center for the Study of the Force Majeure and inspired by the eco-aesthetic philosophy of Newton Harrison, Sensorium Arc reimagines ocean data not as an abstract dataset but as a living narrative. The project demonstrates the potential of conversational AI agents to mediate affective, intuitive access to high-dimensional environmental data and proposes a new paradigm for human-machine-ecosystem.

Sensorium Arc: AI Agent System for Oceanic Data Exploration and Interactive Eco-Art

TL;DR

The paper tackles the challenge of making ocean climate data accessible and emotionally resonant to the public. It introduces Sensorium Arc, a multimodal AI agent system that personifies the ocean through a modular multi-agent LLM pipeline grounded in retrieval-augmented generation and embodied via a Nautilus-inspired interface. Key contributions include the architecture (Visualization Decider, Query Rewriter, Responder), integration of NASA EarthData visuals and the Harrison archive, and an end-to-end pipeline that yields synchronized audiovisual outputs. This work proposes a new paradigm for human–machine–ecosystem interaction, enabling affective inquiry and co-authorship with non-human agents to foster ecological understanding and imagination.

Abstract

Sensorium Arc (AI reflects on climate) is a real-time multimodal interactive AI agent system that personifies the ocean as a poetic speaker and guides users through immersive explorations of complex marine data. Built on a modular multi-agent system and retrieval-augmented large language model (LLM) framework, Sensorium enables natural spoken conversations with AI agents that embodies the ocean's perspective, generating responses that blend scientific insight with ecological poetics. Through keyword detection and semantic parsing, the system dynamically triggers data visualizations and audiovisual playback based on time, location, and thematic cues drawn from the dialogue. Developed in collaboration with the Center for the Study of the Force Majeure and inspired by the eco-aesthetic philosophy of Newton Harrison, Sensorium Arc reimagines ocean data not as an abstract dataset but as a living narrative. The project demonstrates the potential of conversational AI agents to mediate affective, intuitive access to high-dimensional environmental data and proposes a new paradigm for human-machine-ecosystem.

Paper Structure

This paper contains 5 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Discussion
  5. Conclusion

Figures (3)

  • Figure 1: a) Exemplary setup for the Sensorium Arc exhibition b) User interface before activation showing the sensors c) Activated user interface showing the response from the system. Users engage through a natural gesture, whispering into a seashell as if speaking to the ocean.
  • Figure 2: Overall diagram of the Sensorium Arc system
  • Figure 3: Screenshots of the different exemplary scenarios (a) Default globe view before user interaction. (b) Chlorophyll concentration data visualized on the globe surface. (c) Water clarity rendered using 16 stacked layers of hyperspectral diffuse attenuation coefficient (Kd) data from NASA's PACE satellite (2024). (d) Combined visualization of atmospheric CO2 levels and ocean surface wind flow, highlighting multimodal environmental interactions.