InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation

TL;DR

InteractScience addresses the challenge of automatically evaluating LLM-generated interactive scientific demonstrations by introducing a hybrid framework that combines deterministic programmatic testing of interaction logic with visually-grounded qualitative checks. The benchmark comprises 150 problems across five scientific domains, each with an implementation plan, unit tests, reference snapshots, and VLM-based checklists, enabling end-to-end evaluation of both functional correctness and scientific fidelity. Evaluations across 30 models reveal that while models can reliably generate UI components and basic interactivity, achieving correct domain-specific behavior and faithful visualizations remains difficult, with perfect functional coverage being rare. The work provides a reusable, publicly available foundation for advancing reliable, educationally useful front-end code generation in science and education contexts.

Abstract

Large Language Models (LLMs) are increasingly capable of generating complete applications from natural language instructions, creating new opportunities in science and education. In these domains, interactive scientific demonstrations are particularly valuable for explaining concepts, supporting new teaching methods, and presenting research findings. Generating such demonstrations requires models to combine accurate scientific knowledge with the ability to implement interactive front-end code that behaves correctly and responds to user actions. This capability goes beyond the scope of existing benchmarks, which typically evaluate either knowledge question answering without grounding in code or static web code generation without scientific interactivity. To evaluate this integrated ability, we design a hybrid framework that combines programmatic functional testing to rigorously verify interaction logic with visually-grounded qualitative testing to assess rendered outputs against reference snapshots. Building on this framework, we present InteractScience, a benchmark consisting of a substantial set of carefully designed questions across five scientific domains, each paired with unit tests, reference snapshots, and checklists. We evaluate 30 leading open- and closed-source LLMs and report results that highlight ongoing weaknesses in integrating domain knowledge with interactive front-end coding. Our work positions InteractScience as the first benchmark to automatically measure this combined capability with realistic interactive operations, providing a foundation for advancing reliable and educationally useful scientific demonstration code generation. All code and data are publicly available at https://github.com/open-compass/InteractScience.

Figures (17)

• Figure 1: Illustration of three tasks. (a) Knowledge Question Answering: given the query about forces act of a block placed on an inclined plane, an LLM can output a correct textual explanation. (b) Webpage Code Generation: given the instruction of write a blog webpage, an LLM can generate functional static HTML code. (c) Scientific Demonstration Code Generation: generating an interactive demo for the inclined plane scenario, an LLM often fail to produce correct results.
• Figure 2: Pipeline of data collection and evaluation suite synthesis. The data collection step retrieves metadata of scientific demonstrations and corresponding snapshots from the Wolfram Demonstrations Project as seed data. The evaluation suite synthesis step generates implementation plans, test cases, unit tests, and checklist sequentially from the seed data.
• Figure 3: Performance of LLMs across different difficulty levels.
• Figure 4: Performance of LLMs across different disciplines.
• Figure 5: Performance of multimodal LLMs under varying numbers of reference snapshot inputs.