Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Magellan: Guided MCTS for Latent Space Exploration and Novelty Generation

Lufan Chang

TL;DR

This work tackles the challenge that large language models tend to generate safe, high-probability ideas rather than truly novel concepts. It introduces Magellan, a principled framework that reframes creative generation as guided exploration in the LLM latent space using Monte Carlo Tree Search (MCTS) with a hierarchical guidance system: a long-range semantic compass and a landscape-aware, multi-objective value function. The method constructs a semantic compass from automated theme generation and employs a multi-objective reward to balance coherence, novelty, and progress, culminating in a structured final concept extracted along the best search path. Across extensive experiments against strong baselines and AI-for-Science frameworks, Magellan demonstrates superior novelty and overall quality, albeit with higher computational cost, suggesting that principled guided search can meaningfully advance autonomous scientific ideation.

Abstract

Large Language Models (LLMs) often struggle with generating truly innovative ideas, typically defaulting to high-probability, familiar concepts within their training data's "gravity wells." While advanced search-based methods like Tree of Thoughts (ToT) attempt to mitigate this, they are fundamentally limited by their reliance on unprincipled, inconsistent self-evaluation heuristics to guide exploration. To address this gap, we introduce \textbf{Magellan}, a novel framework that reframes creative generation as a principled, guided exploration of an LLM's latent conceptual space. At its core, Magellan employs Monte Carlo Tree Search (MCTS) governed by a hierarchical guidance system. For long-range direction, a "semantic compass" vector, formulated via orthogonal projection, steers the search towards relevant novelty. For local, step-by-step decisions, a landscape-aware value function replaces flawed self-evaluation with an explicit reward structure that balances intrinsic coherence, extrinsic novelty, and narrative progress. Extensive experiments demonstrate that Magellan significantly outperforms strong baselines, including ReAct and ToT, in generating scientific ideas with superior plausibility and innovation. Our work shows that for creative discovery, a principled, guided search is more effective than unconstrained agency, paving the way for LLMs to become more capable partners in innovation.

Magellan: Guided MCTS for Latent Space Exploration and Novelty Generation

TL;DR

This work tackles the challenge that large language models tend to generate safe, high-probability ideas rather than truly novel concepts. It introduces Magellan, a principled framework that reframes creative generation as guided exploration in the LLM latent space using Monte Carlo Tree Search (MCTS) with a hierarchical guidance system: a long-range semantic compass and a landscape-aware, multi-objective value function. The method constructs a semantic compass from automated theme generation and employs a multi-objective reward to balance coherence, novelty, and progress, culminating in a structured final concept extracted along the best search path. Across extensive experiments against strong baselines and AI-for-Science frameworks, Magellan demonstrates superior novelty and overall quality, albeit with higher computational cost, suggesting that principled guided search can meaningfully advance autonomous scientific ideation.

Abstract

Large Language Models (LLMs) often struggle with generating truly innovative ideas, typically defaulting to high-probability, familiar concepts within their training data's "gravity wells." While advanced search-based methods like Tree of Thoughts (ToT) attempt to mitigate this, they are fundamentally limited by their reliance on unprincipled, inconsistent self-evaluation heuristics to guide exploration. To address this gap, we introduce \textbf{Magellan}, a novel framework that reframes creative generation as a principled, guided exploration of an LLM's latent conceptual space. At its core, Magellan employs Monte Carlo Tree Search (MCTS) governed by a hierarchical guidance system. For long-range direction, a "semantic compass" vector, formulated via orthogonal projection, steers the search towards relevant novelty. For local, step-by-step decisions, a landscape-aware value function replaces flawed self-evaluation with an explicit reward structure that balances intrinsic coherence, extrinsic novelty, and narrative progress. Extensive experiments demonstrate that Magellan significantly outperforms strong baselines, including ReAct and ToT, in generating scientific ideas with superior plausibility and innovation. Our work shows that for creative discovery, a principled, guided search is more effective than unconstrained agency, paving the way for LLMs to become more capable partners in innovation.
Paper Structure (43 sections, 10 equations, 1 figure, 7 tables, 1 algorithm)

This paper contains 43 sections, 10 equations, 1 figure, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. LLMs for Scientific Discovery and Structured Reasoning
  4. Advanced Search for Novelty and Diversity
  5. Methodology
  6. Automated Theme Generation and Guidance Vector Formulation
  7. Knowledge Corpus Construction.
  8. Theme Synthesis via Conceptual Bridging.
  9. Guidance Vector Formulation.
  10. Guided Narrative Search via MCTS
  11. Node Evaluation.
  12. Selection.
  13. Expansion and Pruning.
  14. Backpropagation.
  15. Termination Condition.
  16. ...and 28 more sections

Figures (1)

  • Figure 1: t-SNE visualization of the knowledge corpus embeddings. Each point represents a paper, colored by its assigned cluster ID (0-19). The plot shows clear semantic separation between conceptual groups, validating the basis of our cross-disciplinary theme generation strategy.