Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STILTS-NLI: A Natural Language Interface for STILTS

R. A. Shaw, S. Fotopoulou, M. Taylor, M. Bremer

TL;DR

STILTS-NLI is a natural language interface that generates STILTS commands from user prompts, with agentic support for a user-friendly experience, and demonstrates that this specialised model generates valid commands that match and in some cases outperform larger proprietary models.

Abstract

The Starlink Tables Infrastructure Library Tool Set (STILTS) is a powerful suite for astronomical data analysis, particularly useful when dealing with large datasets. However, like other software suites in astronomy its comprehensive syntax creates a significant learning curve to new users. To address this, we present STILTS-NLI, a natural language interface that generates STILTS commands from user prompts, with agentic support for a user-friendly experience. We developed STILTS-NLI by fine-tuning a compact, open-source Large Language Model (LLM) on a synthetically generated dataset. This dataset was curated and validated to ensure both comprehensive coverage of key STILTS functionalities and the syntactic correctness of the resulting commands. Our results demonstrate that this specialised model generates valid commands that match and in some cases outperform larger proprietary models. By leveraging small, open-source models, STILTS-NLI provides an accessible, low-resource solution that lowers the barrier to entry for using STILTS.

STILTS-NLI: A Natural Language Interface for STILTS

TL;DR

STILTS-NLI is a natural language interface that generates STILTS commands from user prompts, with agentic support for a user-friendly experience, and demonstrates that this specialised model generates valid commands that match and in some cases outperform larger proprietary models.

Abstract

The Starlink Tables Infrastructure Library Tool Set (STILTS) is a powerful suite for astronomical data analysis, particularly useful when dealing with large datasets. However, like other software suites in astronomy its comprehensive syntax creates a significant learning curve to new users. To address this, we present STILTS-NLI, a natural language interface that generates STILTS commands from user prompts, with agentic support for a user-friendly experience. We developed STILTS-NLI by fine-tuning a compact, open-source Large Language Model (LLM) on a synthetically generated dataset. This dataset was curated and validated to ensure both comprehensive coverage of key STILTS functionalities and the syntactic correctness of the resulting commands. Our results demonstrate that this specialised model generates valid commands that match and in some cases outperform larger proprietary models. By leveraging small, open-source models, STILTS-NLI provides an accessible, low-resource solution that lowers the barrier to entry for using STILTS.
Paper Structure (21 sections, 2 equations, 7 figures, 3 tables)

This paper contains 21 sections, 2 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Synthetic Prompt-Response Pair Generation
  4. STILTS documentation
  5. Synthetic training data
  6. Validation of training data
  7. Training
  8. Pre-training
  9. Fine-tuning
  10. Model Evaluation
  11. Agentic Software Implementation
  12. Discussion
  13. Comparison to general purpose models
  14. Accessibility
  15. Future extensions
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Workflow for creating synthetic training data.
  • Figure 2: Cosine similarity distribution of the evaluation data. For both the 2B (shaded grey) model and the larger 27B model (dashed outline).
  • Figure 3: Training and evaluation loss curves from fine-tuning the 2B (top) and 27B (bottom) models.
  • Figure 4: STILTS-NLI implementation.
  • Figure 5: Cosine similarity histogram separated by STILTS task.
  • ...and 2 more figures