Overview of the SciHigh Track at FIRE 2025: Research Highlight Generation from Scientific Papers

TL;DR

This paper introduces the SciHigh track, a benchmark for automatically generating concise research highlights from scientific abstracts using the MixSub dataset. It surveys and evaluates diverse approaches, including extractive, abstractive, and retrieval-augmented transformer-based models, with evaluation guided by ROUGE, METEOR, and BERTScore and final rankings based on ROUGE-L F1. Twelve teams participated, with the top system—Text_highlights_gen using Pegasus—reaching a ROUGE-L F1 of 23.45%, demonstrating competitive performance in aligning highlights with author-written references. The study demonstrates the feasibility of automatic highlight generation to reduce reading effort and improve metadata for literature reviews and discovery, and establishes a benchmark with avenues for cross-domain, multilingual, and knowledge-augmented enhancements.

Abstract

`SciHigh: Research Highlight Generation from Scientific Papers' focuses on the task of automatically generating concise, informative, and meaningful bullet-point highlights directly from scientific abstracts. The goal of this task is to evaluate how effectively computational models can generate highlights that capture the key contributions, findings, and novelty of a paper in a concise form. Highlights help readers grasp essential ideas quickly and are often easier to read and understand than longer paragraphs, especially on mobile devices. The track uses the MixSub dataset \cite{10172215}, which provides pairs of abstracts and corresponding author-written highlights. In this inaugural edition of the track, 12 teams participated, exploring various approaches, including pre-trained language models, to generate highlights from this scientific dataset. All submissions were evaluated using established metrics such as ROUGE, METEOR, and BERTScore to measure both alignment with author-written highlights and overall informativeness. Teams were ranked based on ROUGE-L scores. The findings suggest that automatically generated highlights can reduce reading effort, accelerate literature reviews, and enhance metadata for digital libraries and academic search platforms. SciHigh provides a dedicated benchmark for advancing methods aimed at concise and accurate highlight generation from scientific writing.