BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generation
Haoyuan Li, Zhengyuan Shen, Sullam Jeoung, Yueyan Chen, Jiayu Li, Qi Zhu, Shuai Wang, Vassilis Ioannidis, Huzefa Rangwala
TL;DR
BoundRL tackles structured text segmentation by turning it into boundary generation, outputting only starting tokens and segment labels and reconstructing full segments by locating those tokens in the input, which reduces inference to $O(n)$ token processing and lowers hallucination risk. It trains with reinforcement learning with verifiable rewards (RLVR) that jointly optimize reconstruction fidelity via $ ho_{ extrm{rec}}$ and semantic alignment via $ ext{EM}$ and $ ext{F1}_{ extrm{char}}$, augmented by intermediate candidate perturbations to avoid entropy collapse. On StructSeg, BoundRL with small language models matches or exceeds the performance of much larger models using few-shot prompting, with notable improvements in out-of-domain prompts (LangChain). The approach enables efficient and robust segmentation of long, complex prompts containing natural language, code, and structured data, with potential impact on hierarchical document processing and domain-specific prompting tasks.
Abstract
As structured texts become increasingly complex across diverse domains -- from technical reports to generative AI prompts -- the need for text segmentation into semantically meaningful components becomes critical. Such texts often contain elements beyond plain language, including tables, code snippets, and placeholders, which conventional sentence- or paragraph-level segmentation methods cannot handle effectively. To address this challenge, we propose BoundRL, a novel and efficient approach that jointly performs token-level text segmentation and label prediction for long structured texts. Instead of generating complete contents for each segment, it generates only a sequence of starting tokens and reconstructs the complete contents by locating these tokens within the original texts, thereby reducing inference costs by orders of magnitude and minimizing hallucination. To adapt the model for the output format, BoundRL~performs reinforcement learning with verifiable rewards (RLVR) with a specifically designed reward that jointly optimizes document reconstruction fidelity and semantic alignment. To mitigate entropy collapse, it further constructs intermediate candidates by systematically perturbing a fraction of generated sequences of segments to create stepping stones toward higher-quality solutions. To demonstrate BoundRL's effectiveness on particularly challenging structured texts, we focus evaluation on complex prompts used for LLM applications. Experiments show that BoundRL enables small language models (1.7B parameters) to outperform few-shot prompting of much larger models. Moreover, RLVR with our designed reward yields significant improvements over supervised fine-tuning, and incorporating intermediate candidates further improves both performance and generalization.