RSMLP: A light Sampled MLP Structure for Incomplete Utterance Rewrite

TL;DR

This work tackles Incomplete Utterance Rewriting (IUR) in multi-turn dialogues by introducing RSMLP, a lightweight, MLP-based architecture that uses a down-sampling strategy to capture local and global semantics. The model encodes input with a contextual encoder, then applies a Local Feature Extraction Unit and a Global Feature Extraction Unit to produce representations, which feed a Similarity Feature Matrix to generate a token-level edit matrix for rewriting. RSMLP leverages a bottlenecked hidden size and down-sampled subsequences to balance rewriting quality and inference speed, and constructs token-level edits via LCS-based alignment for training. Across RESTORATION-200K, REWRITE, and CANARD, RSMLP achieves competitive rewriting quality while delivering substantially faster inference and solid real-world edge-device performance in a RAG pipeline.

Abstract

The Incomplete Utterance Rewriting (IUR) task has garnered significant attention in recent years. Its goal is to reconstruct conversational utterances to better align with the current context, thereby enhancing comprehension. In this paper, we introduce a novel and versatile lightweight method, Rewritten-Sampled MLP (RSMLP). By employing an MLP based architecture with a carefully designed down-sampling strategy, RSMLP effectively extracts latent semantic information between utterances and makes appropriate edits to restore incomplete utterances. Due to its simple yet efficient structure, our method achieves competitive performance on public IUR datasets and in real-world applications.

Figures (1)

• Figure 1: The architecture of RSMLP.