Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BERT-VBD: Vietnamese Multi-Document Summarization Framework

Tuan-Cuong Vuong, Trang Mai Xuan, Thien Van Luong

TL;DR

The paper tackles Vietnamese multi-document summarization (MDS) by addressing the limitations of purely extractive or abstractive methods through a hybrid two-component pipeline. It combines an SBERT-based extractive stage to select salient content with a VBD-LLaMA2-7B-50b-based abstractive stage to generate fluent, paraphrased summaries, processing input data with Vietnamese-specific pre-processing and clustering. On the VN-MDS dataset, the approach yields competitive ROUGE scores and outperforms several baselines, including non-hybrid methods and certain hybrid models, demonstrating the value of integrating extraction and abstractive rewriting for Vietnamese texts. The work offers practical implications for scalable, content-preserving Vietnamese MDS and lays groundwork for future extensions to additional datasets and unstructured data domains.

Abstract

In tackling the challenge of Multi-Document Summarization (MDS), numerous methods have been proposed, spanning both extractive and abstractive summarization techniques. However, each approach has its own limitations, making it less effective to rely solely on either one. An emerging and promising strategy involves a synergistic fusion of extractive and abstractive summarization methods. Despite the plethora of studies in this domain, research on the combined methodology remains scarce, particularly in the context of Vietnamese language processing. This paper presents a novel Vietnamese MDS framework leveraging a two-component pipeline architecture that integrates extractive and abstractive techniques. The first component employs an extractive approach to identify key sentences within each document. This is achieved by a modification of the pre-trained BERT network, which derives semantically meaningful phrase embeddings using siamese and triplet network structures. The second component utilizes the VBD-LLaMA2-7B-50b model for abstractive summarization, ultimately generating the final summary document. Our proposed framework demonstrates a positive performance, attaining ROUGE-2 scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art baselines.

BERT-VBD: Vietnamese Multi-Document Summarization Framework

TL;DR

The paper tackles Vietnamese multi-document summarization (MDS) by addressing the limitations of purely extractive or abstractive methods through a hybrid two-component pipeline. It combines an SBERT-based extractive stage to select salient content with a VBD-LLaMA2-7B-50b-based abstractive stage to generate fluent, paraphrased summaries, processing input data with Vietnamese-specific pre-processing and clustering. On the VN-MDS dataset, the approach yields competitive ROUGE scores and outperforms several baselines, including non-hybrid methods and certain hybrid models, demonstrating the value of integrating extraction and abstractive rewriting for Vietnamese texts. The work offers practical implications for scalable, content-preserving Vietnamese MDS and lays groundwork for future extensions to additional datasets and unstructured data domains.

Abstract

In tackling the challenge of Multi-Document Summarization (MDS), numerous methods have been proposed, spanning both extractive and abstractive summarization techniques. However, each approach has its own limitations, making it less effective to rely solely on either one. An emerging and promising strategy involves a synergistic fusion of extractive and abstractive summarization methods. Despite the plethora of studies in this domain, research on the combined methodology remains scarce, particularly in the context of Vietnamese language processing. This paper presents a novel Vietnamese MDS framework leveraging a two-component pipeline architecture that integrates extractive and abstractive techniques. The first component employs an extractive approach to identify key sentences within each document. This is achieved by a modification of the pre-trained BERT network, which derives semantically meaningful phrase embeddings using siamese and triplet network structures. The second component utilizes the VBD-LLaMA2-7B-50b model for abstractive summarization, ultimately generating the final summary document. Our proposed framework demonstrates a positive performance, attaining ROUGE-2 scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art baselines.
Paper Structure (18 sections, 1 equation, 1 figure, 4 tables)

This paper contains 18 sections, 1 equation, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Overview
  5. Data Pre-processing
  6. Extractive Summarization
  7. Abstractive Summarization
  8. Performance Evaluation and Comparison
  9. Hardware Configuration
  10. Evaluation Procedure
  11. Experiment on extractive summarization:
  12. Comparison with existing hybrid approaches:
  13. Comparison with existing non-hybrid approaches:
  14. Experiment Results
  15. Experiment on extractive summarization:
  16. ...and 3 more sections

Figures (1)

  • Figure 1: Pipeline for proposed model.