CrosGrpsABS: Cross-Attention over Syntactic and Semantic Graphs for Aspect-Based Sentiment Analysis in a Low-Resource Language

TL;DR

CrosGrpsABS tackles ABSA in Bengali by fusing local syntactic structure and global semantic context through bidirectional cross-attention between syntactic and semantic graphs, refined by a Transformer encoder and a highway gating mechanism. The framework integrates a rule-based syntactic graph ($A_{ ext{syntax}}$) and a cosine-based semantic graph ($A_{ ext{semantic}}$) with graph attention on both streams, followed by cross-attention with transformer embeddings to produce rich, aspect-aware representations. Comprehensive experiments on four Bengali ABSA datasets and the SemEval 2014 English benchmarks show CrosGrpsABS achieving state-of-the-art or competitive results, with ablation studies highlighting the critical contributions of semantic information, transformer embeddings, and cross-graph fusion. The approach advances ABSA for low-resource languages by offering interpretability through attention visualizations and robust performance across domains, while suggesting avenues for future cross-lingual transfer and dynamic graph modeling.

Abstract

Aspect-Based Sentiment Analysis (ABSA) is a fundamental task in natural language processing, offering fine-grained insights into opinions expressed in text. While existing research has largely focused on resource-rich languages like English which leveraging large annotated datasets, pre-trained models, and language-specific tools. These resources are often unavailable for low-resource languages such as Bengali. The ABSA task in Bengali remains poorly explored and is further complicated by its unique linguistic characteristics and a lack of annotated data, pre-trained models, and optimized hyperparameters. To address these challenges, this research propose CrosGrpsABS, a novel hybrid framework that leverages bidirectional cross-attention between syntactic and semantic graphs to enhance aspect-level sentiment classification. The CrosGrpsABS combines transformerbased contextual embeddings with graph convolutional networks, built upon rule-based syntactic dependency parsing and semantic similarity computations. By employing bidirectional crossattention, the model effectively fuses local syntactic structure with global semantic context, resulting in improved sentiment classification performance across both low- and high-resource settings. We evaluate CrosGrpsABS on four low-resource Bengali ABSA datasets and the high-resource English SemEval 2014 Task 4 dataset. The CrosGrpsABS consistently outperforms existing approaches, achieving notable improvements, including a 0.93% F1-score increase for the Restaurant domain and a 1.06% gain for the Laptop domain in the SemEval 2014 Task 4 benchmark.

Figures (5)

• Figure 1: Comparison of syntactic and semantic distances across different aspects (Car, Mobile, Movie, and Restaurant). The results indicate that syntactic distances (blue) are generally larger than semantic distances (red), highlighting the need for a mechanism that effectively integrates syntactic and semantic information for aspect-based sentiment analysis in Bengali texts.
• Figure 2: The overall CrosGrpsABS system flow, depicting the construction of syntactic and semantic graphs, cross-attention fusion with transformer embeddings, and final sentiment classification using a highway gating strategy.
• Figure 3: Effect of GNN Layers on Performance (Car Dataset). The chart compares Accuracy and F1 Score across different numbers of GNN layers, ranging from 1 to 7.
• Figure 4: Attention heatmap illustrating the token-to-token attention weights for a sample Bengali sentence. Higher-intensity regions (yellow) indicate stronger attention signals between corresponding tokens.
• Figure 5: Token-level visualization illustrating the importance scores assigned by CrosGrpsABS. The top visualization is from the SemEval 2014 Task 4 (Laptop domain, English), and the bottom visualization is from our Bengali ABSA dataset. Darker shades represent higher token importance in predicting aspect-level sentiment.