Question Classification with Deep Contextualized Transformer
Haozheng Luo, Ningwei Liu, Charles Feng
TL;DR
The paper tackles QA classification by integrating deep contextualized transformers to model cross-sentence dependencies and complex question forms beyond simple text classification. It introduces a multi-path architecture that combines self-attention, BiLM-based deep contextualized representations, and a Combination-level RNN, supplemented by a parser-tree guided disambiguation module. Evaluations on SwDA and SQuAD show the method achieving higher accuracy than several baselines, including TF-IDF/GloVe and prior dialogue-classification models, with competitive performance from RoBERTa. This approach demonstrates the value of combining contextualized embeddings with structure-aware parsing to enable robust QA classification across datasets and has potential impact on industry-scale QA systems.
Abstract
The latest work for Question and Answer problems is to use the Stanford Parse Tree. We build on prior work and develop a new method to handle the Question and Answer problem with the Deep Contextualized Transformer to manage some aberrant expressions. We also conduct extensive evaluations of the SQuAD and SwDA dataset and show significant improvement over QA problem classification of industry needs. We also investigate the impact of different models for the accuracy and efficiency of the problem answers. It shows that our new method is more effective for solving QA problems with higher accuracy