Transformer-based Named Entity Recognition in Construction Supply Chain Risk Management in Australia

TL;DR

The paper tackles risk management in Australia's construction supply chains by applying transformer-based NER to news articles to extract risk-related entities, enabling real-time insight into vulnerabilities. It builds a bespoke Australian CSCRM risk taxonomy, curates a dataset of ~2000 news articles, and annotates 39,500 entities across six categories using BIO tagging, then benchmarks multiple transformer models (e.g., BERT, RoBERTa, DistilBERT, ALBERT, ELECTRA) on this task. Key findings show RoBERTa achieving the best overall F1 score (~0.858) with strong precision and recall, while T5 and GPT-3 exhibit limitations for sequence labeling tasks; hyper-parameter tuning (grid search) further improves results, especially with lower learning rates and AdamW optimization. The work demonstrates NLP-driven potential for CSCRM, supporting risk monitoring, knowledge-graph construction, and more proactive project management in the Australian construction sector, with practical implications for resilience against global disruptions.

Abstract

The construction industry in Australia is characterized by its intricate supply chains and vulnerability to myriad risks. As such, effective supply chain risk management (SCRM) becomes imperative. This paper employs different transformer models, and train for Named Entity Recognition (NER) in the context of Australian construction SCRM. Utilizing NER, transformer models identify and classify specific risk-associated entities in news articles, offering a detailed insight into supply chain vulnerabilities. By analysing news articles through different transformer models, we can extract relevant entities and insights related to specific risk taxonomies local (milieu) to the Australian construction landscape. This research emphasises the potential of NLP-driven solutions, like transformer models, in revolutionising SCRM for construction in geo-media specific contexts.

Figures (7)

• Figure 1: Evolution of Probabilistic Models in Machine Learning.
• Figure 2: Transformer’s Architecture.
• Figure 3: Transformer’s Architecture.
• Figure 4: F1 scores of different models for each entity.
• Figure 5: Comparative Analysis of Transformer Models’ Performance Using Adam and AdamW Optimizers Across Various Hyperparameters.