Auto311: A Confidence-guided Automated System for Non-emergency Calls

TL;DR

Auto311 introduces a confidence-guided automated system for non-emergency 311 calls, combining incident-type prediction, information itemization, and dialogue optimization to generate complete case reports during calls. Using 11,796 Nashville DEC recordings, it achieves a 92.54% average F-1 on incident-type prediction and 0.93 consistency in itemization, with emulations reaching 94.49% overall categorization accuracy. The system employs a hierarchical, multi-type prediction framework, a YAKE/Sentence-BERT-based information-consistency metric, Monte Carlo Dropout for confidence estimation, and rule-based handover controls to ensure safe operation. The work demonstrates strong real-world deployability, adaptability to shifting incident types, and potential to reduce conversation turns while maintaining dispatch accuracy, offering a scalable path for automating non-emergency calls in city services.

Abstract

Emergency and non-emergency response systems are essential services provided by local governments and critical to protecting lives, the environment, and property. The effective handling of (non-)emergency calls is critical for public safety and well-being. By reducing the burden through non-emergency callers, residents in critical need of assistance through 911 will receive a fast and effective response. Collaborating with the Department of Emergency Communications (DEC) in Nashville, we analyzed 11,796 non-emergency call recordings and developed Auto311, the first automated system to handle 311 non-emergency calls, which (1) effectively and dynamically predicts ongoing non-emergency incident types to generate tailored case reports during the call; (2) itemizes essential information from dialogue contexts to complete the generated reports; and (3) strategically structures system-caller dialogues with optimized confidence. We used real-world data to evaluate the system's effectiveness and deployability. The experimental results indicate that the system effectively predicts incident type with an average F-1 score of 92.54%. Moreover, the system successfully itemizes critical information from relevant contexts to complete reports, evincing a 0.93 average consistency score compared to the ground truth. Additionally, emulations demonstrate that the system effectively decreases conversation turns as the utterance size gets more extensive and categorizes the ongoing call with 94.49% mean accuracy.

Figures (8)

• Figure 1: Auto311 in Emergency Response
• Figure 2: Confidence-guided System Design
• Figure 3: A General Case Study in Confidence Guidance
• Figure 4: Confidence Changes in Shifting Incident Types
• Figure 5: Emulated Usage of Auto311