Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GoNoGo: An Efficient LLM-based Multi-Agent System for Streamlining Automotive Software Release Decision-Making

Arsham Gholamzadeh Khoee, Yinan Yu, Robert Feldt, Andris Freimanis, Patrick Andersson Rhodin, Dhasarathy Parthasarathy

TL;DR

GoNoGo represents an efficient and user-friendly LLM-based solution currently employed in an industrial partner's company to assist with software release decision-making, supporting more informed and timely decisions in the release process for risk-sensitive vehicle systems.

Abstract

Traditional methods for making software deployment decisions in the automotive industry typically rely on manual analysis of tabular software test data. These methods often lead to higher costs and delays in the software release cycle due to their labor-intensive nature. Large Language Models (LLMs) present a promising solution to these challenges. However, their application generally demands multiple rounds of human-driven prompt engineering, which limits their practical deployment, particularly for industrial end-users who need reliable and efficient results. In this paper, we propose GoNoGo, an LLM agent system designed to streamline automotive software deployment while meeting both functional requirements and practical industrial constraints. Unlike previous systems, GoNoGo is specifically tailored to address domain-specific and risk-sensitive systems. We evaluate GoNoGo's performance across different task difficulties using zero-shot and few-shot examples taken from industrial practice. Our results show that GoNoGo achieves a 100% success rate for tasks up to Level 2 difficulty with 3-shot examples, and maintains high performance even for more complex tasks. We find that GoNoGo effectively automates decision-making for simpler tasks, significantly reducing the need for manual intervention. In summary, GoNoGo represents an efficient and user-friendly LLM-based solution currently employed in our industrial partner's company to assist with software release decision-making, supporting more informed and timely decisions in the release process for risk-sensitive vehicle systems.

GoNoGo: An Efficient LLM-based Multi-Agent System for Streamlining Automotive Software Release Decision-Making

TL;DR

GoNoGo represents an efficient and user-friendly LLM-based solution currently employed in an industrial partner's company to assist with software release decision-making, supporting more informed and timely decisions in the release process for risk-sensitive vehicle systems.

Abstract

Traditional methods for making software deployment decisions in the automotive industry typically rely on manual analysis of tabular software test data. These methods often lead to higher costs and delays in the software release cycle due to their labor-intensive nature. Large Language Models (LLMs) present a promising solution to these challenges. However, their application generally demands multiple rounds of human-driven prompt engineering, which limits their practical deployment, particularly for industrial end-users who need reliable and efficient results. In this paper, we propose GoNoGo, an LLM agent system designed to streamline automotive software deployment while meeting both functional requirements and practical industrial constraints. Unlike previous systems, GoNoGo is specifically tailored to address domain-specific and risk-sensitive systems. We evaluate GoNoGo's performance across different task difficulties using zero-shot and few-shot examples taken from industrial practice. Our results show that GoNoGo achieves a 100% success rate for tasks up to Level 2 difficulty with 3-shot examples, and maintains high performance even for more complex tasks. We find that GoNoGo effectively automates decision-making for simpler tasks, significantly reducing the need for manual intervention. In summary, GoNoGo represents an efficient and user-friendly LLM-based solution currently employed in our industrial partner's company to assist with software release decision-making, supporting more informed and timely decisions in the release process for risk-sensitive vehicle systems.
Paper Structure (20 sections, 3 figures, 1 table)

This paper contains 20 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Manual Process of Release Decisions: Insights From the Industry
  3. GoNoGo: Intelligent Software Release Assistant
  4. System Requirements
  5. System Architecture
  6. Planner
  7. Domain-specificity
  8. Risk-sensitivity
  9. Actor
  10. System Implementation
  11. Experiments
  12. Data
  13. Benchmark Overview
  14. Evaluation
  15. Results
  16. ...and 5 more sections

Figures (3)

  • Figure 1: An actual example demonstrating the use of the LLM-based multi-agent system for automating ad-hoc tabular data analysis.
  • Figure 2: Architecture of the LLM-based multi-agent system GoNoGo along with the illustration of the interaction procedure of the system. GoNoGo receives high-level queries from the end user, performs the required data manipulations, and outputs the result table as a decision support resource. GoNoGo comprises a Planner agent, which interprets queries and devises analysis strategies using Chain-of-Thought prompting and self-consistency, supported by a Knowledge Base and Examples for few-shot learning. The Actor includes a Coder LLM with a Self-reflection mechanism, utilizing Memory and Plugins for code generation and error resolution. The total running time of GoNoGo for one user query is approximately 120 seconds, which satisfies typical user requirements.
  • Figure 3: Planner prompting strategies addressing domain-specificity and risk-sensitivity in the LLM-based agent system for tabular data analysis.