Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating hallucinations and omissions in LLMs for invertible problems: An application to hardware logic design automation

Andrew S. Cassidy, Guillaume Garreau, Jay Sivagnaname, Mike Grassi, Bernard Brezzo, John V. Arthur, Dharmendra S. Modha

TL;DR

This work tackles hallucinations and omissions in large language models (LLMs) when translating logic specifications into hardware description languages. It introduces an invertible, lossless-like flow where an LLM acts as an encoder from Logic Condition Tables (LCTs) to Verilog HDL and a second pass as a decoder back to the LCT, enabling identity verification to detect errors. The approach is demonstrated on a 2D Network-on-Chip router (13 units, ~1500–2000 LOC) using seven public LLMs, showing both error detection and productivity gains, and highlighting the value of precise, modular specifications. The findings suggest that LCT-based, invertible design flows can stabilize automated HDL generation and enhance verification in EDA workflows, motivating further domain-specific tuning and exploration of smaller, energy-efficient models.

Abstract

We show for invertible problems that transform data from a source domain (for example, Logic Condition Tables (LCTs)) to a destination domain (for example, Hardware Description Language (HDL) code), an approach of using Large Language Models (LLMs) as a lossless encoder from source to destination followed by as a lossless decoder back to the source, comparable to lossless compression in information theory, can mitigate most of the LLM drawbacks of hallucinations and omissions. Specifically, using LCTs as inputs, we generate the full HDL for a two-dimensional network-on-chip router (13 units, 1500-2000 lines of code) using seven different LLMs, reconstruct the LCTs from the auto-generated HDL, and compare the original and reconstructed LCTs. This approach yields significant productivity improvements, not only confirming correctly generated LLM logic and detecting incorrectly generated LLM logic but also assisting developers in finding design specification errors.

Mitigating hallucinations and omissions in LLMs for invertible problems: An application to hardware logic design automation

TL;DR

This work tackles hallucinations and omissions in large language models (LLMs) when translating logic specifications into hardware description languages. It introduces an invertible, lossless-like flow where an LLM acts as an encoder from Logic Condition Tables (LCTs) to Verilog HDL and a second pass as a decoder back to the LCT, enabling identity verification to detect errors. The approach is demonstrated on a 2D Network-on-Chip router (13 units, ~1500–2000 LOC) using seven public LLMs, showing both error detection and productivity gains, and highlighting the value of precise, modular specifications. The findings suggest that LCT-based, invertible design flows can stabilize automated HDL generation and enhance verification in EDA workflows, motivating further domain-specific tuning and exploration of smaller, energy-efficient models.

Abstract

We show for invertible problems that transform data from a source domain (for example, Logic Condition Tables (LCTs)) to a destination domain (for example, Hardware Description Language (HDL) code), an approach of using Large Language Models (LLMs) as a lossless encoder from source to destination followed by as a lossless decoder back to the source, comparable to lossless compression in information theory, can mitigate most of the LLM drawbacks of hallucinations and omissions. Specifically, using LCTs as inputs, we generate the full HDL for a two-dimensional network-on-chip router (13 units, 1500-2000 lines of code) using seven different LLMs, reconstruct the LCTs from the auto-generated HDL, and compare the original and reconstructed LCTs. This approach yields significant productivity improvements, not only confirming correctly generated LLM logic and detecting incorrectly generated LLM logic but also assisting developers in finding design specification errors.

Paper Structure

This paper contains 12 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Approach
  4. Logic Condition Tables (LCTs)
  5. Closed Loop Design Flow
  6. Forward Transform Prompt
  7. Inverse Transform Prompt
  8. LLM Models
  9. Experiment: 2D Network-on-Chip Router
  10. Results
  11. Discussion
  12. Conclusion

Figures (2)

  • Figure 1: View of an LLM as an invertible transform $\mathcal{T}$ from an input token sequence $\mathcal{S}_{prompt}$ to an output token sequence $\mathcal{S}_{response}$. The input prompt sequence is a specification. The output sequence contains the completed HDL code. By reapplying the LLM transform $\mathcal{T}^{-1}$ in the reverse direction, the input specification can be reconstructed from the output HDL code and checked versus the original specification.
  • Figure 2: 2D NoC Router Design: (Right) Two dimensions array of cores, each core contains a router (RTR) and a processing element (PE). Each core connects to neighboring cores via North (N) and South (S) connections in the Y-dimension, East (E) and West (W) connections in the X-dimension, as well as from the core router to the core processing element via the Local (L) connection. (Upper Left) The Router is comprised of five input channels with input buffers and five output channels, interconnected via a crossbar switch. (Lower Left) The crossbar switch contains the logic for routing the packets.