Java JIT Testing with Template Extraction

TL;DR

LeJit automates end-to-end testing of Java JIT compilers by automatically extracting executable templates from real code, inserting holes, and generating concrete programs that exercise JIT optimizations. It blends test- and pool-based collection with an enhanced, execution-driven generation process rooted in JAttack, enabling dynamic argument construction for instance methods and robust pruning to reduce false positives. Across 62 open-source projects and multiple JVMs, LeJit generated over 886k concrete programs and uncovered 15 JIT bugs (11 new, including two CVEs), outperforming state-of-the-art tools in coverage and bug detection. The approach is complementary to existing techniques, scalable, and provides actionable, reportable source-level bugs that can be quickly reproduced and fixed by compiler developers.

Abstract

We present LeJit, a template-based framework for testing Java just-in-time (JIT) compilers. Like recent template-based frameworks, LeJit executes a template -- a program with holes to be filled -- to generate concrete programs given as inputs to Java JIT compilers. LeJit automatically generates template programs from existing Java code by converting expressions to holes, as well as generating necessary glue code (i.e., code that generates instances of non-primitive types) to make generated templates executable. We have successfully used LeJit to test a range of popular Java JIT compilers, revealing five bugs in HotSpot, nine bugs in OpenJ9, and one bug in GraalVM. All of these bugs have been confirmed by Oracle and IBM developers, and 11 of these bugs were previously unknown, including two CVEs (Common Vulnerabilities and Exposures). Our comparison with several existing approaches shows that LeJit is complementary to them and is a powerful technique for ensuring Java JIT compiler correctness.

Figures (6)

• Figure 1: An existing program from the text project commonstextStrBuilderSource used as a source for template extraction.
• Figure 2: An example (a) template extracted from the program in Figure \ref{['fig:example:original']}, and (b) a concrete program generated from the template by filling in the holes, which crashed OpenJ9 JIT compiler.
• Figure 3: The overview of LeJit. Dotted-dashed lines: test-based approach; dashed lines: pool-based approach.
• Figure 4: Template extraction algorithm.
• Figure 5: The overlap of bugs detected by LeJit variants. $\mathrm{\textsc{LeJit}\xspace}_{\mathrm{NoTmpl}}$: no templates/generated programs; $\mathrm{\textsc{LeJit}\xspace}_t$: LeJit with Test-based approach; $\mathrm{\textsc{LeJit}\xspace}_p$: LeJit with Pool-based approach; $\mathrm{\textsc{LeJit}\xspace}_{\mathrm{NoPool}}$: enhanced JAttack.