Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FormulaCompiler.jl and Margins.jl: Efficient Marginal Effects in Julia

Eric Feltham

TL;DR

The paper introduces Margins.jl and FormulaCompiler.jl, two Julia packages designed to compute marginal effects with unprecedented scale and efficiency. By separating a high-level statistical interface from a low-level, zero-allocation computation engine based on position-mapped compilation, the approach achieves O(p) per-row evaluation and dramatically reduces memory usage compared to existing tools. The 2×2 framework (Population vs Profile × Effects vs Predictions) clarifies the quantities computed (AME/MEM/MER vs AAP/APM/APR) and enables robust methods, including delta-method standard errors and elasticities, across complex models with interactions and categorical variables. Benchmark results show up to 622× average speedup and 460× memory reduction over R's marginaleffects, with successful execution on datasets containing hundreds of thousands of observations where prior tools fail, marking a practical and scalable marginal effects solution for Julia's statistical ecosystem.

Abstract

Marginal effects analysis is fundamental to interpreting statistical models, yet existing implementations face computational constraints that limit analysis at scale. We introduce two Julia packages that address this gap. Margins.jl provides a clean two-function API organizing analysis around a 2-by-2 framework: evaluation context (population vs profile) by analytical target (effects vs predictions). The package supports interaction analysis through second differences, elasticity measures, categorical mixtures for representative profiles, and robust standard errors. FormulaCompiler.jl provides the computational foundation, transforming statistical formulas into zero-allocation, type-specialized evaluators that enable O(p) per-row computation independent of dataset size. Together, these packages achieve 622x average speedup and 460x memory reduction compared to R's marginaleffects package, with successful computation of average marginal effects and delta-method standard errors on 500,000 observations where R fails due to memory exhaustion, providing the first comprehensive and efficient marginal effects implementation for Julia's statistical ecosystem.

FormulaCompiler.jl and Margins.jl: Efficient Marginal Effects in Julia

TL;DR

The paper introduces Margins.jl and FormulaCompiler.jl, two Julia packages designed to compute marginal effects with unprecedented scale and efficiency. By separating a high-level statistical interface from a low-level, zero-allocation computation engine based on position-mapped compilation, the approach achieves O(p) per-row evaluation and dramatically reduces memory usage compared to existing tools. The 2×2 framework (Population vs Profile × Effects vs Predictions) clarifies the quantities computed (AME/MEM/MER vs AAP/APM/APR) and enables robust methods, including delta-method standard errors and elasticities, across complex models with interactions and categorical variables. Benchmark results show up to 622× average speedup and 460× memory reduction over R's marginaleffects, with successful execution on datasets containing hundreds of thousands of observations where prior tools fail, marking a practical and scalable marginal effects solution for Julia's statistical ecosystem.

Abstract

Marginal effects analysis is fundamental to interpreting statistical models, yet existing implementations face computational constraints that limit analysis at scale. We introduce two Julia packages that address this gap. Margins.jl provides a clean two-function API organizing analysis around a 2-by-2 framework: evaluation context (population vs profile) by analytical target (effects vs predictions). The package supports interaction analysis through second differences, elasticity measures, categorical mixtures for representative profiles, and robust standard errors. FormulaCompiler.jl provides the computational foundation, transforming statistical formulas into zero-allocation, type-specialized evaluators that enable O(p) per-row computation independent of dataset size. Together, these packages achieve 622x average speedup and 460x memory reduction compared to R's marginaleffects package, with successful computation of average marginal effects and delta-method standard errors on 500,000 observations where R fails due to memory exhaustion, providing the first comprehensive and efficient marginal effects implementation for Julia's statistical ecosystem.
Paper Structure (43 sections, 8 equations, 4 figures)

This paper contains 43 sections, 8 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Marginal Effects Software
  4. Formula Systems
  5. Positioning
  6. Marginal Effects: Statistical Framework
  7. The 2 \\ times 2 Framework
  8. Variance Estimation
  9. Margins.jl: Statistical Interface
  10. API Design: Two Functions
  11. Population Analysis
  12. Profile Analysis and Reference Grids
  13. Categorical Variables
  14. Categorical Mixtures
  15. Result Types and Output
  16. ...and 28 more sections

Figures (4)

  • Figure 1: System architecture showing Margins.jl providing the statistical interface over FormulaCompiler.jl's computational engine, with integration to Julia's statistical ecosystem including GLM.jl, MixedModels.jl, and StatsModels.jl.
  • Figure 2: Compilation pipeline: FormulaCompiler.jl transforms StatsModels.jl formulas through four stages---(1) term decomposition into atomic operations, (2) position mapping to fixed output indices, (3) type encoding of operations in Julia's type parameters, and (4) code generation with buffer preallocation---to produce zero-allocation evaluators.
  • Figure 3: Sequence diagram showing zero-allocation row evaluation: type-stable dispatch resolves column access at compile time, enabling direct memory writes without intermediate allocations.
  • Figure 4: Per-row evaluation time remains constant as $n$ increases from 10,000 to 1,000,000 observations, demonstrating $O(1)$ complexity in dataset size.