Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Counting Defiers: A Design-Based Model of an Experiment Can Reveal Evidence Beyond the Average Effect

Neil Christy, Amanda Ellen Kowalski

Abstract

Using only a binary intervention and outcome and the design of the randomization within an experiment, we construct a design-based likelihood of the joint distribution of potential outcomes in the sample -- the numbers of always takers, compliers, defiers, and never takers. We develop a visualization to show that samples with defiers can sometimes generate the data in more ways than samples without, yielding a higher likelihood. This likelihood can vary within the Frechet bounds, even though the traditional likelihood does not. Evidence is weak, but it exists, as we illustrate with health applications and our dbmle package.

Counting Defiers: A Design-Based Model of an Experiment Can Reveal Evidence Beyond the Average Effect

Abstract

Using only a binary intervention and outcome and the design of the randomization within an experiment, we construct a design-based likelihood of the joint distribution of potential outcomes in the sample -- the numbers of always takers, compliers, defiers, and never takers. We develop a visualization to show that samples with defiers can sometimes generate the data in more ways than samples without, yielding a higher likelihood. This likelihood can vary within the Frechet bounds, even though the traditional likelihood does not. Evidence is weak, but it exists, as we illustrate with health applications and our dbmle package.

Paper Structure

This paper contains 26 sections, 2 theorems, 47 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Visualization of a Stylized Example: A Design-Based Model Can Reveal Evidence Beyond the Average Effect
  3. A Design-Based Model of an Experiment
  4. A Design-Based Model Implies a Likelihood for the Joint Distribution of Potential Outcomes in the Sample
  5. The Design-Based Likelihood Can Vary with the Number of Defiers within the Fréchet Bounds Determined by the Estimated Marginal Distributions of Potential Outcomes
  6. The Design-Based Likelihood Enables Maximum Likelihood Estimation
  7. Applications: Counting Defiers in Health Care
  8. Contributions to the Literature
  9. Implications for Econometric and Applied Research
  10. Derivations of Likelihoods
  11. Design-Based Likelihoods Can Vary Conditional on the Marginal Distributions of Potential Outcomes
  12. Canonical Sampling-Based Likelihoods are Flat Conditional on the Marginal Distributions of Potential Outcomes
  13. Alternative Sampling-Based Likelihoods Can Vary Conditional on the Marginal Distributions of Potential Outcomes
  14. Justifications for Our Maximum Likelihood Decision Rule
  15. Preliminaries on Statistical Decision Theory
  16. ...and 11 more sections

Key Result

Proposition 1

When utility $u$ takes the form in (eq:util) and the prior belief $\pi$ takes the form in (eq:prior), the maximum likelihood decision rule $\widehat{{ \boldsymbol{\theta} }}_{MLE}$ is Bayes optimal:

Figures (5)

  • Figure 1: In an Experiment with Six Patients, the Likelihood Varies Among Joint Distributions of Potential Outcomes with the Same Marginal Distributions as the Data (2/3 Takeup in Intervention, 1/3 Takeup in Control) and is Maximized with Four Compliers and Two Defiers
  • Figure 2: The Likelihood Varies with the Number of Defiers within the Estimated Fréchet Set, And The Smallest Collection of Likelihoods to Contain 95% of the Mass Excludes 105 to 116 Defiers (in Lighter Shading) and the Midpoint
  • Figure B.1: Visualization of Our Proposed Statistical Decision Rule: How the MLE Varies with All Possible Data from Samples of 50 and 200 with Half in Intervention
  • Figure B.2: Under Optimality Conditions, For Increasing Sample Sizes, Performance of Proposed Maximum Likelihood Rule Increases Relative to Fréchet and Monotonicity Rules
  • Figure C.3: Standard Statistics, Proposed Design-Based Maximum Likelihood Estimates, and Auxiliary Statistics

Theorems & Definitions (4)

  • Proposition 1
  • proof
  • Proposition 2
  • proof