Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Group Means and Into the World of Individuals: A Distributional Spotlight for Experimental Effects on Individuals

Roussel Rahman

TL;DR

This paper addresses how experimental effects on individuals can be better understood by moving beyond group means to analyze full RT distributions. It introduces a distributional spotlight that quantifies distributional changes via Relative Entropy and Overlapping Index, and compares these nonparametric measures to Ex-Gaussian and Drift-Diffusion Model parameters using RT data from a visual search task. The findings reveal distinct subgroups of individuals with different distributional changes, and show that intra-individual changes are small relative to inter-individual differences, illustrating the limitations of averaging. The work suggests distributional-level analyses as a robust tool for capturing nuanced, personalized effects and for informing cognitive modeling and individualized interventions.

Abstract

Traditionally, experimental effects on humans are investigated at the group level. In this work, we present a distributional ``spotlight'' to investigate experimental effects at the individual level. Specifically, we estimate the effects on individuals through the changes in the probability distributions of their experimental data across conditions. We test this approach on Reaction Time (RT) data from 10 individuals in a visual search task, examining the effects of (1) information set sizes and (2) the presence or absence of a target on their processing speed. The changes in individuals' RT distributions are measured using three approaches: (i) direct measurements of distributional changes are compared against the changes captured by two established models of RT: (ii) the ex-Gaussian distribution and (iii) the Drift-Diffusion model. We find that direct measurement of distributional changes provides the clearest view of the effects on individuals and highlights the presence of two sub-groups based on the effects experienced: one that shows neither effect and the other showing only the target-presence effect. Moreover, the intra-individual changes across conditions (i.e., the experimental effects) appear much smaller than the inter-individual differences (i.e., the random effects). Generally, these results highlight the merits of going beyond group means and examining the effects on individuals, as well as the effectiveness of the distributional spotlight in such pursuits.

Beyond Group Means and Into the World of Individuals: A Distributional Spotlight for Experimental Effects on Individuals

TL;DR

This paper addresses how experimental effects on individuals can be better understood by moving beyond group means to analyze full RT distributions. It introduces a distributional spotlight that quantifies distributional changes via Relative Entropy and Overlapping Index, and compares these nonparametric measures to Ex-Gaussian and Drift-Diffusion Model parameters using RT data from a visual search task. The findings reveal distinct subgroups of individuals with different distributional changes, and show that intra-individual changes are small relative to inter-individual differences, illustrating the limitations of averaging. The work suggests distributional-level analyses as a robust tool for capturing nuanced, personalized effects and for informing cognitive modeling and individualized interventions.

Abstract

Traditionally, experimental effects on humans are investigated at the group level. In this work, we present a distributional ``spotlight'' to investigate experimental effects at the individual level. Specifically, we estimate the effects on individuals through the changes in the probability distributions of their experimental data across conditions. We test this approach on Reaction Time (RT) data from 10 individuals in a visual search task, examining the effects of (1) information set sizes and (2) the presence or absence of a target on their processing speed. The changes in individuals' RT distributions are measured using three approaches: (i) direct measurements of distributional changes are compared against the changes captured by two established models of RT: (ii) the ex-Gaussian distribution and (iii) the Drift-Diffusion model. We find that direct measurement of distributional changes provides the clearest view of the effects on individuals and highlights the presence of two sub-groups based on the effects experienced: one that shows neither effect and the other showing only the target-presence effect. Moreover, the intra-individual changes across conditions (i.e., the experimental effects) appear much smaller than the inter-individual differences (i.e., the random effects). Generally, these results highlight the merits of going beyond group means and examining the effects on individuals, as well as the effectiveness of the distributional spotlight in such pursuits.

Paper Structure

This paper contains 19 sections, 5 equations, 3 figures, 1 table.

Table of Contents

  1. When to Go beyond the Mean?
  2. Distributional Analyses of Experimental Effects on Individuals
  3. Goals and Scope of this work
  4. Modeling RT Distributions
  5. Ex-G Distribution
  6. Drift-Diffusion Model of RT
  7. Measuring Difference (or Similarity) between RT Distributions
  8. Relative Entropy
  9. Overlapping Index
  10. Methods
  11. Experimental Task and RT Dataset
  12. Estimating Model Parameters from RT Data
  13. Six Measures of Distributional Differences
  14. Results
  15. RT Distributions of Individuals
  16. ...and 4 more sections

Figures (3)

  • Figure 1: RT Distributions of 10 participants in all experimental conditions. The solid lines represent distributions under target-present conditions, and the dashed lines represent target-absent conditions. Each distribution is plotted in a range that contains 99.8% of the distribution.
  • Figure 2: Distributional changes across conditions calculated using six different measures. For each measure, the distributions from all pairs of conditions are compared and presented as a map.
  • Figure 3: Average experimental effects on participants measured in six ways. As we see, RE and OI provide the clearest separation between the two groups.