Bayesian Evidence Synthesis for the common effect model

TL;DR

This paper develops Bayesian evidence synthesis methods for the common-effect meta-analytic framework, deriving analytic links between Bayes Factors and standard frequentist test statistics in simple regression and outlining practical pooling procedures across multiple studies. It introduces three data-availability scenarios (D, P, L) and corresponding g-prior and Jeffreys–Zellner–Siow prior–based pooling strategies (Meta-$BF_g^{(D)}$, Meta-$BF_g^{(P)}$, Meta-$BF_g^{(L)}$; plus a $JZS$-prior approach) to combine evidence when full raw data are not available. Through extensive simulations, the authors show that detailed-information and partial-information methods perform well in recovering full-data Bayes factors, while the limited-information approach struggles when the number of studies grows; the $JZS$ prior often yields more conservative behavior. An illustrative positive psychology meta-analysis demonstrates practical implementation and the impact of different priors on pooled evidence. The discussion highlights the limitations of the common-effect assumption and outlines avenues for extending these methods to heterogeneity and more complex evidence-synthesis settings, bridging frequentist and Bayesian perspectives in meta-analysis.

Abstract

Bayes Factors, the Bayesian tool for hypothesis testing, are receiving increasing attention in the literature. Compared to their frequentist rivals ($p$-values or test statistics), Bayes Factors have the conceptual advantage of providing evidence both for and against a null hypothesis, and they can be calibrated so that they do not depend so heavily on the sample size. Research on the synthesis of Bayes Factors arising from individual studies has received increasing attention, mostly for the fixed effects model for meta-analysis. In this work, we review and propose methods for combining Bayes Factors from multiple studies, depending on the level of information available, focusing on the common effect model. In the process, we provide insights with respect to the interplay between frequentist and Bayesian evidence. We assess the performance of the methods discussed via a simulation study and apply the methods in an example from the field of positive psychology.

Figures (2)

• Figure 1: Scatter plots and weighted Cohen's $\kappa$ statistics for the agreement between $BF$ and $\widetilde{BF}$ for binary $X$ ($t$-test) and EQ scenario. $2\log BF_{10}$ are shown, except for $\beta=0$ where $2\log BF_{01}$ is shown. Grid lines indicate the categories of level of evidence as described in Table \ref{['tab:1']}. Values $<$-1 and $>$ 12 are shown as equal.
• Figure 2: Bias, RMSE and weighted Cohen's $\kappa$ versus $K$ (number of subsamples) for random sample sizes. The scale is $2\log BF$. Columns indicate different models (regression and $t$-test) while sub-rows within each metric correspond to different values of $\beta$. The value of 0 for the Bias is highlighted by a dashed line.