A totally empirical basis of science

Abstract

Statistical hypothesis testing is the central method to demarcate scientific theories in both exploratory and inferential analyses. However, whether this method befits such purpose remains a matter of debate. Established approaches to hypothesis testing make several assumptions on the data generation process beyond the scientific theory. Most of these assumptions not only remain unmet in realistic datasets, but often introduce unwarranted bias in the analysis. Here, we depart from such restrictive assumptions to propose an alternative framework of total empiricism. We derive the Information-test ($I$-test) which allows for testing versatile hypotheses including non-null effects. To exemplify the adaptability of the $I$-test to application and study design, we revisit the hypothesis of interspecific metabolic scaling in mammals, ultimately rejecting both competing theories of pure allometry.

Figures (2)

• Figure 1: Total Empiricism. Probability simplex for three manifestations, where the only structural condition is the normalization. Starting from the data $\mathfrak{f}$ (orange dot), the hypothesis distribution $\mathfrak{p}_\text{H}$ (purple dot) is obtained by the $I$-projection of $\mathfrak{f}$ on the hypothesis TotEmplex $\mathcal{T}_\text{H}$ (purple line). The hypothesis distribution $\mathfrak{p}_\text{H}$ as any other distribution $\mathfrak{q} \in \mathcal{T}_\text{H}$ (purple cross) fulfills the hypothesis conditions $\mathbf{G}\mathfrak{q} = \boldsymbol{\mu}_\text{H}$ (purple dot on the right). The probability of observing a deviation from the hypothesis as in the data is governed by the $I$-projection of $\mathfrak{p}_\text{H}$ on the alternative TotEmplex $\mathcal{T}_\text{A}$ (orange line), which often coincides with the data $\mathfrak{p}_\text{A}=\mathfrak{f}$. Every member $\mathfrak{q}$ (orange cross) of the alternative TotEmplex $\mathcal{T}_\text{A}$ fulfils $\mathbf{G}\mathfrak{q} = \mathbf{G}\mathfrak{f} = \boldsymbol{\mu}_\text{A}$ (orange dot on the right). For each possible deviation from the hypothesis, there is a TotEmplex (horizontal parallel lines) with associated $I$-projection of $\mathfrak p_\text{H}$ (orange and gray dots). The corresponding statistic $t = 2N\mathop{\mathrm{D}}\limits\infdivx{\mathfrak q}{\mathfrak{p}_\text{H}}$ with $\mathfrak q=\mathfrak{p}_\text{A},\mathfrak{p}_{\text{A}'}$ is $\chi^2$-distributed (on the left) with the degrees of freedom $k = 1$ being the number of hypothesis conditions.
• Figure S1: Normal QQ Plot of the Residuals of the fitted Model. Shown are the theoretical quantiles ($x$-axis) versus the quantiles of the residuals ($y$-axis in $\ell~\text{O}_2~\text{h}^{-1}$) for the fitted models using ordinary least squares- (OLS), major axis- (MA), standardized major axis- (SMA), reduced major axis- (RMA), and phylogenetic generalized least squares (PGLS)-regression. The almost horizontal line in each plot indicates the quantiles expected for the normal distribution. For each fitting method the $p$-value of the Shapiro-Wilk test is indicated, which is in all cases smaller than the machine precision of 2.2e-16.