Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Induced replication and the assessment of models

Heather Battey, Nancy Reid

Abstract

We study the assessment of semiparametric and other highly-parametrised models from the perspective of foundational principles of parametric statistical inference. In doing so, we highlight the possibility of avoiding the usual semiparametric considerations, which typically require estimation of nuisance components through kernel smoothing or basis expansion, with the associated difficulties of tuning-parameter choice that blur the distinction between estimation and model assessment. A key aspect is the availability of preliminary manoeuvres that induce an internal replication of known form under the postulated model. This can be cast as a generalised version of the Fisherian sufficiency/co-sufficiency separation, replacing out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error. Framed in this light are new methodological contributions in multiple example settings, including model assessment for the proportional hazards model, for a time-dependent Poisson process with semiparametric intensity function, and for matched-pair and two-group examples. Also subsumed within the framework is a post-reduction inference approach to the construction of confidence sets of sparse regression models. Numerical work confirms recovery of nominal error rates under the postulated model and high sensitivity to departures in the direction of semiparametric alternatives. We conclude by emphasising open challenges and unifying perspectives.

Induced replication and the assessment of models

Abstract

We study the assessment of semiparametric and other highly-parametrised models from the perspective of foundational principles of parametric statistical inference. In doing so, we highlight the possibility of avoiding the usual semiparametric considerations, which typically require estimation of nuisance components through kernel smoothing or basis expansion, with the associated difficulties of tuning-parameter choice that blur the distinction between estimation and model assessment. A key aspect is the availability of preliminary manoeuvres that induce an internal replication of known form under the postulated model. This can be cast as a generalised version of the Fisherian sufficiency/co-sufficiency separation, replacing out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error. Framed in this light are new methodological contributions in multiple example settings, including model assessment for the proportional hazards model, for a time-dependent Poisson process with semiparametric intensity function, and for matched-pair and two-group examples. Also subsumed within the framework is a post-reduction inference approach to the construction of confidence sets of sparse regression models. Numerical work confirms recovery of nominal error rates under the postulated model and high sensitivity to departures in the direction of semiparametric alternatives. We conclude by emphasising open challenges and unifying perspectives.

Paper Structure

This paper contains 29 sections, 2 theorems, 59 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Broad formulation
  3. Joint assessment of a model and its interest parameters
  4. Two parallel analyses
  5. Model assessment using a null-consistent estimator of $\psi^*$
  6. Falsifiable internal replication
  7. Non-matched examples
  8. Introduction
  9. Semiparametric time-dependent Poisson process
  10. Post-reduction confidence sets of models
  11. Proportional hazards and conditional co-sufficiency
  12. Matched-pair and two-group examples
  13. Inferential structures for matched-pair examples
  14. Some local power analyses
  15. Extension to unbalanced strata
  16. ...and 14 more sections

Key Result

Proposition 2.1

Let $Z_j$ be any continuous random variable for which $U_j(\psi)=g(Z_j, \psi)$ is standard uniformly distributed for some unknown value of $\psi$ under the postulated model. Let $\hat{\psi}$ be an estimator satisfying $\hat{\psi} \rightarrow_p \psi_0^*$ as some $n\rightarrow \infty$, where $\psi_0^*

Figures (1)

  • Figure 1: Logarithm of $\mathbb{E}(R_j)$ and $\mathbb{V}(R_j)$ calculated from \ref{['eqUMisspec']} and \ref{['eqIntParts']}, showing how the sensitivity to detect model misspecification varies with $\varsigma$ and $\psi^*$. Dashed lines indicate regions of the parameter space for the true Weibull model where $\mathbb{E}(R_j)$ and $\mathbb{V}(R_j)$ coincide with their putative values under the hypothesised exponential model.

Theorems & Definitions (12)

  • Example 1.1
  • Proposition 2.1
  • Remark 2.1
  • Example 4.1
  • Example 4.2
  • Proposition 4.1
  • proof
  • Example 4.3
  • Example 4.4
  • Example 4.5
  • ...and 2 more