Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To ignore dependencies is perhaps not a sin

Douglas P. Wiens

TL;DR

The paper analyzes minimax robustness of ordinary least squares (OLS) among generalized least squares (GLS) estimates under potential dependence and heteroscedasticity. A key lemma shows that any Loewner-order–increasing loss over covariance structures attains its worst case at $C=\\eta^2 I_n$, making independence the least favorable form of dependence and driving OLS to be minimax in many settings. The authors extend the analysis to misspecified response models, showing OLS is minimax under uniform designs but not universally; they develop minimax designs (often uniform on their support, achieved by clustering replicates) and demonstrate that these designs, together with minimax precision matrices, yield robust inference. They also explore continuous design spaces, arguing that randomized densities near I-optimal points preserve OLS minimaxity, and provide simulations and theoretical complements to guide robust experimental design under possible dependencies. Overall, the work provides a principled case for the robustness of OLS under broad covariance-structure uncertainty and offers design strategies to enforce minimax performance in practice.

Abstract

We present a result according to which certain functions of covariance matrices are maximized at scalar multiples of the identity matrix. In a statistical context in which such functions measure loss, this says that the least favourable form of dependence is in fact independence, so that a procedure optimal for i.i.d.\ data can be minimax. In particular, the ordinary least squares (\textsc{ols}) estimate of a correctly specified regression response is minimax among generalized least squares (\textsc{gls}) estimates, when the maximum is taken over certain classes of error covariance structures and the loss function possesses a natural monotonicity property. An implication is that it can be not only safe, but optimal to ignore such departures from the usual assumption of i.i.d.\ errors. We then consider regression models in which the response function is possibly misspecified, and show that \textsc{ols} is minimax if the design is uniform on its support, but that this often fails otherwise. We go on to investigate the interplay between minimax \textsc{gls} procedures and minimax designs, leading us to extend, to robustness against dependencies, an existing observation -- that robustness against model misspecifications is increased by splitting replicates into clusters of observations at nearby locations.

To ignore dependencies is perhaps not a sin

TL;DR

The paper analyzes minimax robustness of ordinary least squares (OLS) among generalized least squares (GLS) estimates under potential dependence and heteroscedasticity. A key lemma shows that any Loewner-order–increasing loss over covariance structures attains its worst case at , making independence the least favorable form of dependence and driving OLS to be minimax in many settings. The authors extend the analysis to misspecified response models, showing OLS is minimax under uniform designs but not universally; they develop minimax designs (often uniform on their support, achieved by clustering replicates) and demonstrate that these designs, together with minimax precision matrices, yield robust inference. They also explore continuous design spaces, arguing that randomized densities near I-optimal points preserve OLS minimaxity, and provide simulations and theoretical complements to guide robust experimental design under possible dependencies. Overall, the work provides a principled case for the robustness of OLS under broad covariance-structure uncertainty and offers design strategies to enforce minimax performance in practice.

Abstract

We present a result according to which certain functions of covariance matrices are maximized at scalar multiples of the identity matrix. In a statistical context in which such functions measure loss, this says that the least favourable form of dependence is in fact independence, so that a procedure optimal for i.i.d.\ data can be minimax. In particular, the ordinary least squares (\textsc{ols}) estimate of a correctly specified regression response is minimax among generalized least squares (\textsc{gls}) estimates, when the maximum is taken over certain classes of error covariance structures and the loss function possesses a natural monotonicity property. An implication is that it can be not only safe, but optimal to ignore such departures from the usual assumption of i.i.d.\ errors. We then consider regression models in which the response function is possibly misspecified, and show that \textsc{ols} is minimax if the design is uniform on its support, but that this often fails otherwise. We go on to investigate the interplay between minimax \textsc{gls} procedures and minimax designs, leading us to extend, to robustness against dependencies, an existing observation -- that robustness against model misspecifications is increased by splitting replicates into clusters of observations at nearby locations.
Paper Structure (8 sections, 4 theorems, 45 equations, 1 figure, 3 tables)

This paper contains 8 sections, 4 theorems, 45 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction and summary
  2. A useful lemma
  3. Generalized least squares regression estimates when the response is correctly specified
  4. Minimax precision matrices in misspecified response models
  5. Simulations
  6. Theoretical complements
  7. Minimax precision matrices and minimax designs
  8. Continuous design spaces

Key Result

Lemma 1

For $\eta ^{2}>0$, covariance matrix $C$ and induced norm $\left\Vert C\right\Vert _{M}$, define For the norm $\left\Vert \mathbf{\cdot }\right\Vert _{E}$ an equivalent definition is Then: (i) In any such class $\mathcal{C}_{M}$, $\max_{\mathcal{C}_{M}}\mathcal{L}\left( C\right) =\mathcal{L}\left( \eta ^{2}I_{n}\right)$. (ii) If $\mathcal{C}^{\prime }$$\mathcal{\subseteq C}_{M}$ and $\eta ^{2}I_{

Figures (1)

  • Figure 1: Minimax design frequencies for a cubic model; $n=20$.

Theorems & Definitions (12)

  • Lemma 1
  • Remark 1
  • Theorem 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Theorem 2
  • Remark 5
  • proof : Proof of Theorem \ref{['thm: maxima']}
  • Lemma 2
  • ...and 2 more