Topics in Probability, Parametric Estimation and Stochastic Calculus
Levi Lopes de Lima
TL;DR
The notes address core probability tools for parametric estimation, emphasizing a geometric view of Gaussian vectors and building a bridge from fundamentals to estimation methods such as MLE, least squares, GLMs, sufficiency, and hypothesis testing. The methodology leans on Fourier-analytic tools (characteristic functions) and extends to stochastic calculus via Brownian motion and Itô's formula, illustrating powerful applications including Gaussian concentration, the Feynman-Kac representation, and the Black-Scholes framework. The text also surveys high-dimensional probability and concentration phenomena, introducing Johnson-Lindenstrauss dimension reduction and phase-transition phenomena in Erdős-Rényi graphs, thereby connecting classical inference with modern data-science context. Overall, the work provides a cohesive, geometry-inspired treatment of probability and statistics, blending rigorous theory with practical probabilistic methods for inference and applications.
Abstract
We begin our journey by recalling the fundamentals of Probability Theory that underlie one of its most significant applications to real-world problems: Parametric Estimation. Throughout the text, we systematically develop this theme by presenting and discussing the main tools it encompasses (concentration inequalities, limit theorems, confidence intervals, maximum likelihood, least squares, and hypothesis testing) always with an eye toward both their theoretical underpinnings and practical relevance. While our approach follows the broad contours of conventional expositions, we depart from tradition by consistently exploring the geometric aspects of probability, particularly the invariance properties of normally distributed random vectors. This geometric perspective is taken further in an extended appendix, where we introduce the rudiments of Brownian motion and the corresponding stochastic calculus, culminating in Itô's celebrated change-of-variables formula. To highlight its scope and elegance, we present some of its most striking applications: the sharp Gaussian concentration inequality (a central example of the "concentration of measure phenomenon"), the Feynman-Kac formula (used to derive a path integral representation for the Laplacian heat kernel), and, as a concluding delicacy, the Black-Scholes strategy in Finance.