Mathematical Statistics II · Linear models and projection geometry
All random variables below are square-integrable and centered: 𝔼[x]=𝔼[z]=𝔼[y]=0
Problem
All random variables below are square-integrable and centered: $\mathbb{E}[x]=\mathbb{E}[z]=\mathbb{E}[y]=0$. The structural equation is \[ y=\beta_1 x+\beta_2 z+u, \qquad \mathbb{E}[u\mid x,z]=0. \] A researcher regresses $y$ on $x$ alone and forms the population (and, by the WLLN, probability-limit) slope \[ \mathrm{plim}\,\hat\beta_1 =\frac{\mathrm{Cov}(x,y)}{\mathrm{Var}(x)}. \] 1. Prove the omitted-variable identity \[ \mathrm{plim}\,\hat\beta_1 =\beta_1+\beta_2\frac{\mathrm{Cov}(x,z)}{\mathrm{Var}(x)}. \] 2. Specialize to $\beta_1=2$, $\beta_2=3$, $\mathrm{Var}(x)=4$, and $\mathrm{Cov}(x,z)=2$. Prove \[ \mathrm{plim}\,\hat\beta_1 =\frac72. \] 3. The same covariance is realized by the linear projection $z=\tfrac12 x+v$ with $\mathrm{Cov}(x,v)=0$. Prove that this representation reproduces $\mathrm{Cov}(x,z)=2$ and the same probability limit $7/2$. 4. Prove that the bias term vanishes if $\beta_2=0$ or $\mathrm{Cov}(x,z)=0$, and that the sign of the bias equals the sign of the product $\beta_2\,\mathrm{Cov}(x,z)$ (for $\mathrm{Var}(x)>0$). 5. Audit both sentences: (i) “the probability limit is $\beta_1+\beta_2\mathrm{Var}(x)/\mathrm{Cov}(x,z)$, or $\beta_1+\beta_2\mathrm{Cov}(x,z)$, giving $14$ or $8$”; (ii) “bias remains when $\beta_2=0$, or the sign follows $\beta_1$ alone, or $z=\tfrac12 x+v$ changes the limit, or this is a WLS / ridge exercise.” Stay inside the centered linear structural equation with $\mathbb{E}[u\mid x,z]=0$. Do not introduce an intercept estimation problem, and do not replace the omitted-variable identity by a ridge shrinkage.
Hint
$\mathrm{Cov}(x,y)=\beta_1\mathrm{Var}(x)+\beta_2\mathrm{Cov}(x,z)$ because $\mathrm{Cov}(x,u)=0$. Divide by $\mathrm{Var}(x)$.
Check your work
Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.
The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Mathematical Statistics II sample problem: Try the free sample problem.
More Mathematical Statistics II practice problems
- A multiple-testing procedure makes R rejections, of which V are rejections of true null…Neyman-Pearson testing and power
- A list can have a controlled expected fraction of false discoveries while a shorter…Neyman-Pearson testing and power
- Let X₁,X₂,… be iid with mean μ and variance 0<σ²<∞Consistency, asymptotic normality, and delta methods
- Let X₁,…,Xₙ be iid N(θ²,1), with parameter θ ≥ 0Consistency, asymptotic normality, and delta methods
- A categorical probability vector (P₁,P₂,P₃) has a Dirichlet prior with parameters (1,2,3)Bayesian inference and decision rules
- Choosing a model and then reporting its conditional credible interval can discard a…Bayesian inference and decision rules
- Let X be a fixed n×p matrix with linearly independent columns, and let y ∈ ℝⁿLinear models and projection geometry
- Let X₁,…,Xₙ be iid from an arbitrary continuous distributionRank and distribution-free methods
- The sample is x=(1,2,4) with n=3Bootstrap, permutation, and simulation-based inference