Let X be a fixed n×p matrix with linearly independent columns, and let y ∈ ℝⁿ

Let \(X\) be a fixed \(n\times p\) matrix with linearly independent columns, and let \(y\in\mathbb R^n\). Minimize \[ \operatorname{SSE}(b)=\|y-Xb\|^2 \] over \(b\in\mathbb R^p\). Derive the normal equations and prove that their solution is the unique minimizer. For \[ X= \begin{pmatrix} 1&-1\\ 1&0\\ 1&1 \end{pmatrix}, \qquad y= \begin{pmatrix}0\\1\\3\end{pmatrix}, \] calculate the coefficient estimate, fitted values, residuals, and residual sum of squares. Check the normal equations directly from the residual vector.

Expand the squared norm as a quadratic in the coefficient vector.