Skip to main content

Mathematical Statistics I · Likelihood, score, and Fisher information

Bernoulli information identities hold, then the bound can vanish

Problem

Let $X_1,\ldots,X_n$ be independent $\operatorname{Bernoulli}(p)$ observations with $0<p<1$ unknown, and write $S=\sum_{i=1}^n X_i$. Starting from the joint log-likelihood, derive the score $U_n(p)$. Prove the three regular identities $$ \mathbb E_p[U_n(p)]=0, \qquad I_n(p)=\operatorname{Var}_p(U_n(p)) =-\mathbb E_p[\partial_p U_n(p)], $$ and compute $I_n(p)$ explicitly. Deduce the Cramér–Rao lower bound for every unbiased estimator of $p$, and show that $\bar X$ attains it. Now let $\tau(p)=p(1-p)$. Compute $\tau'(p)$ and the Cramér–Rao expression for unbiased estimators of $\tau(p)$. Evaluate that expression at $p=1/2$ and explain why a vanishing bound does not produce a zero-variance estimator of $1/4$. Contrast this regular, fixed-support calculation with a model in which the support moves with the parameter: the identities above need not hold there, but do not re-derive a shifted-exponential example. Do not construct a most-powerful test of $H_0:p=p_0$.

Hint

The log-likelihood is $S\log p+(n-S)\log(1-p)$. Differentiate once and twice.

Check your work

Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.

The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Mathematical Statistics I sample problem: Try the free sample problem.

More Mathematical Statistics I practice problems