Mathematical Statistics I · Likelihood, score, and Fisher information
Bernoulli information identities hold, then the bound can vanish
Problem
Let $X_1,\ldots,X_n$ be independent $\operatorname{Bernoulli}(p)$ observations with $0<p<1$ unknown, and write $S=\sum_{i=1}^n X_i$. Starting from the joint log-likelihood, derive the score $U_n(p)$. Prove the three regular identities $$ \mathbb E_p[U_n(p)]=0, \qquad I_n(p)=\operatorname{Var}_p(U_n(p)) =-\mathbb E_p[\partial_p U_n(p)], $$ and compute $I_n(p)$ explicitly. Deduce the Cramér–Rao lower bound for every unbiased estimator of $p$, and show that $\bar X$ attains it. Now let $\tau(p)=p(1-p)$. Compute $\tau'(p)$ and the Cramér–Rao expression for unbiased estimators of $\tau(p)$. Evaluate that expression at $p=1/2$ and explain why a vanishing bound does not produce a zero-variance estimator of $1/4$. Contrast this regular, fixed-support calculation with a model in which the support moves with the parameter: the identities above need not hold there, but do not re-derive a shifted-exponential example. Do not construct a most-powerful test of $H_0:p=p_0$.
Hint
The log-likelihood is $S\log p+(n-S)\log(1-p)$. Differentiate once and twice.
Check your work
Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.
The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Mathematical Statistics I sample problem: Try the free sample problem.
More Mathematical Statistics I practice problems
- A parametric model is a collection of probability laws indexed by a parameterParametric models and sampling distributions
- For a family of densities or mass functions with respect to one fixed dominating…Sufficiency, completeness, and exponential families
- An iid Bernoulli sample of size n ≥ 1 contains K successesLikelihood, score, and Fisher information
- A hidden state θ is either zero or oneLoss, risk, and introductory decision theory
- A Cauchy mean is another Cauchy, so the average does not settleParametric models and sampling distributions
- A complete exponential sum yields the survival UMVUESufficiency, completeness, and exponential families
- The unbiased sample variance is not the MSE winnerBias, risk, consistency, and point estimation
- Invariance moves the MLE but not the UMVUEBias, risk, consistency, and point estimation
- A known-scale normal mean has a nonrandom interval lengthPivots and confidence sets