Information Theory and Digital Communications · Discrete channels, mutual information, and capacity
Let X → Z → Y be the cascade of two independent binary symmetric channels: X → Z has…
Problem
Information Theory and Digital Communications practice problemLet be the cascade of two independent binary symmetric channels: has crossover and has crossover . The input is uniform on . Write for binary entropy. The data-processing inequality asserts and along this Markov chain.
1. Prove that the composition is itself a BSC, and compute its
crossoveras a reduced fraction.
2. Compute for the uniform input, in the form .
Also record and in the same shape.
3. Compare with and on the interval
, and use the increase of on that interval to give a strict data-processing audit: .
4. Audit both sentences: (i) “the cascade crossover is ,
or else the product ”; (ii) “data processing is an equality, so , or else the mutual informations add.”
Do not replace the cascade by a single BEC, and do not compute an information density in place of the equivalent crossover.
Hint
HintY differs from X when exactly one of the two hops flips. That event has probability (1/10)(4/5)+(9/10)(1/5).
Check your work
Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.
The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Information Theory and Digital Communications sample problem: Try the free sample problem.
More Information Theory and Digital Communications practice problems
- Let X → Y be a binary symmetric channel of crossover p=1/4, and let X be uniform on {0,1}Discrete channels, mutual information, and capacity