{"schemaVersion":"proofanvil.formula-drill-faces.v1","courseId":"probability-1","cards":[{"id":"axioms-counting/probability-spaces-and-axioms/probability-space","section":"Probability spaces and axioms","prompt":"

Probability space

","answer":"

A triple (Ω,F,P)(\\Omega, \\mathcal{F}, P)(Ω,F,P): a sample space Ω\\OmegaΩ of outcomes, a σ\\sigmaσ-algebra F\\mathcal{F}F of events (contains Ω\\OmegaΩ, closed under complements and countable unions), and a probability measure PPP on F\\mathcal{F}F.

"},{"id":"axioms-counting/probability-spaces-and-axioms/axioms-of-probability","section":"Probability spaces and axioms","prompt":"

Axioms of probability

","answer":"

(1) P(A)≥0P(A) \\ge 0P(A)≥0 for every event AAA. (2) P(Ω)=1P(\\Omega) = 1P(Ω)=1. (3) Countable additivity: if A1,A2,…A_1, A_2, \\dotsA1​,A2​,… are pairwise disjoint, then P(⋃iAi)=∑iP(Ai)P\\left(\\bigcup_i A_i\\right) = \\sum_i P(A_i)P(⋃i​Ai​)=∑i​P(Ai​).

"},{"id":"axioms-counting/probability-spaces-and-axioms/complement-rule","section":"Probability spaces and axioms","prompt":"

Complement rule

","answer":"
P(Ac)=1−P(A)P(A^c) = 1 - P(A)P(Ac)=1−P(A)

In particular P(∅)=0P(\\emptyset) = 0P(∅)=0.

"},{"id":"axioms-counting/probability-spaces-and-axioms/monotonicity","section":"Probability spaces and axioms","prompt":"

Monotonicity

","answer":"
A⊆B  ⟹  P(A)≤P(B)A \\subseteq B \\implies P(A) \\le P(B)A⊆B⟹P(A)≤P(B)
"},{"id":"axioms-counting/probability-spaces-and-axioms/inclusion-exclusion-two-events","section":"Probability spaces and axioms","prompt":"

Inclusion–exclusion (two events)

","answer":"
P(A∪B)=P(A)+P(B)−P(A∩B)P(A \\cup B) = P(A) + P(B) - P(A \\cap B)P(A∪B)=P(A)+P(B)−P(A∩B)
"},{"id":"axioms-counting/probability-spaces-and-axioms/inclusion-exclusion-n-events","section":"Probability spaces and axioms","prompt":"

Inclusion–exclusion (n events)

","answer":"
P(⋃i=1nAi)=∑k=1n(−1)k+1∑1≤i1<⋯<ik≤nP(Ai1∩⋯∩Aik)P\\Big(\\bigcup_{i=1}^n A_i\\Big) = \\sum_{k=1}^{n} (-1)^{k+1} \\sum_{1 \\le i_1 < \\cdots < i_k \\le n} P(A_{i_1} \\cap \\cdots \\cap A_{i_k})P(i=1⋃n​Ai​)=k=1∑n​(−1)k+11≤i1​<⋯<ik​≤n∑​P(Ai1​​∩⋯∩Aik​​)
"},{"id":"axioms-counting/probability-spaces-and-axioms/union-bound-boole-s-inequality","section":"Probability spaces and axioms","prompt":"

Union bound (Boole's inequality)

","answer":"
P(⋃iAi)≤∑iP(Ai)P\\Big(\\bigcup_{i} A_i\\Big) \\le \\sum_{i} P(A_i)P(i⋃​Ai​)≤i∑​P(Ai​)

Holds for finite or countable collections, disjoint or not.

"},{"id":"axioms-counting/probability-spaces-and-axioms/continuity-of-probability","section":"Probability spaces and axioms","prompt":"

Continuity of probability

","answer":"
A1⊆A2⊆⋯  ⟹  P(⋃nAn)=lim⁡n→∞P(An)A_1 \\subseteq A_2 \\subseteq \\cdots \\implies P\\Big(\\bigcup_{n} A_n\\Big) = \\lim_{n \\to \\infty} P(A_n)A1​⊆A2​⊆⋯⟹P(n⋃​An​)=n→∞lim​P(An​)

Likewise, if A1⊇A2⊇⋯A_1 \\supseteq A_2 \\supseteq \\cdotsA1​⊇A2​⊇⋯ then P(⋂nAn)=lim⁡nP(An)P\\left(\\bigcap_n A_n\\right) = \\lim_n P(A_n)P(⋂n​An​)=limn​P(An​).

"},{"id":"axioms-counting/counting/equally-likely-outcomes","section":"Counting","prompt":"

Equally likely outcomes

","answer":"
P(A)=∣A∣∣Ω∣P(A) = \\frac{|A|}{|\\Omega|}P(A)=∣Ω∣∣A∣​

Only when Ω\\OmegaΩ is finite and every outcome has the same probability.

"},{"id":"axioms-counting/counting/multiplication-principle","section":"Counting","prompt":"

Multiplication principle

","answer":"

If a procedure has rrr stages and stage jjj can be done in njn_jnj​ ways regardless of earlier choices, the whole procedure can be done in n1n2⋯nrn_1 n_2 \\cdots n_rn1​n2​⋯nr​ ways.

"},{"id":"axioms-counting/counting/ordered-with-replacement","section":"Counting","prompt":"

Ordered, with replacement

","answer":"
nkn^knk

Sequences of length kkk from nnn symbols.

"},{"id":"axioms-counting/counting/ordered-without-replacement-permutations","section":"Counting","prompt":"

Ordered, without replacement (permutations)

","answer":"
n(n−1)⋯(n−k+1)=n!(n−k)!n(n-1)\\cdots(n-k+1) = \\frac{n!}{(n-k)!}n(n−1)⋯(n−k+1)=(n−k)!n!​

With k=nk = nk=n this is n!n!n!, the number of orderings of nnn objects.

"},{"id":"axioms-counting/counting/unordered-without-replacement-combinations","section":"Counting","prompt":"

Unordered, without replacement (combinations)

","answer":"
(nk)=n!k! (n−k)!\\binom{n}{k} = \\frac{n!}{k!\\,(n-k)!}(kn​)=k!(n−k)!n!​
"},{"id":"axioms-counting/counting/unordered-with-replacement-stars-and-bars","section":"Counting","prompt":"

Unordered, with replacement (stars and bars)

","answer":"
(n+k−1k)\\binom{n+k-1}{k}(kn+k−1​)

Multisets of size kkk from nnn types; equivalently, solutions of x1+⋯+xn=kx_1 + \\cdots + x_n = kx1​+⋯+xn​=k in nonnegative integers.

"},{"id":"axioms-counting/counting/multinomial-coefficient","section":"Counting","prompt":"

Multinomial coefficient

","answer":"
(nn1,n2,…,nr)=n!n1! n2!⋯nr!,n1+⋯+nr=n\\binom{n}{n_1, n_2, \\dots, n_r} = \\frac{n!}{n_1!\\,n_2!\\cdots n_r!}, \\quad n_1 + \\cdots + n_r = n(n1​,n2​,…,nr​n​)=n1​!n2​!⋯nr​!n!​,n1​+⋯+nr​=n

Ways to split nnn labelled objects into groups of sizes n1,…,nrn_1,\\dots,n_rn1​,…,nr​.

"},{"id":"axioms-counting/counting/binomial-theorem","section":"Counting","prompt":"

Binomial theorem

","answer":"
(x+y)n=∑k=0n(nk)xkyn−k(x+y)^n = \\sum_{k=0}^{n} \\binom{n}{k} x^k y^{n-k}(x+y)n=k=0∑n​(kn​)xkyn−k
"},{"id":"axioms-counting/counting/vandermonde-s-identity","section":"Counting","prompt":"

Vandermonde's identity

","answer":"
(m+nk)=∑j=0k(mj)(nk−j)\\binom{m+n}{k} = \\sum_{j=0}^{k} \\binom{m}{j}\\binom{n}{k-j}(km+n​)=j=0∑k​(jm​)(k−jn​)
"},{"id":"conditioning/conditional-probability/conditional-probability","section":"Conditional probability","prompt":"

Conditional probability

","answer":"
P(A∣B)=P(A∩B)P(B),P(B)>0P(A \\mid B) = \\frac{P(A \\cap B)}{P(B)}, \\quad P(B) > 0P(A∣B)=P(B)P(A∩B)​,P(B)>0
"},{"id":"conditioning/conditional-probability/conditioning-gives-a-new-probability","section":"Conditional probability","prompt":"

Conditioning gives a new probability

","answer":"

For fixed BBB with P(B)>0P(B) > 0P(B)>0, the map A↦P(A∣B)A \\mapsto P(A \\mid B)A↦P(A∣B) satisfies all three axioms, so complements, inclusion–exclusion and the union bound all hold conditionally.

"},{"id":"conditioning/conditional-probability/multiplication-chain-rule","section":"Conditional probability","prompt":"

Multiplication (chain) rule

","answer":"
P(A1∩⋯∩An)=P(A1) P(A2∣A1)⋯P(An∣A1∩⋯∩An−1),P(A1∩⋯∩An−1)>0P(A_1 \\cap \\cdots \\cap A_n) = P(A_1)\\,P(A_2 \\mid A_1) \\cdots P(A_n \\mid A_1 \\cap \\cdots \\cap A_{n-1}), \\quad P(A_1 \\cap \\cdots \\cap A_{n-1}) > 0P(A1​∩⋯∩An​)=P(A1​)P(A2​∣A1​)⋯P(An​∣A1​∩⋯∩An−1​),P(A1​∩⋯∩An−1​)>0
"},{"id":"conditioning/conditional-probability/law-of-total-probability","section":"Conditional probability","prompt":"

Law of total probability

","answer":"
P(A)=∑iP(A∣Bi) P(Bi)P(A) = \\sum_{i} P(A \\mid B_i)\\,P(B_i)P(A)=i∑​P(A∣Bi​)P(Bi​)

B1,B2,…B_1, B_2, \\dotsB1​,B2​,… form a partition of Ω\\OmegaΩ (disjoint, union is Ω\\OmegaΩ) with each P(Bi)>0P(B_i) > 0P(Bi​)>0.

"},{"id":"conditioning/conditional-probability/bayes-theorem","section":"Conditional probability","prompt":"

Bayes' theorem

","answer":"
P(Bj∣A)=P(A∣Bj) P(Bj)∑iP(A∣Bi) P(Bi)P(B_j \\mid A) = \\frac{P(A \\mid B_j)\\,P(B_j)}{\\sum_{i} P(A \\mid B_i)\\,P(B_i)}P(Bj​∣A)=∑i​P(A∣Bi​)P(Bi​)P(A∣Bj​)P(Bj​)​

Same partition as above, and P(A)>0P(A) > 0P(A)>0.

"},{"id":"conditioning/independence/independent-events","section":"Independence","prompt":"

Independent events

","answer":"

AAA and BBB are independent if P(A∩B)=P(A)P(B)P(A \\cap B) = P(A)P(B)P(A∩B)=P(A)P(B). When P(B)>0P(B) > 0P(B)>0 this is the same as P(A∣B)=P(A)P(A \\mid B) = P(A)P(A∣B)=P(A).

"},{"id":"conditioning/independence/mutual-independence","section":"Independence","prompt":"

Mutual independence

","answer":"

Events A1,…,AnA_1, \\dots, A_nA1​,…,An​ are mutually independent if P(Ai1∩⋯∩Aik)=P(Ai1)⋯P(Aik)P(A_{i_1} \\cap \\cdots \\cap A_{i_k}) = P(A_{i_1}) \\cdots P(A_{i_k})P(Ai1​​∩⋯∩Aik​​)=P(Ai1​​)⋯P(Aik​​) for every subcollection of indices. Pairwise independence alone does not imply this.

"},{"id":"conditioning/independence/complements-keep-independence","section":"Independence","prompt":"

Complements keep independence

","answer":"
A⊥B  ⟹  A⊥Bc,    Ac⊥B,    Ac⊥BcA \\perp B \\implies A \\perp B^c, \\;\\; A^c \\perp B, \\;\\; A^c \\perp B^cA⊥B⟹A⊥Bc,Ac⊥B,Ac⊥Bc
"},{"id":"conditioning/independence/at-least-one-of-independent-events","section":"Independence","prompt":"

At least one of independent events

","answer":"
P(⋃i=1nAi)=1−∏i=1n(1−P(Ai))P\\Big(\\bigcup_{i=1}^n A_i\\Big) = 1 - \\prod_{i=1}^n \\big(1 - P(A_i)\\big)P(i=1⋃n​Ai​)=1−i=1∏n​(1−P(Ai​))

Valid when A1,…,AnA_1,\\dots,A_nA1​,…,An​ are mutually independent.

"},{"id":"conditioning/independence/conditional-independence","section":"Independence","prompt":"

Conditional independence

","answer":"

AAA and BBB are conditionally independent given CCC (with P(C)>0P(C) > 0P(C)>0) if P(A∩B∣C)=P(A∣C) P(B∣C)P(A \\cap B \\mid C) = P(A \\mid C)\\,P(B \\mid C)P(A∩B∣C)=P(A∣C)P(B∣C). It neither implies nor is implied by ordinary independence.

"},{"id":"conditioning/independence/disjoint-is-not-independent","section":"Independence","prompt":"

Disjoint is not independent

","answer":"

If A∩B=∅A \\cap B = \\emptysetA∩B=∅ and both have positive probability, then P(A∩B)=0≠P(A)P(B)P(A \\cap B) = 0 \\ne P(A)P(B)P(A∩B)=0=P(A)P(B), so they are dependent.

"},{"id":"random-variables/random-variables-and-distribution-functions/random-variable","section":"Random variables and distribution functions","prompt":"

Random variable

","answer":"

A function X:Ω→RX: \\Omega \\to \\mathbb{R}X:Ω→R with {X≤x}∈F\\{X \\le x\\} \\in \\mathcal{F}{X≤x}∈F for every real xxx, so probabilities of statements about XXX are defined.

"},{"id":"random-variables/random-variables-and-distribution-functions/cumulative-distribution-function","section":"Random variables and distribution functions","prompt":"

Cumulative distribution function

","answer":"

F(x)=P(X≤x)F(x) = P(X \\le x)F(x)=P(X≤x). Every CDF is nondecreasing, right-continuous, with F(x)→0F(x) \\to 0F(x)→0 as x→−∞x \\to -\\inftyx→−∞ and F(x)→1F(x) \\to 1F(x)→1 as x→∞x \\to \\inftyx→∞. The CDF determines the distribution.

"},{"id":"random-variables/random-variables-and-distribution-functions/probabilities-from-the-cdf","section":"Random variables and distribution functions","prompt":"

Probabilities from the CDF

","answer":"
P(a<X≤b)=F(b)−F(a),P(X=x)=F(x)−F(x−)P(a < X \\le b) = F(b) - F(a), \\qquad P(X = x) = F(x) - F(x^-)P(a<X≤b)=F(b)−F(a),P(X=x)=F(x)−F(x−)

F(x−)F(x^-)F(x−) is the left limit. For a continuous XXX, P(X=x)=0P(X = x) = 0P(X=x)=0 for every xxx.

"},{"id":"random-variables/random-variables-and-distribution-functions/pmf-and-pdf","section":"Random variables and distribution functions","prompt":"

pmf and pdf

","answer":"

Discrete: pmf p(x)=P(X=x)p(x) = P(X = x)p(x)=P(X=x) with ∑xp(x)=1\\sum_x p(x) = 1∑x​p(x)=1. Continuous: pdf f≥0f \\ge 0f≥0 with F(x)=∫−∞xf(u) duF(x) = \\int_{-\\infty}^{x} f(u)\\,duF(x)=∫−∞x​f(u)du, so F′(x)=f(x)F'(x) = f(x)F′(x)=f(x) wherever fff is continuous (and F′=fF' = fF′=f almost everywhere) and P(X∈A)=∫Af(x) dxP(X \\in A) = \\int_A f(x)\\,dxP(X∈A)=∫A​f(x)dx.

"},{"id":"random-variables/random-variables-and-distribution-functions/density-of-a-monotone-transformation","section":"Random variables and distribution functions","prompt":"

Density of a monotone transformation

","answer":"
fY(y)=fX(g−1(y))∣ddyg−1(y)∣f_Y(y) = f_X\\big(g^{-1}(y)\\big) \\left| \\frac{d}{dy} g^{-1}(y) \\right|fY​(y)=fX​(g−1(y))​dyd​g−1(y)​

Y=g(X)Y = g(X)Y=g(X) with ggg strictly monotone and differentiable on the support of XXX. Otherwise work through the CDF P(g(X)≤y)P(g(X) \\le y)P(g(X)≤y).

"},{"id":"random-variables/random-variables-and-distribution-functions/quantile-function-and-inverse-transform","section":"Random variables and distribution functions","prompt":"

Quantile function and inverse transform

","answer":"
F−1(u)=inf⁡{x:F(x)≥u},U∼Unif(0,1)  ⟹  F−1(U)∼FF^{-1}(u) = \\inf\\{x : F(x) \\ge u\\}, \\qquad U \\sim \\mathrm{Unif}(0,1) \\implies F^{-1}(U) \\sim FF−1(u)=inf{x:F(x)≥u},U∼Unif(0,1)⟹F−1(U)∼F

Conversely, if FFF is continuous then F(X)∼Unif(0,1)F(X) \\sim \\mathrm{Unif}(0,1)F(X)∼Unif(0,1).

"},{"id":"random-variables/random-variables-and-distribution-functions/memoryless-property","section":"Random variables and distribution functions","prompt":"

Memoryless property

","answer":"
P(X>s+t∣X>s)=P(X>t)P(X > s + t \\mid X > s) = P(X > t)P(X>s+t∣X>s)=P(X>t)

Holds for the Exponential (all s,t≥0s, t \\ge 0s,t≥0) and for the Geometric on {1,2,… }\\{1,2,\\dots\\}{1,2,…} (integers s,t≥0s, t \\ge 0s,t≥0). The Exponential is the only memoryless continuous law on (0,∞)(0,\\infty)(0,∞), and the Geometric the only memoryless law on {1,2,… }\\{1,2,\\dots\\}{1,2,…}.

"},{"id":"random-variables/common-discrete-distributions/bernoulli","section":"Common discrete distributions","prompt":"

Bernoulli

p∈[0,1]p \\in [0,1]p∈[0,1], the success probability.

","answer":"
pmf
pk(1−p)1−k\\displaystyle p^k (1-p)^{1-k}pk(1−p)1−k for k∈{0,1}k \\in \\{0,1\\}k∈{0,1}
Mean
p\\displaystyle pp
Variance
p(1−p)\\displaystyle p(1-p)p(1−p)
mgf
1−p+pet\\displaystyle 1 - p + pe^t1−p+pet
"},{"id":"random-variables/common-discrete-distributions/binomial","section":"Common discrete distributions","prompt":"

Binomial

n∈{1,2,… }n \\in \\{1,2,\\dots\\}n∈{1,2,…} trials, success probability p∈[0,1]p \\in [0,1]p∈[0,1]. Counts successes in nnn independent Bernoulli(ppp) trials.

","answer":"
pmf
(nk)pk(1−p)n−k\\displaystyle \\binom{n}{k} p^k (1-p)^{n-k}(kn​)pk(1−p)n−k for k∈{0,1,…,n}k \\in \\{0,1,\\dots,n\\}k∈{0,1,…,n}
Mean
np\\displaystyle npnp
Variance
np(1−p)\\displaystyle np(1-p)np(1−p)
mgf
(1−p+pet)n\\displaystyle (1 - p + pe^t)^n(1−p+pet)n
"},{"id":"random-variables/common-discrete-distributions/geometric","section":"Common discrete distributions","prompt":"

Geometric

p∈(0,1]p \\in (0,1]p∈(0,1]. Convention here: XXX is the number of trials up to and including the first success, so X∈{1,2,… }X \\in \\{1,2,\\dots\\}X∈{1,2,…}. (If you count failures before the first success instead, use X−1X-1X−1: mean (1−p)/p(1-p)/p(1−p)/p, same variance.)

","answer":"
pmf
p(1−p)k−1\\displaystyle p(1-p)^{k-1}p(1−p)k−1 for k∈{1,2,3,… }k \\in \\{1,2,3,\\dots\\}k∈{1,2,3,…}
Mean
1p\\displaystyle \\frac{1}{p}p1​
Variance
1−pp2\\displaystyle \\frac{1-p}{p^2}p21−p​
mgf
pet1−(1−p)et,  t<−ln⁡(1−p)\\displaystyle \\frac{pe^t}{1 - (1-p)e^t}, \\; t < -\\ln(1-p)1−(1−p)etpet​,t<−ln(1−p)
"},{"id":"random-variables/common-discrete-distributions/negative-binomial","section":"Common discrete distributions","prompt":"

Negative Binomial

r∈{1,2,… }r \\in \\{1,2,\\dots\\}r∈{1,2,…}, p∈(0,1]p \\in (0,1]p∈(0,1]. Convention here: XXX is the number of trials needed to get the rrr-th success, so X∈{r,r+1,… }X \\in \\{r, r+1, \\dots\\}X∈{r,r+1,…}. It is a sum of rrr independent Geometric(ppp) variables; r=1r=1r=1 gives the Geometric.

","answer":"
pmf
(k−1r−1)pr(1−p)k−r\\displaystyle \\binom{k-1}{r-1} p^r (1-p)^{k-r}(r−1k−1​)pr(1−p)k−r for k∈{r,r+1,r+2,… }k \\in \\{r, r+1, r+2, \\dots\\}k∈{r,r+1,r+2,…}
Mean
rp\\displaystyle \\frac{r}{p}pr​
Variance
r(1−p)p2\\displaystyle \\frac{r(1-p)}{p^2}p2r(1−p)​
mgf
(pet1−(1−p)et)r,  t<−ln⁡(1−p)\\displaystyle \\left(\\frac{pe^t}{1 - (1-p)e^t}\\right)^r, \\; t < -\\ln(1-p)(1−(1−p)etpet​)r,t<−ln(1−p)
"},{"id":"random-variables/common-discrete-distributions/poisson","section":"Common discrete distributions","prompt":"

Poisson

Rate λ>0\\lambda > 0λ>0 (expected count).

","answer":"
pmf
e−λλkk!\\displaystyle \\frac{e^{-\\lambda}\\lambda^k}{k!}k!e−λλk​ for k∈{0,1,2,… }k \\in \\{0,1,2,\\dots\\}k∈{0,1,2,…}
Mean
λ\\displaystyle \\lambdaλ
Variance
λ\\displaystyle \\lambdaλ
mgf
exp⁡(λ(et−1))\\displaystyle \\exp\\big(\\lambda(e^t - 1)\\big)exp(λ(et−1))
"},{"id":"random-variables/common-discrete-distributions/hypergeometric","section":"Common discrete distributions","prompt":"

Hypergeometric

Population of NNN items, KKK of them successes (0≤K≤N0 \\le K \\le N0≤K≤N); draw nnn items without replacement (0≤n≤N0 \\le n \\le N0≤n≤N, N≥2N \\ge 2N≥2 for the variance formula). XXX counts successes drawn.

","answer":"
pmf
(Kk)(N−Kn−k)(Nn)\\displaystyle \\frac{\\binom{K}{k}\\binom{N-K}{n-k}}{\\binom{N}{n}}(nN​)(kK​)(n−kN−K​)​ for k∈{max⁡(0, n−N+K),…,min⁡(n,K)}k \\in \\{\\max(0,\\,n-N+K), \\dots, \\min(n,K)\\}k∈{max(0,n−N+K),…,min(n,K)}
Mean
nKN\\displaystyle \\frac{nK}{N}NnK​
Variance
n KN(1−KN)N−nN−1\\displaystyle n\\,\\frac{K}{N}\\left(1 - \\frac{K}{N}\\right)\\frac{N-n}{N-1}nNK​(1−NK​)N−1N−n​
mgf

Exists (finite support) but has no simple closed form; it is not used. Note the finite-population factor N−nN−1\\frac{N-n}{N-1}N−1N−n​ in the variance compared with the Binomial.

"},{"id":"random-variables/common-discrete-distributions/discrete-uniform","section":"Common discrete distributions","prompt":"

Discrete Uniform

Integers a≤ba \\le ba≤b; each of the b−a+1b-a+1b−a+1 values is equally likely.

","answer":"
pmf
1b−a+1\\displaystyle \\frac{1}{b-a+1}b−a+11​ for k∈{a,a+1,…,b}k \\in \\{a, a+1, \\dots, b\\}k∈{a,a+1,…,b}
Mean
a+b2\\displaystyle \\frac{a+b}{2}2a+b​
Variance
(b−a+1)2−112\\displaystyle \\frac{(b-a+1)^2 - 1}{12}12(b−a+1)2−1​
mgf
eat−e(b+1)t(b−a+1)(1−et),  t≠0    (M(0)=1)\\displaystyle \\frac{e^{at} - e^{(b+1)t}}{(b-a+1)(1 - e^t)}, \\; t \\ne 0 \\;\\; (M(0) = 1)(b−a+1)(1−et)eat−e(b+1)t​,t=0(M(0)=1)
"},{"id":"random-variables/common-continuous-distributions/continuous-uniform","section":"Common continuous distributions","prompt":"

Continuous Uniform

Endpoints a<ba < ba<b.

","answer":"
pdf
1b−a\\displaystyle \\frac{1}{b-a}b−a1​ for x∈[a,b]x \\in [a, b]x∈[a,b]
Mean
a+b2\\displaystyle \\frac{a+b}{2}2a+b​
Variance
(b−a)212\\displaystyle \\frac{(b-a)^2}{12}12(b−a)2​
mgf
etb−etat(b−a),  t≠0    (M(0)=1)\\displaystyle \\frac{e^{tb} - e^{ta}}{t(b-a)}, \\; t \\ne 0 \\;\\; (M(0) = 1)t(b−a)etb−eta​,t=0(M(0)=1)
"},{"id":"random-variables/common-continuous-distributions/exponential","section":"Common continuous distributions","prompt":"

Exponential

RATE λ>0\\lambda > 0λ>0 (so the mean is 1/λ1/\\lambda1/λ). Same as Gamma with shape 111 and rate λ\\lambdaλ.

","answer":"
pdf
λe−λx\\displaystyle \\lambda e^{-\\lambda x}λe−λx for x≥0x \\ge 0x≥0
Mean
1λ\\displaystyle \\frac{1}{\\lambda}λ1​
Variance
1λ2\\displaystyle \\frac{1}{\\lambda^2}λ21​
mgf
λλ−t,  t<λ\\displaystyle \\frac{\\lambda}{\\lambda - t}, \\; t < \\lambdaλ−tλ​,t<λ
"},{"id":"random-variables/common-continuous-distributions/gamma","section":"Common continuous distributions","prompt":"

Gamma

Shape α>0\\alpha > 0α>0 and RATE β>0\\beta > 0β>0 (scale =1/β= 1/\\beta=1/β). A sum of nnn independent Exponential(λ\\lambdaλ) variables is Gamma(nnn, λ\\lambdaλ).

","answer":"
pdf
βαΓ(α)xα−1e−βx\\displaystyle \\frac{\\beta^\\alpha}{\\Gamma(\\alpha)} x^{\\alpha-1} e^{-\\beta x}Γ(α)βα​xα−1e−βx for x>0x > 0x>0
Mean
αβ\\displaystyle \\frac{\\alpha}{\\beta}βα​
Variance
αβ2\\displaystyle \\frac{\\alpha}{\\beta^2}β2α​
mgf
(ββ−t)α,  t<β\\displaystyle \\left(\\frac{\\beta}{\\beta - t}\\right)^{\\alpha}, \\; t < \\beta(β−tβ​)α,t<β
"},{"id":"random-variables/common-continuous-distributions/beta","section":"Common continuous distributions","prompt":"

Beta

Shapes α>0\\alpha > 0α>0, β>0\\beta > 0β>0. Here B(α,β)=Γ(α)Γ(β)Γ(α+β)B(\\alpha,\\beta) = \\frac{\\Gamma(\\alpha)\\Gamma(\\beta)}{\\Gamma(\\alpha+\\beta)}B(α,β)=Γ(α+β)Γ(α)Γ(β)​. Beta(1,11,11,1) is Uniform(0,10,10,1).

","answer":"
pdf
xα−1(1−x)β−1B(α,β)\\displaystyle \\frac{x^{\\alpha-1}(1-x)^{\\beta-1}}{B(\\alpha,\\beta)}B(α,β)xα−1(1−x)β−1​ for x∈(0,1)x \\in (0, 1)x∈(0,1)
Mean
αα+β\\displaystyle \\frac{\\alpha}{\\alpha+\\beta}α+βα​
Variance
αβ(α+β)2(α+β+1)\\displaystyle \\frac{\\alpha\\beta}{(\\alpha+\\beta)^2(\\alpha+\\beta+1)}(α+β)2(α+β+1)αβ​
mgf

Exists for every ttt (bounded support) but has no elementary closed form; use moments instead: E[Xk]=∏r=0k−1α+rα+β+rE[X^k] = \\prod_{r=0}^{k-1} \\frac{\\alpha + r}{\\alpha + \\beta + r}E[Xk]=∏r=0k−1​α+β+rα+r​.

"},{"id":"random-variables/common-continuous-distributions/normal","section":"Common continuous distributions","prompt":"

Normal

Mean μ∈R\\mu \\in \\mathbb{R}μ∈R and VARIANCE σ2>0\\sigma^2 > 0σ2>0, written N(μ,σ2)N(\\mu, \\sigma^2)N(μ,σ2). Standardize with Z=(X−μ)/σ∼N(0,1)Z = (X-\\mu)/\\sigma \\sim N(0,1)Z=(X−μ)/σ∼N(0,1).

","answer":"
pdf
12πσ2exp⁡(−(x−μ)22σ2)\\displaystyle \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp\\left(-\\frac{(x-\\mu)^2}{2\\sigma^2}\\right)2πσ2​1​exp(−2σ2(x−μ)2​) for x∈Rx \\in \\mathbb{R}x∈R
Mean
μ\\displaystyle \\muμ
Variance
σ2\\displaystyle \\sigma^2σ2
mgf
exp⁡(μt+σ2t22)\\displaystyle \\exp\\left(\\mu t + \\frac{\\sigma^2 t^2}{2}\\right)exp(μt+2σ2t2​)
"},{"id":"expectation/expectation-and-moments/expectation","section":"Expectation and moments","prompt":"

Expectation

","answer":"
E[X]=∑xx p(x)orE[X]=∫−∞∞xf(x) dxE[X] = \\sum_{x} x\\,p(x) \\quad \\text{or} \\quad E[X] = \\int_{-\\infty}^{\\infty} x f(x)\\,dxE[X]=x∑​xp(x)orE[X]=∫−∞∞​xf(x)dx

Defined when the sum or integral converges absolutely, that is E∣X∣<∞E|X| < \\inftyE∣X∣<∞.

"},{"id":"expectation/expectation-and-moments/expectation-of-a-function-lotus","section":"Expectation and moments","prompt":"

Expectation of a function (LOTUS)

","answer":"
E[g(X)]=∑xg(x) p(x)orE[g(X)]=∫−∞∞g(x)f(x) dxE[g(X)] = \\sum_{x} g(x)\\,p(x) \\quad \\text{or} \\quad E[g(X)] = \\int_{-\\infty}^{\\infty} g(x) f(x)\\,dxE[g(X)]=x∑​g(x)p(x)orE[g(X)]=∫−∞∞​g(x)f(x)dx

No need to find the distribution of g(X)g(X)g(X) first. Valid when E∣g(X)∣<∞E|g(X)| < \\inftyE∣g(X)∣<∞ (or g≥0g \\ge 0g≥0).

"},{"id":"expectation/expectation-and-moments/linearity","section":"Expectation and moments","prompt":"

Linearity

","answer":"
E[aX+bY+c]=a E[X]+b E[Y]+cE[aX + bY + c] = a\\,E[X] + b\\,E[Y] + cE[aX+bY+c]=aE[X]+bE[Y]+c

Always true when the expectations exist; independence is not needed.

"},{"id":"expectation/expectation-and-moments/indicators-method-of-indicators","section":"Expectation and moments","prompt":"

Indicators (method of indicators)

","answer":"
E[1A]=P(A),X=∑i=1n1Ai  ⟹  E[X]=∑i=1nP(Ai)E[\\mathbf{1}_A] = P(A), \\qquad X = \\sum_{i=1}^{n} \\mathbf{1}_{A_i} \\implies E[X] = \\sum_{i=1}^{n} P(A_i)E[1A​]=P(A),X=i=1∑n​1Ai​​⟹E[X]=i=1∑n​P(Ai​)

1A\\mathbf{1}_A1A​ is 111 on the event AAA and 000 off it. By linearity the AiA_iAi​ need not be independent or disjoint. Example: a uniformly random permutation of {1,…,n}\\{1,\\dots,n\\}{1,…,n} has on average n⋅1n=1n \\cdot \\tfrac{1}{n} = 1n⋅n1​=1 fixed point.

"},{"id":"expectation/expectation-and-moments/tail-sum-formulas","section":"Expectation and moments","prompt":"

Tail-sum formulas

","answer":"
E[X]=∫0∞P(X>x) dx    (X≥0),E[X]=∑k=1∞P(X≥k)    (X∈{0,1,2,… })E[X] = \\int_0^\\infty P(X > x)\\,dx \\;\\; (X \\ge 0), \\qquad E[X] = \\sum_{k=1}^{\\infty} P(X \\ge k) \\;\\; (X \\in \\{0,1,2,\\dots\\})E[X]=∫0∞​P(X>x)dx(X≥0),E[X]=k=1∑∞​P(X≥k)(X∈{0,1,2,…})
"},{"id":"expectation/expectation-and-moments/variance","section":"Expectation and moments","prompt":"

Variance

","answer":"
Var⁡(X)=E[(X−μ)2]=E[X2]−(E[X])2\\operatorname{Var}(X) = E\\big[(X - \\mu)^2\\big] = E[X^2] - (E[X])^2Var(X)=E[(X−μ)2]=E[X2]−(E[X])2

μ=E[X]\\mu = E[X]μ=E[X]. The standard deviation is σ=Var⁡(X)\\sigma = \\sqrt{\\operatorname{Var}(X)}σ=Var(X)​.

"},{"id":"expectation/expectation-and-moments/variance-of-a-linear-function","section":"Expectation and moments","prompt":"

Variance of a linear function

","answer":"
Var⁡(aX+b)=a2Var⁡(X)\\operatorname{Var}(aX + b) = a^2 \\operatorname{Var}(X)Var(aX+b)=a2Var(X)
"},{"id":"expectation/expectation-and-moments/moments","section":"Expectation and moments","prompt":"

Moments

","answer":"

The kkk-th moment is E[Xk]E[X^k]E[Xk] and the kkk-th central moment is E[(X−μ)k]E[(X-\\mu)^k]E[(X−μ)k]. Skewness is E[(X−μ)3]/σ3E[(X-\\mu)^3]/\\sigma^3E[(X−μ)3]/σ3. If E∣X∣k<∞E|X|^k < \\inftyE∣X∣k<∞ then all lower moments are finite too.

"},{"id":"expectation/expectation-and-moments/product-of-independent-variables","section":"Expectation and moments","prompt":"

Product of independent variables

","answer":"
X⊥Y  ⟹  E[XY]=E[X] E[Y]X \\perp Y \\implies E[XY] = E[X]\\,E[Y]X⊥Y⟹E[XY]=E[X]E[Y]

More generally E[g(X)h(Y)]=E[g(X)] E[h(Y)]E[g(X)h(Y)] = E[g(X)]\\,E[h(Y)]E[g(X)h(Y)]=E[g(X)]E[h(Y)] for independent X,YX, YX,Y.

"},{"id":"expectation/transforms/moment-generating-function","section":"Transforms","prompt":"

Moment generating function

","answer":"
MX(t)=E[etX]M_X(t) = E\\big[e^{tX}\\big]MX​(t)=E[etX]

Most useful when finite for all ttt in an open interval around 000.

"},{"id":"expectation/transforms/moments-from-the-mgf","section":"Transforms","prompt":"

Moments from the mgf

","answer":"
E[Xk]=MX(k)(0)=dkdtkMX(t)∣t=0,MX finite on an open interval around 0E[X^k] = M_X^{(k)}(0) = \\frac{d^k}{dt^k} M_X(t)\\Big|_{t=0}, \\quad M_X \\text{ finite on an open interval around } 0E[Xk]=MX(k)​(0)=dtkdk​MX​(t)​t=0​,MX​ finite on an open interval around 0
"},{"id":"expectation/transforms/uniqueness-of-mgfs","section":"Transforms","prompt":"

Uniqueness of mgfs

","answer":"

If MX(t)=MY(t)M_X(t) = M_Y(t)MX​(t)=MY​(t) for all ttt in an open interval around 000 (both finite there), then XXX and YYY have the same distribution.

"},{"id":"expectation/transforms/mgf-rules","section":"Transforms","prompt":"

Mgf rules

","answer":"
MaX+b(t)=ebtMX(at),X⊥Y  ⟹  MX+Y(t)=MX(t) MY(t)M_{aX+b}(t) = e^{bt} M_X(at), \\qquad X \\perp Y \\implies M_{X+Y}(t) = M_X(t)\\,M_Y(t)MaX+b​(t)=ebtMX​(at),X⊥Y⟹MX+Y​(t)=MX​(t)MY​(t)
"},{"id":"expectation/transforms/probability-generating-function","section":"Transforms","prompt":"

Probability generating function

","answer":"
GX(s)=E[sX]=∑k≥0P(X=k) sk,P(X=k)=GX(k)(0)k!G_X(s) = E\\big[s^X\\big] = \\sum_{k \\ge 0} P(X = k)\\,s^k, \\quad P(X = k) = \\frac{G_X^{(k)}(0)}{k!}GX​(s)=E[sX]=k≥0∑​P(X=k)sk,P(X=k)=k!GX(k)​(0)​

For X∈{0,1,2,… }X \\in \\{0,1,2,\\dots\\}X∈{0,1,2,…}. Also GX′(1)=E[X]G_X'(1) = E[X]GX′​(1)=E[X] and GX′′(1)=E[X(X−1)]G_X''(1) = E[X(X-1)]GX′′​(1)=E[X(X−1)], and MX(t)=GX(et)M_X(t) = G_X(e^t)MX​(t)=GX​(et).

"},{"id":"expectation/transforms/characteristic-function","section":"Transforms","prompt":"

Characteristic function

","answer":"
φX(t)=E[eitX],∣φX(t)∣≤1\\varphi_X(t) = E\\big[e^{itX}\\big], \\quad |\\varphi_X(t)| \\le 1φX​(t)=E[eitX],∣φX​(t)∣≤1

Exists for every XXX and every real ttt, and determines the distribution. If E∣X∣k<∞E|X|^k < \\inftyE∣X∣k<∞ then E[Xk]=i−kφX(k)(0)E[X^k] = i^{-k}\\varphi_X^{(k)}(0)E[Xk]=i−kφX(k)​(0).

"},{"id":"expectation/inequalities-for-expectations/jensen-s-inequality","section":"Inequalities for expectations","prompt":"

Jensen's inequality

","answer":"
g convex  ⟹  g(E[X])≤E[g(X)]g \\text{ convex} \\implies g\\big(E[X]\\big) \\le E\\big[g(X)\\big]g convex⟹g(E[X])≤E[g(X)]

Reversed for concave ggg. Example: (E[X])2≤E[X2](E[X])^2 \\le E[X^2](E[X])2≤E[X2].

"},{"id":"expectation/inequalities-for-expectations/cauchy-schwarz-inequality","section":"Inequalities for expectations","prompt":"

Cauchy–Schwarz inequality

","answer":"
(E[XY])2≤E[X2] E[Y2]\\big(E[XY]\\big)^2 \\le E[X^2]\\,E[Y^2](E[XY])2≤E[X2]E[Y2]
"},{"id":"joint-laws/joint-marginal-and-conditional-laws/joint-pmf-and-pdf","section":"Joint, marginal and conditional laws","prompt":"

Joint pmf and pdf

","answer":"

Discrete: p(x,y)=P(X=x,Y=y)p(x,y) = P(X = x, Y = y)p(x,y)=P(X=x,Y=y). Continuous: f(x,y)≥0f(x,y) \\ge 0f(x,y)≥0 with P((X,Y)∈A)=∬Af(x,y) dx dyP((X,Y) \\in A) = \\iint_A f(x,y)\\,dx\\,dyP((X,Y)∈A)=∬A​f(x,y)dxdy for every (nice) region AAA, and total integral 111.

"},{"id":"joint-laws/joint-marginal-and-conditional-laws/marginals","section":"Joint, marginal and conditional laws","prompt":"

Marginals

","answer":"
fX(x)=∫−∞∞f(x,y) dy,pX(x)=∑yp(x,y)f_X(x) = \\int_{-\\infty}^{\\infty} f(x,y)\\,dy, \\qquad p_X(x) = \\sum_{y} p(x,y)fX​(x)=∫−∞∞​f(x,y)dy,pX​(x)=y∑​p(x,y)
"},{"id":"joint-laws/joint-marginal-and-conditional-laws/independence-criterion","section":"Joint, marginal and conditional laws","prompt":"

Independence criterion

","answer":"
X⊥Y  ⟺  f(x,y)=fX(x) fY(y) for almost every (x,y)X \\perp Y \\iff f(x,y) = f_X(x)\\,f_Y(y) \\text{ for almost every } (x, y)X⊥Y⟺f(x,y)=fX​(x)fY​(y) for almost every (x,y)

For densities, equality is only needed almost everywhere: a joint density may be changed on a set of area zero. For pmfs it must hold at every point, p(x,y)=pX(x) pY(y)p(x,y) = p_X(x)\\,p_Y(y)p(x,y)=pX​(x)pY​(y). Equivalently the joint density factors as g(x)h(y)g(x)h(y)g(x)h(y) almost everywhere. A support that differs from a product set by more than a set of area zero (like 0<x<y0 < x < y0<x<y) rules out independence; a mismatch on a set of area zero does not.

"},{"id":"joint-laws/joint-marginal-and-conditional-laws/conditional-density","section":"Joint, marginal and conditional laws","prompt":"

Conditional density

","answer":"
fY∣X(y∣x)=f(x,y)fX(x),fX(x)>0f_{Y \\mid X}(y \\mid x) = \\frac{f(x,y)}{f_X(x)}, \\quad f_X(x) > 0fY∣X​(y∣x)=fX​(x)f(x,y)​,fX​(x)>0

Same formula with pmfs in the discrete case.

"},{"id":"joint-laws/conditional-expectation/conditional-expectation","section":"Conditional expectation","prompt":"

Conditional expectation

","answer":"

E[Y∣X=x]=∫y fY∣X(y∣x) dyE[Y \\mid X = x] = \\int y\\,f_{Y \\mid X}(y \\mid x)\\,dyE[Y∣X=x]=∫yfY∣X​(y∣x)dy is a function h(x)h(x)h(x); then E[Y∣X]=h(X)E[Y \\mid X] = h(X)E[Y∣X]=h(X) is a random variable. Taking out what is known: E[g(X)Y∣X]=g(X) E[Y∣X]E[g(X)Y \\mid X] = g(X)\\,E[Y \\mid X]E[g(X)Y∣X]=g(X)E[Y∣X].

"},{"id":"joint-laws/conditional-expectation/law-of-total-expectation","section":"Conditional expectation","prompt":"

Law of total expectation

","answer":"
E[Y]=E[E[Y∣X]]E[Y] = E\\big[E[Y \\mid X]\\big]E[Y]=E[E[Y∣X]]
"},{"id":"joint-laws/conditional-expectation/law-of-total-variance","section":"Conditional expectation","prompt":"

Law of total variance

","answer":"
Var⁡(Y)=E[Var⁡(Y∣X)]+Var⁡(E[Y∣X])\\operatorname{Var}(Y) = E\\big[\\operatorname{Var}(Y \\mid X)\\big] + \\operatorname{Var}\\big(E[Y \\mid X]\\big)Var(Y)=E[Var(Y∣X)]+Var(E[Y∣X])
"},{"id":"joint-laws/covariance-and-correlation/covariance","section":"Covariance and correlation","prompt":"

Covariance

","answer":"
Cov⁡(X,Y)=E[(X−μX)(Y−μY)]=E[XY]−E[X] E[Y]\\operatorname{Cov}(X,Y) = E\\big[(X - \\mu_X)(Y - \\mu_Y)\\big] = E[XY] - E[X]\\,E[Y]Cov(X,Y)=E[(X−μX​)(Y−μY​)]=E[XY]−E[X]E[Y]

Independence gives Cov⁡(X,Y)=0\\operatorname{Cov}(X,Y) = 0Cov(X,Y)=0; the converse fails in general.

"},{"id":"joint-laws/covariance-and-correlation/bilinearity","section":"Covariance and correlation","prompt":"

Bilinearity

","answer":"
Cov⁡(∑iaiXi,∑jbjYj)=∑i∑jaibjCov⁡(Xi,Yj)\\operatorname{Cov}\\Big(\\sum_i a_i X_i, \\sum_j b_j Y_j\\Big) = \\sum_i \\sum_j a_i b_j \\operatorname{Cov}(X_i, Y_j)Cov(i∑​ai​Xi​,j∑​bj​Yj​)=i∑​j∑​ai​bj​Cov(Xi​,Yj​)
"},{"id":"joint-laws/covariance-and-correlation/correlation","section":"Covariance and correlation","prompt":"

Correlation

","answer":"
ρ(X,Y)=Cov⁡(X,Y)σX σY,0<σX,σY<∞,−1≤ρ≤1\\rho(X,Y) = \\frac{\\operatorname{Cov}(X,Y)}{\\sigma_X\\,\\sigma_Y}, \\quad 0 < \\sigma_X, \\sigma_Y < \\infty, \\quad -1 \\le \\rho \\le 1ρ(X,Y)=σX​σY​Cov(X,Y)​,0<σX​,σY​<∞,−1≤ρ≤1

∣ρ∣=1|\\rho| = 1∣ρ∣=1 exactly when Y=aX+bY = aX + bY=aX+b with probability 1 for some a≠0a \\ne 0a=0.

"},{"id":"joint-laws/covariance-and-correlation/variance-of-a-sum","section":"Covariance and correlation","prompt":"

Variance of a sum

","answer":"
Var⁡(∑i=1nXi)=∑i=1nVar⁡(Xi)+2∑i<jCov⁡(Xi,Xj)\\operatorname{Var}\\Big(\\sum_{i=1}^n X_i\\Big) = \\sum_{i=1}^n \\operatorname{Var}(X_i) + 2\\sum_{i<j} \\operatorname{Cov}(X_i, X_j)Var(i=1∑n​Xi​)=i=1∑n​Var(Xi​)+2i<j∑​Cov(Xi​,Xj​)

For uncorrelated (for example independent) XiX_iXi​ the covariance terms vanish.

"},{"id":"joint-laws/transformations-sums-and-order-statistics/change-of-variables-jacobian","section":"Transformations, sums and order statistics","prompt":"

Change of variables (Jacobian)

","answer":"
fU,V(u,v)=fX,Y(x(u,v), y(u,v))∣det⁡∂(x,y)∂(u,v)∣f_{U,V}(u,v) = f_{X,Y}\\big(x(u,v),\\, y(u,v)\\big)\\left|\\det \\frac{\\partial(x,y)}{\\partial(u,v)}\\right|fU,V​(u,v)=fX,Y​(x(u,v),y(u,v))​det∂(u,v)∂(x,y)​​

For a one-to-one differentiable map (X,Y)↦(U,V)(X,Y) \\mapsto (U,V)(X,Y)↦(U,V) with inverse x(u,v)x(u,v)x(u,v), y(u,v)y(u,v)y(u,v). Remember to transform the support too.

"},{"id":"joint-laws/transformations-sums-and-order-statistics/convolution-independent-sum","section":"Transformations, sums and order statistics","prompt":"

Convolution (independent sum)

","answer":"
fX+Y(z)=∫−∞∞fX(x) fY(z−x) dxf_{X+Y}(z) = \\int_{-\\infty}^{\\infty} f_X(x)\\,f_Y(z - x)\\,dxfX+Y​(z)=∫−∞∞​fX​(x)fY​(z−x)dx

Requires X⊥YX \\perp YX⊥Y. Discrete version: P(X+Y=z)=∑xpX(x) pY(z−x)P(X+Y = z) = \\sum_x p_X(x)\\,p_Y(z-x)P(X+Y=z)=∑x​pX​(x)pY​(z−x).

"},{"id":"joint-laws/transformations-sums-and-order-statistics/sums-of-independent-normal-and-poisson-variables","section":"Transformations, sums and order statistics","prompt":"

Sums of independent normal and Poisson variables

","answer":"
N(μ1,σ12)+N(μ2,σ22)∼N(μ1+μ2, σ12+σ22),Poisson⁡(λ1)+Poisson⁡(λ2)∼Poisson⁡(λ1+λ2)N(\\mu_1,\\sigma_1^2) + N(\\mu_2,\\sigma_2^2) \\sim N(\\mu_1+\\mu_2,\\, \\sigma_1^2+\\sigma_2^2), \\qquad \\operatorname{Poisson}(\\lambda_1) + \\operatorname{Poisson}(\\lambda_2) \\sim \\operatorname{Poisson}(\\lambda_1+\\lambda_2)N(μ1​,σ12​)+N(μ2​,σ22​)∼N(μ1​+μ2​,σ12​+σ22​),Poisson(λ1​)+Poisson(λ2​)∼Poisson(λ1​+λ2​)

Requires the two summands to be independent. Likewise Binomial⁡(n,p)+Binomial⁡(m,p)∼Binomial⁡(n+m,p)\\operatorname{Binomial}(n,p) + \\operatorname{Binomial}(m,p) \\sim \\operatorname{Binomial}(n+m,p)Binomial(n,p)+Binomial(m,p)∼Binomial(n+m,p) for independent summands with the same ppp. Each follows from multiplying mgfs.

"},{"id":"joint-laws/transformations-sums-and-order-statistics/order-statistic-density","section":"Transformations, sums and order statistics","prompt":"

Order statistic density

","answer":"
fX(k)(x)=n!(k−1)! (n−k)! F(x)k−1[1−F(x)]n−kf(x)f_{X_{(k)}}(x) = \\frac{n!}{(k-1)!\\,(n-k)!}\\,F(x)^{k-1}\\big[1 - F(x)\\big]^{n-k} f(x)fX(k)​​(x)=(k−1)!(n−k)!n!​F(x)k−1[1−F(x)]n−kf(x)

X1,…,XnX_1,\\dots,X_nX1​,…,Xn​ iid with cdf FFF and pdf fff. For the extremes: FX(n)(x)=F(x)nF_{X_{(n)}}(x) = F(x)^nFX(n)​​(x)=F(x)n and FX(1)(x)=1−[1−F(x)]nF_{X_{(1)}}(x) = 1 - [1 - F(x)]^nFX(1)​​(x)=1−[1−F(x)]n.

"},{"id":"joint-laws/bivariate-normal/bivariate-normal-density","section":"Bivariate normal","prompt":"

Bivariate normal density

","answer":"
f(x,y)=12πσXσY1−ρ2exp⁡(−12(1−ρ2)[(x−μX)2σX2−2ρ(x−μX)(y−μY)σXσY+(y−μY)2σY2])f(x,y) = \\frac{1}{2\\pi\\sigma_X\\sigma_Y\\sqrt{1-\\rho^2}} \\exp\\left(-\\frac{1}{2(1-\\rho^2)}\\left[\\frac{(x-\\mu_X)^2}{\\sigma_X^2} - \\frac{2\\rho(x-\\mu_X)(y-\\mu_Y)}{\\sigma_X\\sigma_Y} + \\frac{(y-\\mu_Y)^2}{\\sigma_Y^2}\\right]\\right)f(x,y)=2πσX​σY​1−ρ2​1​exp(−2(1−ρ2)1​[σX2​(x−μX​)2​−σX​σY​2ρ(x−μX​)(y−μY​)​+σY2​(y−μY​)2​])

Parameters: means μX,μY\\mu_X, \\mu_YμX​,μY​, standard deviations σX,σY>0\\sigma_X, \\sigma_Y > 0σX​,σY​>0, correlation ∣ρ∣<1|\\rho| < 1∣ρ∣<1. Marginals are N(μX,σX2)N(\\mu_X, \\sigma_X^2)N(μX​,σX2​) and N(μY,σY2)N(\\mu_Y, \\sigma_Y^2)N(μY​,σY2​).

"},{"id":"joint-laws/bivariate-normal/conditional-distribution","section":"Bivariate normal","prompt":"

Conditional distribution

","answer":"
Y∣X=x  ∼  N(μY+ρ σYσX(x−μX),    σY2(1−ρ2))Y \\mid X = x \\;\\sim\\; N\\left(\\mu_Y + \\rho\\,\\frac{\\sigma_Y}{\\sigma_X}(x - \\mu_X),\\;\\; \\sigma_Y^2(1 - \\rho^2)\\right)Y∣X=x∼N(μY​+ρσX​σY​​(x−μX​),σY2​(1−ρ2))

The conditional mean is linear in xxx and the conditional variance does not depend on xxx. For a jointly normal pair, ρ=0\\rho = 0ρ=0 implies independence.

"},{"id":"joint-laws/multinomial-distribution/multinomial","section":"Multinomial distribution","prompt":"

Multinomial

n∈{1,2,… }n \\in \\{1,2,\\dots\\}n∈{1,2,…} independent trials, each landing in one of kkk categories with probabilities p1,…,pk≥0p_1,\\dots,p_k \\ge 0p1​,…,pk​≥0, ∑ipi=1\\sum_i p_i = 1∑i​pi​=1. XiX_iXi​ counts trials in category iii. Mean and variance are per coordinate.

","answer":"
pmf
n!x1! x2!⋯xk! p1x1p2x2⋯pkxk\\displaystyle \\frac{n!}{x_1!\\,x_2!\\cdots x_k!}\\, p_1^{x_1} p_2^{x_2} \\cdots p_k^{x_k}x1​!x2​!⋯xk​!n!​p1x1​​p2x2​​⋯pkxk​​ for (x1,…,xk)∈{0,1,…,n}k,  x1+⋯+xk=n(x_1,\\dots,x_k) \\in \\{0,1,\\dots,n\\}^k, \\; x_1 + \\cdots + x_k = n(x1​,…,xk​)∈{0,1,…,n}k,x1​+⋯+xk​=n
Mean
E[Xi]=npi\\displaystyle E[X_i] = np_iE[Xi​]=npi​
Variance
Var⁡(Xi)=npi(1−pi)\\displaystyle \\operatorname{Var}(X_i) = np_i(1-p_i)Var(Xi​)=npi​(1−pi​)
mgf
M(t1,…,tk)=(p1et1+⋯+pketk)n\\displaystyle M(t_1,\\dots,t_k) = \\left(p_1e^{t_1} + \\cdots + p_ke^{t_k}\\right)^nM(t1​,…,tk​)=(p1​et1​+⋯+pk​etk​)n
"},{"id":"joint-laws/multinomial-distribution/multinomial-marginals","section":"Multinomial distribution","prompt":"

Multinomial marginals

","answer":"
Xi∼Binomial⁡(n,pi),Xi+Xj∼Binomial⁡(n,pi+pj)    (i≠j)X_i \\sim \\operatorname{Binomial}(n, p_i), \\qquad X_i + X_j \\sim \\operatorname{Binomial}(n, p_i + p_j) \\;\\; (i \\ne j)Xi​∼Binomial(n,pi​),Xi​+Xj​∼Binomial(n,pi​+pj​)(i=j)

Lumping categories together gives another multinomial.

"},{"id":"joint-laws/multinomial-distribution/multinomial-covariance","section":"Multinomial distribution","prompt":"

Multinomial covariance

","answer":"
Cov⁡(Xi,Xj)=−n pi pj,i≠j\\operatorname{Cov}(X_i, X_j) = -n\\,p_i\\,p_j, \\quad i \\ne jCov(Xi​,Xj​)=−npi​pj​,i=j

Negative because the counts compete for the same nnn trials. Hence, for 0<pi,pj<10 < p_i, p_j < 10<pi​,pj​<1, ρ(Xi,Xj)=−pipj(1−pi)(1−pj)\\rho(X_i, X_j) = -\\sqrt{\\frac{p_i p_j}{(1-p_i)(1-p_j)}}ρ(Xi​,Xj​)=−(1−pi​)(1−pj​)pi​pj​​​.

"},{"id":"limit-theorems/tail-inequalities/markov-s-inequality","section":"Tail inequalities","prompt":"

Markov's inequality

","answer":"
P(X≥a)≤E[X]a,X≥0,  a>0P(X \\ge a) \\le \\frac{E[X]}{a}, \\quad X \\ge 0, \\; a > 0P(X≥a)≤aE[X]​,X≥0,a>0
"},{"id":"limit-theorems/tail-inequalities/chebyshev-s-inequality","section":"Tail inequalities","prompt":"

Chebyshev's inequality

","answer":"
P(∣X−μ∣≥k)≤σ2k2,k>0P\\big(|X - \\mu| \\ge k\\big) \\le \\frac{\\sigma^2}{k^2}, \\quad k > 0P(∣X−μ∣≥k)≤k2σ2​,k>0

μ=E[X]\\mu = E[X]μ=E[X], σ2=Var⁡(X)<∞\\sigma^2 = \\operatorname{Var}(X) < \\inftyσ2=Var(X)<∞. Apply Markov to (X−μ)2(X-\\mu)^2(X−μ)2.

"},{"id":"limit-theorems/modes-of-convergence/almost-surely","section":"Modes of convergence","prompt":"

Almost surely

","answer":"

Xn→a.s.XX_n \\xrightarrow{\\text{a.s.}} XXn​a.s.​X if P(lim⁡n→∞Xn=X)=1P\\left(\\lim_{n\\to\\infty} X_n = X\\right) = 1P(limn→∞​Xn​=X)=1.

"},{"id":"limit-theorems/modes-of-convergence/in-probability","section":"Modes of convergence","prompt":"

In probability

","answer":"

Xn→pXX_n \\xrightarrow{p} XXn​p​X if for every ε>0\\varepsilon > 0ε>0, P(∣Xn−X∣>ε)→0P(|X_n - X| > \\varepsilon) \\to 0P(∣Xn​−X∣>ε)→0 as n→∞n \\to \\inftyn→∞.

"},{"id":"limit-theorems/modes-of-convergence/in-mean-of-order-p","section":"Modes of convergence","prompt":"

In mean of order p

","answer":"

Xn→LpXX_n \\xrightarrow{L^p} XXn​Lp​X (p≥1p \\ge 1p≥1) if E∣Xn−X∣p→0E|X_n - X|^p \\to 0E∣Xn​−X∣p→0. The case p=2p = 2p=2 is mean-square convergence.

"},{"id":"limit-theorems/modes-of-convergence/in-distribution","section":"Modes of convergence","prompt":"

In distribution

","answer":"

Xn→dXX_n \\xrightarrow{d} XXn​d​X if FXn(x)→FX(x)F_{X_n}(x) \\to F_X(x)FXn​​(x)→FX​(x) at every point xxx where FXF_XFX​ is continuous.

"},{"id":"limit-theorems/modes-of-convergence/how-the-modes-relate","section":"Modes of convergence","prompt":"

How the modes relate

","answer":"
Xn→a.s.X  ⟹  Xn→pX  ⟹  Xn→dX,Xn→LpX  ⟹  Xn→pXX_n \\xrightarrow{\\text{a.s.}} X \\implies X_n \\xrightarrow{p} X \\implies X_n \\xrightarrow{d} X, \\qquad X_n \\xrightarrow{L^p} X \\implies X_n \\xrightarrow{p} XXn​a.s.​X⟹Xn​p​X⟹Xn​d​X,Xn​Lp​X⟹Xn​p​X

The reverse implications fail in general. One exception: convergence in distribution to a constant ccc implies convergence in probability to ccc.

"},{"id":"limit-theorems/laws-of-large-numbers-and-the-clt/weak-law-of-large-numbers","section":"Laws of large numbers and the CLT","prompt":"

Weak law of large numbers

","answer":"
X‾n=1n∑i=1nXi→pμ\\overline{X}_n = \\frac{1}{n}\\sum_{i=1}^n X_i \\xrightarrow{p} \\muXn​=n1​i=1∑n​Xi​p​μ

XiX_iXi​ iid with E∣X1∣<∞E|X_1| < \\inftyE∣X1​∣<∞ and μ=E[X1]\\mu = E[X_1]μ=E[X1​]. With finite variance it follows from Chebyshev: P(∣X‾n−μ∣≥ε)≤σ2/(nε2)P(|\\overline{X}_n - \\mu| \\ge \\varepsilon) \\le \\sigma^2/(n\\varepsilon^2)P(∣Xn​−μ∣≥ε)≤σ2/(nε2).

"},{"id":"limit-theorems/laws-of-large-numbers-and-the-clt/strong-law-of-large-numbers","section":"Laws of large numbers and the CLT","prompt":"

Strong law of large numbers

","answer":"
X‾n→a.s.μ\\overline{X}_n \\xrightarrow{\\text{a.s.}} \\muXn​a.s.​μ

XiX_iXi​ iid with E∣X1∣<∞E|X_1| < \\inftyE∣X1​∣<∞.

"},{"id":"limit-theorems/laws-of-large-numbers-and-the-clt/central-limit-theorem","section":"Laws of large numbers and the CLT","prompt":"

Central limit theorem

","answer":"
n (X‾n−μ)σ→dN(0,1)\\frac{\\sqrt{n}\\,(\\overline{X}_n - \\mu)}{\\sigma} \\xrightarrow{d} N(0,1)σn​(Xn​−μ)​d​N(0,1)

XiX_iXi​ iid with mean μ\\muμ and variance 0<σ2<∞0 < \\sigma^2 < \\infty0<σ2<∞. In practice P(X‾n≤x)≈Φ(n(x−μ)/σ)P(\\overline{X}_n \\le x) \\approx \\Phi\\big(\\sqrt{n}(x - \\mu)/\\sigma\\big)P(Xn​≤x)≈Φ(n​(x−μ)/σ) for large nnn.

"},{"id":"limit-theorems/laws-of-large-numbers-and-the-clt/normal-approximation-to-the-binomial-de-moivre-laplace","section":"Laws of large numbers and the CLT","prompt":"

Normal approximation to the binomial (de Moivre–Laplace)

","answer":"
P(S≤k)≈Φ(k+12−npnp(1−p)),S∼Binomial⁡(n,p)P(S \\le k) \\approx \\Phi\\left(\\frac{k + \\tfrac12 - np}{\\sqrt{np(1-p)}}\\right), \\qquad S \\sim \\operatorname{Binomial}(n,p)P(S≤k)≈Φ(np(1−p)​k+21​−np​),S∼Binomial(n,p)

For integer kkk, fixed 0<p<10 < p < 10<p<1 and large nnn; a common rule of thumb is np≥10np \\ge 10np≥10 and n(1−p)≥10n(1-p) \\ge 10n(1−p)≥10. The 12\\tfrac1221​ is the continuity correction; for P(S≥k)P(S \\ge k)P(S≥k) use 1−Φ((k−12−np)/np(1−p))1 - \\Phi\\big((k - \\tfrac12 - np)/\\sqrt{np(1-p)}\\big)1−Φ((k−21​−np)/np(1−p)​). This is the CLT for SSS as a sum of nnn iid Bernoulli(ppp) variables.

"},{"id":"limit-theorems/laws-of-large-numbers-and-the-clt/poisson-limit-of-the-binomial","section":"Laws of large numbers and the CLT","prompt":"

Poisson limit of the binomial

","answer":"
(nk)pnk(1−pn)n−k→e−λλkk!as n→∞,npn→λ>0\\binom{n}{k} p_n^k (1-p_n)^{n-k} \\to \\frac{e^{-\\lambda}\\lambda^k}{k!} \\quad \\text{as } n \\to \\infty, \\qquad n p_n \\to \\lambda > 0(kn​)pnk​(1−pn​)n−k→k!e−λλk​as n→∞,npn​→λ>0

For each fixed k∈{0,1,2,… }k \\in \\{0,1,2,\\dots\\}k∈{0,1,2,…}, so Binomial⁡(n,pn)→dPoisson⁡(λ)\\operatorname{Binomial}(n, p_n) \\xrightarrow{d} \\operatorname{Poisson}(\\lambda)Binomial(n,pn​)d​Poisson(λ). In practice, for large nnn and small ppp, Binomial⁡(n,p)\\operatorname{Binomial}(n,p)Binomial(n,p) is approximately Poisson⁡(np)\\operatorname{Poisson}(np)Poisson(np).

"},{"id":"limit-theorems/tools-for-limits/continuity-theorem-for-mgfs","section":"Tools for limits","prompt":"

Continuity theorem for mgfs

","answer":"

If MXn(t)→MX(t)M_{X_n}(t) \\to M_X(t)MXn​​(t)→MX​(t) for every ttt in an open interval around 000, where MXM_XMX​ is finite there, then Xn→dXX_n \\xrightarrow{d} XXn​d​X. With characteristic functions, φXn(t)→φX(t)\\varphi_{X_n}(t) \\to \\varphi_X(t)φXn​​(t)→φX​(t) for every real ttt is enough.

"},{"id":"limit-theorems/tools-for-limits/continuous-mapping-theorem","section":"Tools for limits","prompt":"

Continuous mapping theorem

","answer":"

If ggg is continuous and Xn→XX_n \\to XXn​→X almost surely, in probability, or in distribution, then g(Xn)→g(X)g(X_n) \\to g(X)g(Xn​)→g(X) in the same mode.

"},{"id":"limit-theorems/tools-for-limits/slutsky-s-theorem","section":"Tools for limits","prompt":"

Slutsky's theorem

","answer":"
Xn→dX,    Yn→pc  ⟹  Xn+Yn→dX+c,    XnYn→dcX,    XnYn→dXc    (c≠0)X_n \\xrightarrow{d} X, \\;\\; Y_n \\xrightarrow{p} c \\implies X_n + Y_n \\xrightarrow{d} X + c, \\;\\; X_n Y_n \\xrightarrow{d} cX, \\;\\; \\frac{X_n}{Y_n} \\xrightarrow{d} \\frac{X}{c} \\;\\; (c \\ne 0)Xn​d​X,Yn​p​c⟹Xn​+Yn​d​X+c,Xn​Yn​d​cX,Yn​Xn​​d​cX​(c=0)

ccc is a constant. Typical use: replace an unknown σ\\sigmaσ by a consistent estimate in the CLT.

"}]}