Probability Theory Reference
Reference for probability: distributions, expectation, variance, and limit theorems.
Fundamentals
Probability measures assign values in [0,1] to events, with P(Ω)=1 and countable additivity for disjoint events. Conditional probability P(A|B) = P(A∩B)/P(B). Independence: P(A∩B) = P(A)P(B). Bayes' theorem: P(A|B) = P(B|A)P(A)/P(B). Expectation E[X] = sum(x * P(X=x)) for discrete, integral for continuous. Variance Var(X) = E[X^2] - E[X]^2. Standard deviation is sqrt(Var). The law of total probability decomposes over partitions. These foundations underpin statistics, machine learning, and cryptography.
- Bayes
- P(A|B) = P(B|A)P(A)/P(B)
- E[X]
- Expected value
- Var
- E[X^2] - E[X]^2
inferencemeanspreadDistributions and Limit Theorems
Bernoulli: 0/1 with probability p. Binomial: n Bernoulli trials. Poisson: rare events, λ rate. Uniform: equal probability over range. Normal (Gaussian): bell curve, mean μ, variance σ^2. Exponential: memoryless waiting time. The Law of Large Numbers: sample mean converges to E[X]. Central Limit Theorem: sum of iid variables approaches Normal. These theorems justify using Normal approximation for large samples. The Chernoff bound provides exponential tail bounds for sums. Cryptographic random sources aim for uniform distribution.
- Normal
- Gaussian distribution
- CLT
- Sum -> Normal
- Chernoff
- Exponential tail bound
CLT limitlarge samplesconcentration