Random variables and distributions
Thermal noise is normal, the arrival of photons at a detector is Poisson and the time to failure of a component is exponential. Choosing the wrong distribution gives answers that look reasonable and are not.
01From events to numbers
A random variable is a function that assigns a number to each outcome of the experiment. That step, which looks bureaucratic, is what makes it possible to calculate averages, variances and everything else: events cannot be added, but numbers can.
| Discrete | Continuous | |
|---|---|---|
| Values | Countable: 0, 1, 2… | A whole interval |
| Described by | \( p(x) = P(X = x) \) | Density \( f(x) \) |
| \( P(X = a) \) | Can be positive | Always zero |
| Probability | \( \sum p(x) \) | \( \int f(x)\,dx \): it is an area |
The third row is uncomfortable and important: for a continuous variable, the probability of an exact value is zero. Not because it is impossible, but because there are infinitely many possible values. What makes sense to ask is the probability of falling in an interval, and that is the area under the density.
02The five distributions of electronics
There are dozens of distributions, but in electronics five of them show up again and again. Each one models a different mechanism, and choosing the wrong one gives answers that look reasonable and are not.
| Distribution | Models | Where it shows up |
|---|---|---|
| Binomial | How many successes in \( n \) trials | Good parts in a batch; yield of a wafer |
| Poisson | How many rare events in an interval | Photons at a detector, failures per year, defects per wafer |
| Uniform | All values equally likely | Quantization error of a converter |
| Normal | Sum of many small effects | Thermal noise, manufacturing tolerances |
| Exponential | Time until the next event | Useful life with a constant failure rate |
Each one with its parameters and its electronics case. The upper curve is the density and the lower one the cumulative distribution; the shaded band is the probability of the interval you choose.
03Why the normal distribution shows up everywhere
It is not a coincidence or a convention: it is a theorem. The central limit theorem says that the sum of many independent contributions tends to a normal distribution, almost regardless of what each one looks like.
| Interval | Probability | In practice |
|---|---|---|
| \( \mu \pm 1\sigma \) | 68.27 % | The standard uncertainty |
| \( \mu \pm 2\sigma \) | 95.45 % | Close to the usual 95 % confidence |
| \( \mu \pm 3\sigma \) | 99.73 % | Control limits of a process |
| \( \mu \pm 6\sigma \) | 99.9999998 % | Two defects per billion |
The thermal noise of a resistor is the perfect example: it is the sum of the motion of an astronomical number of carriers, each contributing very little, and the result is an almost perfect Gaussian. Its rms value is \( \sqrt{4kTRB} \) —thermodynamics showing up again—, but its shape comes from the central limit theorem.
A Gaussian has no maximum: however small, any value has a nonzero probability. That is why talking about the “peak value” of noise makes no sense without saying with what probability. The practical convention is to use the peak-to-peak at 99.7 %, which is six standard deviations, or sometimes 6.6 sigma for 99.9 %. A design that assumes the noise never exceeds three sigma will fail, every now and then, exactly 0.27 % of the time.
04In the lab
Digitize the noise of an amplifier with its input shorted and build the histogram of the samples. Verify the bell shape, calculate the standard deviation and compare it with the theoretical thermal noise of the input resistance.
Count the pulses from a photodiode under very dim light in fixed intervals, many times. Build the histogram of counts and verify that the variance equals the mean, which is the signature of a Poisson distribution.
Digitize a slow ramp with a converter of few bits and calculate the error relative to the ideal ramp. Build its histogram, verify that it is uniform and compare its standard deviation with the theoretical \( q/\sqrt{12} \).
05Common mistakes
- Calculating \( P(X = a) \) for a continuous variable. It is always zero: you have to ask about an interval.
- Confusing density with probability. The density can be greater than 1; the probability is the area.
- Using the normal distribution to count rare events. That is what Poisson is for, especially with few counts.
- Always assuming a constant failure rate. The exponential holds in the useful life, not in infant mortality or wear-out.
- Talking about the peak value of noise. Without an associated probability it means nothing.
- Applying the central limit theorem with few terms. It needs many, and none of them dominating.
06Self-assessment
What is \( P(X = 5) \) if \( X \) is continuous?
Zero. For a continuous variable only intervals have nonzero probability.
Which distribution models thermal noise?
The normal, because of the central limit theorem: it is the sum of a huge number of small, independent contributions.
What is the signature of a Poisson distribution?
That the variance equals the mean.
What is the probability of falling within \( \mu \pm 3\sigma \)?
99.73 %. 27 cases in every ten thousand fall outside.
What is the standard deviation of the quantization error?
\( q/\sqrt{12} \), with \( q \) the converter’s step size, because the error is uniform over one step.
When does the exponential distribution hold for useful life?
As long as the failure rate is constant: the flat part of the bathtub curve.
07Further reading
- Douglas C. Montgomery and George C. Runger. Applied Statistics and Probability for Engineers. 7th ed. (Spanish edition, “Probabilidad y estadística aplicadas a la ingeniería”), Wiley, 2018. Chapters 3 and 4: discrete and continuous random variables, with the five distributions of this unit.
- Athanasios Papoulis and S. Unnikrishna Pillai. Probability, Random Variables and Stochastic Processes. 4th ed., McGraw-Hill, 2002. The reference for anyone continuing on to stochastic processes and noise in communication systems.
- C. D. Motchenbacher and J. A. Connelly. Low-Noise Electronic System Design. Wiley, 1993. Noise treated as a random variable, with the step from the standard deviation to the peak-to-peak value that a designer uses.