Inference and estimation
“95 % confidence” does not mean there is a 95 % probability that the value lies in that interval. The 95 % is a property of the procedure, not of the interval you ended up with, and that distinction is not understood by reading about it: you have to watch it fail.
01From the sample to the population
You never measure everything. You measure a sample and want to say something about the entire population: the whole batch, all the parts the process will ever produce, the true value of the resistance. That leap is inference, and it has rules.
| Population | Sample | |
|---|---|---|
| What it is | The entire set of interest | The data you actually have |
| Mean | \( \mu \), unknown | \( \bar{x} \), computable |
| Standard deviation | \( \sigma \), unknown | \( s \), computable |
| It is called | Parameter | Estimator |
An estimator is a random variable: if you take another sample, you get another number. Being unbiased means that on average it gives the true parameter, and being consistent means that it approaches it as the sample grows. The sample mean has both properties.
02The confidence interval
Giving a single number as an estimate hides the uncertainty. A confidence interval reports a range built with a procedure that, repeated many times, captures the true parameter a known proportion of the time.
It does not mean that there is a 95 % probability that \( \mu \) lies in that interval. Once calculated, the interval either contains \( \mu \) or it does not: there is no chance involved, what is missing is information. What the 95 % describes is the procedure: if the sampling were repeated many times, 95 % of the intervals built this way would contain the true value. The lab below is made so that you can see this, because reading about it is not enough.
Forty samples taken from the same population, each with its interval. The vertical line is the true value, which in real life nobody knows. Count how many intervals miss it.
03How many measurements are needed
Given a target precision, the sample size comes from solving for the half-width of the interval:
| If you want | You need |
|---|---|
| Half the error | Four times as many measurements |
| One tenth of the error | A hundred times as many measurements |
| Going from 95 % to 99 % confidence | 73 % more measurements, for the same error |
The third row shows the other trade-off. For the same amount of data, more confidence means a wider interval: you can be more certain, or you can be more precise, but not both without measuring more. A 100 % confidence interval would be “from minus infinity to plus infinity,” perfectly true and perfectly useless.
The \( \sqrt{n} \) reduces the random component of the uncertainty, not the systematic one. If the instrument is out of calibration and always reads 2 mV too high, measuring a million times gives an average with a tiny statistical uncertainty, centered 2 mV away from the true value. That is the difference between precision and accuracy, and the reason calibration exists.
04In the lab
Measure the same voltage twenty times. Calculate the mean, the standard deviation and the 95 % confidence interval using Student’s t. Repeat with the first five measurements and compare the widths of the two intervals.
Divide two hundred measurements into twenty groups of ten and build the 90 % interval of each group. Count how many contain the mean of the two hundred and compare with the two that theory predicts will fail.
With the standard deviation estimated in activity 1, calculate how many measurements would be needed to know the voltage with an error of 0.1 mV at 95 %. Evaluate whether the number is practical and what would be advisable if it is not.
05Common mistakes
- Saying there is a 95 % probability that \( \mu \) lies in the interval. The 95 % belongs to the procedure, not to that interval.
- Using \( z \) with small samples. With an estimated \( \sigma \) and few data points, use t.
- Confusing \( s \) with the standard error. The latter is \( s/\sqrt{n} \) and is much smaller.
- Believing that averaging corrects a systematic error. It only reduces the random one.
- Asking for more confidence expecting more precision. They go in opposite directions.
- Forgetting the square in the sample size. Improving tenfold costs a hundred times more.
06Self-assessment
What does a 95 % interval mean?
That the procedure, repeated many times, produces intervals that contain the parameter 95 % of the time.
What is the standard error of the mean?
\( \sigma/\sqrt{n} \), or \( s/\sqrt{n} \) if the standard deviation was estimated from the sample.
When do you use t instead of z?
When \( \sigma \) is unknown and is estimated with \( s \). With large \( n \) the two coincide.
How many measurements are needed to cut the error in half?
Four times as many: the size goes with the square.
What happens to the width when the confidence goes up?
It grows. More confidence with the same data is paid for with less precision.
Does averaging correct a calibration error?
No. It reduces the random error and leaves the systematic one intact. That is what calibration is for.
07Further reading
- Douglas C. Montgomery and George C. Runger. Applied Statistics and Probability for Engineers. 7th ed. (Spanish edition, “Probabilidad y estadística aplicadas a la ingeniería”), Wiley, 2018. Chapters 7 and 8: sampling distributions, point estimation and confidence intervals.
- Sheldon Ross. Introduction to Probability and Statistics for Engineers and Scientists. 5th ed. (Spanish edition, “Introducción a la probabilidad y estadística para ingenieros”), Elsevier, 2018. The sampling distribution treated with care, including where Student’s t comes from.
- JCGM. Evaluation of measurement data — Guide to the expression of uncertainty in measurement. JCGM 100:2008 (Spanish edition published by the BIPM). How Type A and Type B uncertainties are combined, and how the expanded uncertainty is reported with its coverage factor.