The sampling theorem
Keeping the value every T seconds and throwing away the rest seems to lose information for good. And yet, if the signal is bandlimited and sampled fast enough, the samples contain everything.
01The statement, and why it is not obvious
Sampling means keeping the value of a signal every \( T \) seconds and throwing away everything that happened in between. It seems that information is lost forever, and yet:
That is the sampling theorem, and it is the reason why digital audio, cellular telephony and any instrument with an A/D converter exist. The frequency \( 2f_{\text{max}} \) is called the Nyquist rate, and the statement is attributed to Nyquist, Shannon, Kotelnikov and Whittaker depending on the country where it is studied.
02What sampling does to the spectrum
Ideal sampling is multiplying the signal by a train of impulses spaced \( T \) apart. And by the modulation theorem, multiplying in time is convolving in frequency. Since the transform of an impulse train is another impulse train, the result is:
That is the whole theorem, and it is easier to see than to prove. If the replicas do not touch, the original remains intact in the middle and a low-pass filter is enough to recover it. If they overlap, they add up and there is no filter that can separate them: that is aliasing.
On top, the spectrum of the signal and its replicas every \( f_s \). Below, the original signal and the one reconstructed from the samples. Lower the sampling frequency until the replicas touch.
03How the signal is recovered
If the replicas do not overlap, you filter with an ideal low-pass with cutoff \( f_s/2 \) and the original is left. In time, that filter is a sinc, and the reconstruction takes the form of an interpolation:
That formula is exact but unrealizable: the sinc extends to infinity in both directions, and we already saw that this filter is neither causal nor BIBO stable. In practice a D/A converter does something else: it holds each sample until the next one.
Holding the value between samples is equivalent to convolving with a rectangular pulse of width \( T \), whose spectrum is a sinc. That attenuates the high frequencies of the useful band: at the edge, \( f_s/2 \), the drop is \( 2/\pi \), that is, −3.92 dB. That is why audio converters include a compensation filter —which appears in datasheets as sinc correction— and why it pays to oversample: if the useful band occupies only a fraction of \( f_s/2 \), the sinc roll-off in that region is negligible.
04The numbers in practice
| Application | Useful band | \( f_s \) | Margin |
|---|---|---|---|
| Telephony | 3.4 kHz | 8 kHz | 1.18 times the Nyquist rate |
| Compact disc | 20 kHz | 44.1 kHz | 1.10 times |
| Professional audio | 20 kHz | 48 or 96 kHz | 1.2 or 2.4 times |
| Digital oscilloscope | 100 MHz | 1 GS/s or more | 5 times or more |
None of them works right at the limit, for two reasons. The first is that a real anti-aliasing filter does not cut off vertically: it needs a transition band between the useful frequency and \( f_s/2 \), and that band is the margin. The second is the oscilloscope row: to see a waveform it is not enough to be able to reconstruct it in theory; many samples per period are needed for the drawing to be recognizable.
Telephony shows the tightest case: with an 8 kHz sampling rate and a 3.4 kHz band, barely 600 Hz are left for the filter to go from passing to fully attenuating. That is why the filters in telephone channels are of high order.
05In the lab
Using an A/D converter, sample a 1 kHz sinusoid at 10, 4, 2.5 and 1.8 kS/s, reconstruct with the D/A and observe the output. Identify from what frequency it stops resembling the original.
Sweep the frequency of a sinusoid between DC and \( f_s/2 \) while keeping the input amplitude constant, and measure the amplitude at the D/A output. Compare the measured curve with \( \operatorname{sinc}(f/f_s) \) and verify the −3.92 dB at the edge.
Design the anti-aliasing filter for a system with a 4 kHz useful band and \( f_s = 10 \) kHz. Determine what order is needed to attenuate 60 dB at 5 kHz starting from 0 dB at 4 kHz.
06Common mistakes
- Using exactly \( f_s = 2f_{\text{max}} \). The theorem requires more than twice; at the exact limit a sinusoid can always be sampled at its zero crossings.
- Forgetting the anti-aliasing filter. The theorem requires the signal to be bandlimited, and that does not happen by itself: it has to be enforced.
- Confusing the Nyquist rate with \( f_s/2 \). The Nyquist rate is \( 2f_{\text{max}} \); \( f_s/2 \) is the limit of the representable band.
- Believing that the D/A reconstructs exactly. The staircase attenuates; it has to be compensated.
- Sampling at exactly twice because “the theorem allows it”. Without margin, the anti-aliasing filter is unrealizable.
07Self-assessment
What does sampling do to the spectrum?
It repeats it every \( f_s \). If the replicas do not overlap, the original remains whole.
What condition does the theorem require?
That the signal be bandlimited and that \( f_s \) be greater than twice its maximum frequency.
What is the exact interpolator?
The sinc: each sample is replaced by a sinc centered on it and they are all added.
Why is the sinc not used in practice?
Because it extends infinitely to both sides: it is neither causal nor BIBO stable.
How much does the D/A staircase attenuate at \( f_s/2 \)?
\( 2/\pi \), that is, −3.92 dB. That is why converters include sinc compensation.
Why does the CD use 44.1 kHz and not 40?
To leave margin for the anti-aliasing filter between the 20 kHz of useful band and the 22.05 kHz of \( f_s/2 \).
08Further reading
- Alan V. Oppenheim and Alan S. Willsky. Signals and Systems. 2nd ed. (Spanish edition, “Señales y sistemas”), Prentice Hall, 1997. The complete Chapter 7: sampling, replicas, reconstruction and the zero-order hold.
- Walt Kester (ed.). Data Conversion Handbook. Analog Devices / Newnes, 2005. Sampling from the side of the real converter: jitter, aperture and sinc compensation.
- John G. Proakis and Dimitris G. Manolakis. Digital Signal Processing: Principles, Algorithms, and Applications. 4th ed. (Spanish edition, “Tratamiento digital de señales”), Prentice Hall, 2006. Sampling and reconstruction with the mathematical detail and the filter design criteria.