Vector spaces
Signals, polynomials and matrices are added and scaled by the same rules as arrows in the plane. That common structure is what is studied here.
01The same structure in very different things
Arrows in the plane can be added and multiplied by numbers. So can 2 × 3 matrices. And polynomials of degree at most 3. And voltage signals as functions of time. In all four cases the same rules hold, so it makes sense to study them once and for all: that is a vector space.
Formally, a set V with an addition and a multiplication by scalars is a vector space if the addition is commutative and associative, has an identity element (the zero vector) and additive inverses, and the multiplication by scalars is associative, has 1 as its identity and distributes over both additions. That makes eight axioms, and all of them hold in the examples above.
Everything that is proved from the eight axioms automatically holds for arrows, matrices, polynomials and signals. When it is later said that a periodic signal is a linear combination of sines and cosines —the Fourier series—, exactly this language is being used: a basis of a function space.
| Space | Its “vectors” | Dimension |
|---|---|---|
| \( \mathbb{R}^2 \), \( \mathbb{R}^3 \), \( \mathbb{R}^n \) | Pairs, triples, n-tuples of numbers | 2 · 3 · n |
| \( M_{2 \times 3} \) | Matrices with 2 rows and 3 columns | 6 |
| P3 | Polynomials of degree \( \le 3 \) | 4 |
| C[a, b] | Continuous functions on an interval | Infinite |
| Solutions of \( A \cdot \vec{x} = \vec{0} \) | The vectors that A sends to zero | n − rank(A) |
02Subspaces
A subspace is a subset that, with the same operations, is again a vector space. You do not need to verify the eight axioms: three conditions are enough.
S ⊆ V is a subspace if it contains the zero vector, if the sum of two elements of S stays in S and if the scalar multiple of an element of S stays in S.
That is why a line through the origin is a subspace of R², and one that does not pass through the origin is not: it does not contain the zero vector. The same goes for planes in R³.
The most important example for engineering: the solution set of a homogeneous system A·x = 0 is always a subspace, called the kernel or null space of A. Its dimension is n − rank(A), and that is exactly the number of free parameters that appeared in systems of linear equations.
03Linear combinations and spanning sets
A linear combination of v₁, …, vk is any expression a₁v₁ + a₂v₂ + … + akvk. The set of all possible linear combinations is called the span and is written span{v₁, …, vk}: it is always a subspace, the smallest one that contains those vectors.
In R², a nonzero vector spans a line; two non-parallel vectors span the whole plane; three vectors span the plane just the same, but one of them is superfluous.
04Linear dependence and independence
Said without formulas: they are dependent when one of them can be written in terms of the others, that is, when there is surplus information. To decide, build the matrix with the vectors as rows and compute the rank: if it equals the number of vectors, they are independent. With n vectors of Rn the determinant is enough: independent if and only if it is nonzero.
05Basis and dimension
A basis is a set of vectors that is both spanning and independent: just what is needed and no more. All the bases of a given space have the same number of elements, and that number is the dimension.
- The standard basis of R³ is {(1,0,0), (0,1,0), (0,0,1)}, that is {î, ĵ, k̂}.
- That of P₃ is {1, x, x², x³}: four elements, dimension 4.
- In a space of dimension n, any set of n independent vectors is a basis, and any set of more than n vectors is necessarily dependent.
The existence of a basis is what makes it possible to work with coordinates: once a basis is fixed, each vector is written in exactly one way as a linear combination of it, and that list of coefficients identifies it.
Drag the two basis vectors and the target point. The oblique grid is the one generated by that basis.
06Change of basis
The coordinates of a vector depend on the chosen basis: the vector is the same, what changes is how it is measured. If B = {b₁, b₂} is a basis of R² and P is the matrix that has b₁ and b₂ as columns, then:
Changing basis is the key step of many methods: choosing the “right” basis turns a tangled problem into a simple one. The diagonalization in eigenvalues and eigenvectors is precisely that: looking for the basis in which a transformation reduces to stretching each axis.
07Orthogonal bases and signals
When the space has an inner product defined —the generalization of the dot product— you can talk about magnitudes, angles and perpendicularity. An orthogonal basis is one in which all the vectors are perpendicular to each other, and it is far more convenient: each coordinate is computed separately, by projecting, without solving any system.
In the space of periodic signals, the inner product of two signals is the integral of their product over one period. With that definition, the sines and cosines of the different harmonics turn out to be mutually orthogonal and form a basis. The Fourier coefficients of a signal are its coordinates in that basis, and that is why they are computed one by one with an integral: it is exactly the projection. The spectrum shown by an analyzer is nothing other than that list of coordinates. It is covered in signals and Fourier and in the instrument simulator.
08In the lab
For each set, decide whether it is a subspace of R³ and justify it with the criterion: the vectors with x + y + z = 0; those satisfying x + y + z = 1; those with a positive first component; the solutions of a homogeneous system of your own choosing.
Solve a 3 × 4 homogeneous system, write the general solution with parameters and extract from it a basis of the solution space. Verify that its dimension is 4 − rank(A).
With B = {(1, 1), (1, −1)}, find the coordinates of (5, 1) by solving the system and verify with P−1. Check the result with the lab in section 5.
09Common mistakes
- Calling a line that does not pass through the origin a subspace. Without the zero vector, it is not one.
- Confusing a spanning set with a basis: a spanning set can have surplus vectors.
- Believing that four vectors of R³ can be independent. In dimension 3, four are always dependent.
- Forgetting that coordinates depend on the basis and comparing coordinates from different bases.
- Building the change-of-basis matrix with the vectors as rows instead of columns.
10Self-assessment
Is the set of (x, y) with y = 2x a subspace of R²?
Yes: it contains (0,0), the sum of two of its elements satisfies the condition again and so does the scalar multiple. It is the line through the origin with slope 2.
Are (1, 2), (2, 4) and (0, 1) independent?
No: the second is twice the first. Besides, three vectors of R² can never be independent.
What is the dimension of the space of symmetric 3 × 3 matrices?
6: the three on the diagonal plus the three above it, because those below are determined by symmetry.
If A is 5 × 7 with rank 4, what is the dimension of the solution space of A·x = 0?
7 − 4 = 3.
Coordinates of (5, 1) in the basis {(1, 1), (1, −1)}.
a(1,1) + b(1,−1) = (5,1) gives a + b = 5 and a − b = 1: a = 3 and b = 2. The coordinates are (3, 2).
Why is Fourier said to be a change of basis?
Because it writes a signal as a linear combination of an orthogonal basis of sines and cosines; its coefficients are the coordinates of the signal in that basis.
11Further reading
- Stanley I. Grossman. Álgebra lineal. 6th ed., McGraw-Hill, 2008 (in Spanish). Spaces and subspaces, with the examples of matrices, polynomials and functions well developed.
- David C. Lay. Linear Algebra and Its Applications. 5th ed. (Spanish edition, “Álgebra lineal y sus aplicaciones”), Pearson, 2016. Very clear on basis, dimension, coordinates and change of basis.
- Gilbert Strang. Linear Algebra and Its Applications. 4th ed. (Spanish edition, “Álgebra lineal y sus aplicaciones”), Cengage Learning, 2007. The four fundamental subspaces of a matrix, and the connection with Fourier.