Component-wise, and geometrically
Addition
Add matching coordinates. Nothing crosses between components — the first coordinate never sees the second.
Geometrically: place the tail of the second arrow at the tip of the first, and the sum runs from the original origin to the final tip. Do it in the other order and you land in the same place, which is exactly the parallelogram in the hero.
Scalar multiplication
Multiply every coordinate by the same number. Again nothing mixes.
Geometrically: stretch the arrow. c > 1 lengthens, 0 < c < 1 shortens, c < 0 flips it through the origin, and c = 0 collapses it to the zero vector. The direction line never changes — only where you sit along it.
The rules those operations must obey
A vector space is any set where you can define these two operations such that eight properties hold. The properties are unexciting individually — they mostly say "behaves the way arithmetic does" — but together they are exactly what every theorem in linear algebra is built from.
| Axiom | Statement | What it rules out |
|---|---|---|
| closure (+) | u + v is in V | Adding two members can't escape the set |
| closure (·) | cv is in V | Nor can scaling one |
| commutative | u + v = v + u | Order of addition can't matter |
| associative | (u + v) + w = u + (v + w) | Grouping can't matter, so sums need no brackets |
| zero vector | There is a 0 with v + 0 = v | Every vector space has an origin. This is not optional |
| additive inverse | Every v has a −v with v + (−v) = 0 | You can always get back to the origin |
| distributive over vectors | c(u + v) = cu + cv | Scaling a sum equals summing the scalings |
| distributive over scalars | (c + d)v = cv + dv | The two operations agree with each other |
| compatible | c(dv) = (cd)v | Scaling twice is scaling once by the product |
| scalar identity | 1v = v | Multiplying by one does nothing |
A vector space is remarkably bare
The most instructive thing about the definition is what it leaves out. None of the following are available in a plain vector space, and each has to be added deliberately as extra structure.
| Not included | What it would need | What that buys |
|---|---|---|
| length of a vector | A norm | Magnitude, distance, convergence. L1 and L2 are different choices |
| angle between vectors | An inner product | Orthogonality, projections, cosine similarity, least squares |
| vector × vector | An algebra | Not part of the definition at all. The dot product returns a scalar, not a vector |
| comparing two vectors | An order | There is no "greater than" for vectors in general |
| coordinates | A chosen basis | Vectors exist without coordinates; coordinates are a description you pick |
The only thing you can build
Scale each vector by some number, add the results. This is a linear combination, and it is the only expression the two operations permit. Everything reachable from a set of vectors has this form; nothing else does.
All coefficients zero
Gives the zero vector, always. Called the trivial combination — and whether it's the only way to reach zero is precisely the question of linear independence.
One coefficient nonzero
A scalar multiple of a single vector. The set of all of these is a line through the origin.
All coefficients free
The full reachable set, which is the span. Section 05.
Everywhere you can get to
The span of a set of vectors is the set of all their linear combinations — the complete reachable territory. Turn on "show span" in the hero and it's drawn for you.
What span looks like
- span{}
- Just the origin. The empty combination.
- span{v}
- A line through the origin, in the direction of v.
- span{v, w}
- A plane, if they point in genuinely different directions. A line if they don't.
- span{e₁, e₂}
- All of ℝ² — the standard basis reaches everything.
Notice that every span contains the origin, because setting all coefficients to zero is always allowed. There is no such thing as a span that misses the origin.
Adding a vector may add nothing
Throw a third vector into a set and one of two things happens. Either it lies outside the current span, in which case the span grows a dimension — or it was already reachable, in which case the span is completely unchanged.
In the hero, drag w onto the line through v. The span was a plane; now it's a line. The second vector stopped contributing anything the first didn't already give you.
When a vector earns its place
A set is linearly independent if the only way to combine them into the zero vector is to set every coefficient to zero. If some non-trivial combination hits zero, one of them is redundant — you can solve for it in terms of the others.
Equivalent phrasings
- No vector in the set lies in the span of the others.
- Removing any one of them shrinks the span.
- Every vector in the span has exactly one representation as a combination.
- For a square matrix of them: the determinant is non-zero.
That third one is doing quiet work. Independence is what makes coordinates well-defined — without it, the same point has infinitely many descriptions.
Enough vectors, and not one more
A basis is a set that is both independent and spanning — just enough vectors to reach everything, with none to spare. Every vector space has one, and every basis for a given space has the same number of elements. That number is the dimension.
Spanning but dependent
Too many vectors. Reaches everything, but representations aren't unique — the same point can be built several ways.
Independent but not spanning
Too few. Every representation is unique, but there are points you can't reach at all.
Basis
Exactly right. Everything is reachable, and each point has precisely one set of coefficients.
Change of basis, in ML
- PCA finds a new orthonormal basis ordered by variance, then keeps the first few directions. The data hasn't changed — you re-expressed it and discarded coordinates that were mostly noise.
- Fourier and wavelet transforms are changes of basis for functions. Same signal, different coefficients, and compression is throwing small coefficients away.
- Embeddings are coordinates in a learned basis. The dimensions individually mean nothing; the geometry between points is what carries information.
Dimension is intrinsic
That every basis has the same size is a genuine theorem, not a convention. It's what makes "dimension" a property of the space rather than of the description you happened to pick.
It also underwrites the rank-nullity theorem: for a linear map, the dimensions of the image and the kernel always sum to the dimension of the domain. Whatever a map collapses, it loses from its output exactly.
A vector space inside a vector space
A subset that is itself a vector space under the same two operations. The test is short, because the messier axioms are inherited automatically.
The three-part test
- It contains the zero vector.
- It is closed under addition.
- It is closed under scalar multiplication.
The first condition eliminates most candidates immediately. A line that doesn't pass through the origin fails — scale any of its points by zero and you leave the set.
Column space
The span of a matrix's columns — everything Ax can produce. A system Ax = b is solvable exactly when b lies in it.
Null space
Every x with Ax = 0 — what the map destroys. If it contains more than the origin, solutions are never unique.
Row space
The span of the rows. Its dimension equals the column space's, which is the somewhat surprising fact that row rank equals column rank.
Anything you can add and scale
The definition never mentions arrows, coordinates, or geometry. Any set with two operations obeying the axioms qualifies — and the payoff is that all the machinery transfers. Below, five functions are being combined the same way two arrows were.
Five coefficients, five basis functions, one linear combination. The faint curves are the scaled components; the amber curve is their sum. Structurally identical to a·v + b·w — only the objects changed.
| The "vectors" are | Addition means | Dimension |
|---|---|---|
| Arrows in ℝⁿ | Coordinate-wise | n |
| Polynomials of degree ≤ n | Add coefficients | n + 1 |
| Continuous functions on [0,1] | Pointwise: (f+g)(x) = f(x)+g(x) | Infinite |
| m × n matrices | Entry-wise | mn |
| Solutions to a linear ODE | Add solutions | Order of the equation |
| Random variables with finite variance | Add pointwise | Infinite — and covariance is its inner product |
| Grayscale images | Add pixel intensities | Number of pixels |
| Word embeddings | Coordinate-wise | Model width |
What happens when you restrict the coefficients
Linear combinations let coefficients be anything. Constrain them and you get smaller, more useful sets — and three of the most common operations in machine learning are exactly these constrained cases.
| Kind | Constraint | Reachable set from two points | Where it shows up |
|---|---|---|---|
| linear | none | The whole plane through the origin | Span, rank, layer weights |
| affine | coefficients sum to 1 | The infinite line through both points | A layer with a bias. Regression lines |
| convex | sum to 1, all ≥ 0 | The segment between them | Averaging, interpolation, attention, mixtures |
| conic | all ≥ 0 | The wedge spanned by both directions | Non-negative matrix factorization |
Latent-space interpolation is the convex case made visual. Walking from one point to another in a generative model's latent space by t·z₁ + (1−t)·z₂ is a convex combination, and the smooth morph you see in the output is the decoder's image of that straight segment. Whether the path stays in a region the model handles well is a separate question — and the reason spherical interpolation is often preferred to the straight line.
Drills
| # | Do this | You'll know it when |
|---|---|---|
| 1 | Add and scale vectors by hand, then draw the tip-to-tail and parallelogram constructions | You can predict the sum before computing it |
| 2 | Verify all eight axioms for ℝ², then for polynomials of degree ≤ 2 | The abstraction stops feeling arbitrary |
| 3 | Find a set that fails exactly one axiom | You know which axiom does which job |
| 4 | Decide by hand whether five given subsets of ℝ³ are subspaces | You check for the origin first, automatically |
| 5 | Take three vectors in ℝ³ and determine independence via the determinant | Zero determinant and "flat" mean the same thing to you |
| 6 | Express one vector in two different bases and confirm it's the same vector | Coordinates and vectors are separate ideas in your head |
| 7 | Run PCA and reconstruct the data from the top components | Change of basis and compression are one operation |
| 8 | Write a gradient descent step and identify each of the two operations in it | Optimization looks like linear algebra |
| 9 | Interpolate between two embeddings and inspect the midpoint | Convex combination is something you've done, not read |
The one-paragraph summary
A vector space is a set equipped with exactly two operations — adding two elements and scaling one by a number — subject to eight axioms that amount to "behaves like arithmetic". Everything reachable is a linear combination, scale-then-add, and the whole vocabulary of the subject describes what those combinations can produce: the span is the reachable set, independence means no element is already reachable from the others, a basis is a minimal spanning set whose size is the dimension, and a subspace is a subset closed under both operations and therefore containing the origin. Notably absent from the definition are length, angle, and any way to multiply two vectors together — those require a norm or an inner product, which is why cosine similarity is a choice you make rather than a property vectors have. Because the axioms mention nothing about arrows, they apply equally to polynomials, functions, matrices, images, random variables and embeddings, and every theorem transfers for free. Constraining the coefficients gives affine combinations when they sum to one and convex combinations when they are additionally non-negative, which is exactly what a layer's bias term and an attention head's softmax weights respectively produce.