Linear Algebra

Linear algebra begins with the humblest object in mathematics: the straight-line equation. Before any symbols, here is one in disguise. Suppose sandwiches cost 2 dollars each and drinks cost 3 dollars each, and your total comes to 12 dollars. If s counts sandwiches and d counts drinks, the receipt says 2s + 3d = 12. Each unknown gets multiplied by a plain number, the pieces are added, and the sum is set equal to a constant. That is the entire recipe, and everything in this course grows out of it.

Here is the general form. A linear equation in the variables x1, x2, ..., xn is any equation that can be written as a1 x1 + a2 x2 + ... + an xn = b. Read it aloud: a-one times x-one, plus a-two times x-two, and so on, equals b. The coefficients a1, ..., an and the constant b are known numbers; the x's are the unknowns we want. The defining feature is that every variable appears to the first power only: no squares, no products of variables, no roots, no trig or exponential functions.

Let us test that rule on a few candidates. The equation 2x - 3y = 5 is linear: each term is a number times one variable. But x^2 + y = 1 is not, because x^2 means x times x, a second power. The equation xy = 4 is not linear either, because two different unknowns are multiplied together; its graph is a curve, not a line. And sqrt(x) + y = 1 fails because a square root is not a first power. A useful habit: scan term by term, and ask of each one, is this a plain number times a single first-power variable?

Sometimes an equation is linear in disguise. Take 3(x - 2) + y = 7. Distribute the 3: 3x - 6 + y = 7. Add 6 to both sides: 3x + y = 13. Now it matches the pattern exactly. Rewriting an equation so the variable terms sit on the left, in a fixed order, and the constant sits on the right is called putting it in standard form, and we will do it before every computation.

Key idea: linear means each variable appears alone, to the first power, multiplied by a known number.

Systems of equations

A system of linear equations is a collection of one or more linear equations in the same variables. A solution is a list of values, one for each variable, that satisfies every equation at once. Think of each equation as a separate clue about the same hidden numbers; the solution is the one story consistent with all the clues. For two variables, each equation draws a line in the plane, and a solution is a point where the lines cross.

Consider the pair:

x + y = 5 and x - y = 1

Two numbers that add to 5 and differ by 1: you might guess 3 and 2. Let us get that answer by a method that scales up, working one small step at a time.

1. Add the left sides and add the right sides: (x + y) + (x - y) = 5 + 1. This is allowed because both equations are true at the solution, and adding equal quantities to equal quantities keeps a true statement true.
2. Combine the left side: x + y + x - y = 2x, because +y and -y cancel. The right side is 6. So 2x = 6.
3. Divide both sides by 2: x = 3.
4. Put x = 3 back into x + y = 5: that reads 3 + y = 5, so y = 2.

What happened there: adding the equations made y disappear, leaving one equation in one unknown, and everything after that was ordinary arithmetic. Check the answer in both original equations: 3 + 2 = 5 and 3 - 2 = 1. Both hold, so the single solution is the point (3, 2), exactly where the two lines meet. In three variables the picture upgrades: each equation like x + y + z = 6 describes a flat plane in space, and a solution is a point lying on every plane simultaneously.

Systems come from real questions. Suppose adult movie tickets cost 9 dollars, child tickets cost 5 dollars, and a family bought 5 tickets for 33 dollars. Let a and c count adult and child tickets. Counting tickets gives a + c = 5; counting dollars gives 9a + 5c = 33.

Multiply the first equation by 5 to get 5a + 5c = 25, and subtract it from the second: (9a + 5c) - (5a + 5c) = 33 - 25, so 4a = 8 and a = 2. Then c = 3. Check the money: 9*2 + 5*3 = 18 + 15 = 33. It fits.

Key idea: a solution must satisfy every equation in the system at the same time, not one equation at a time.

The augmented matrix

Writing the variables over and over is wasteful. Once every equation is in standard form, the letters are pure scaffolding: all of the information lives in the numbers. So we record only the numbers, in a rectangular array called a matrix. The coefficient matrix holds the coefficients, and the augmented matrix attaches the right-hand-side constants as an extra column, marked off conceptually by a bar. For the system x + y = 5, x - y = 1:

Coefficient matrix	Augmented matrix
[ 1  1 ; 1 -1 ]	[ 1  1 | 5 ; 1 -1 | 1 ]

Read the layout like a table. Each row is one equation. Each column belongs to one variable, in a fixed order, and the last column holds the constants. To decode row 1 of the augmented matrix, reattach the letters: 1x + 1y = 5. Nothing was lost; we only stopped rewriting what never changes.

Two habits keep the translation honest. First, put every equation in the same variable order before copying numbers. The system y = 2x + 1 and 3x + y = 10 must first become -2x + y = 1 and 3x + y = 10; then the augmented matrix is [-2 1 | 1 ; 3 1 | 10].

Second, a missing variable is really there with coefficient 0. In the variables x, y, z, the system x + 2z = 5, y - z = 2 becomes the rows [1 0 2 | 5] and [0 1 -1 | 2]. The zeros hold each variable's place in its column.

Why bother with this bookkeeping? Because it scales. A weather model or an engineering simulation can involve thousands of equations in thousands of unknowns, and a computer stores exactly this array of numbers. Better still, the next lesson shows how to solve the system by doing arithmetic directly on the rows of the augmented matrix, with the variables never slowing us down.

Key idea: the augmented matrix is the whole system with the letters stripped away; rows are equations, columns are variables, and the last column is the constants.

Three possible outcomes

No matter how many equations or variables, a linear system has exactly one of three fates:

• Exactly one solution - the equations pin down a single point. The system is consistent and independent. Example: the crossing lines above.
• No solution - the equations contradict each other. The system is inconsistent. Example: x + y = 2 and x + y = 5 describe parallel lines that never meet.
• Infinitely many solutions - the equations are secretly the same constraint. The system is consistent and dependent. Example: x + y = 2 and 2x + 2y = 4 are the same line.

Let us see why each case looks the way it does. Solve both parallel-line equations for y: the first gives y = 2 - x and the second gives y = 5 - x. Both lines drop 1 unit for each step right (same tilt), but one sits 3 units higher. Same direction, different height: they never touch. Algebra says the same thing more bluntly. Subtract the first equation from the second: (x + y) - (x + y) = 5 - 2, which reads 0 = 3. A false sentence with no unknowns left is the algebraic signature of an inconsistent system.

Now the third case. Divide 2x + 2y = 4 by 2 and you get x + y = 2: the second equation repeats the first, contributing no new information. One line, infinitely many points. We can list them all with a parameter: let x = t, where t is any real number; then y = 2 - t, and every solution has the form (t, 2 - t). Setting t = 0 gives (0, 2); setting t = 1 gives (1, 1); setting t = 7.5 gives (7.5, -5.5). One free choice generates the whole family.

Why can a system never have, say, exactly two solutions? Suppose (0, 2) and (1, 1) both satisfy some linear system. Average them coordinate by coordinate to get (0.5, 1.5), and test it in the equation x + y = 2: the sum 0.5 + 1.5 = 2 holds. That is no accident.

For any linear equation ax + by = c, averaging two solutions gives a times the average of the x's plus b times the average of the y's, which is (c + c)/2 = c again. The same works for every equation in the system at once, and for all the in-between blends, not only the midpoint. So two distinct solutions instantly breed infinitely many, which is why the menu has exactly three options.

Key idea: one solution, none, or infinitely many; elimination will tell us which, mechanically, by producing either a unique point, a false sentence like 0 = 3, or a free parameter.

A real use: keeping traffic moving

Traffic engineers use linear systems constantly, because intersections obey a conservation rule: over an hour, cars in must equal cars out. Picture two intersections, A and B. Into A flow 400 cars per hour from the west. Out of A, x cars per hour take Main Street toward B and y cars per hour turn off on Oak Avenue. Balance at A: x + y = 400. Into B flow the x cars from Main plus 100 from a side street, and 350 leave B heading east. Balance at B: x + 100 = 350.

The second equation contains one unknown: x = 250. Feed that into the first: 250 + y = 400, so y = 150. The engineer now knows Oak Avenue must carry 150 cars per hour, perhaps deciding its signal timing. As an augmented matrix in the variables x, y, the system is [1 1 | 400 ; 1 0 | 250]. A real street grid produces dozens of balance equations, one per intersection, and the identical bookkeeping balances electric circuits, where currents into a junction must equal currents out, and even chemical equations. Whenever quantities must balance at many points simultaneously, a linear system appears.

Common wrong turns

• Calling xy = 4 linear. Two unknowns multiplied together is a product of variables, which the definition forbids; the graph is a hyperbola, not a line. Test every term one at a time.
• Copying coefficients before lining up the variables. If one equation is written y + x = 5 and the next x - y = 1, copying numbers left to right scrambles the columns, and the matrix describes a different system. Standard form first, always.
• Skipping the 0 for a missing variable. In x + 2z = 5 with variables x, y, z, the row is [1 0 2 | 5]. Dropping the 0 shifts the 2 into the y-column, silently changing the equation to x + 2y = 5.
• Checking a guess in only one equation. The point (4, 1) satisfies x + y = 5 but fails x - y = 1, since 4 - 1 = 3. Membership in the solution set requires passing every equation.
• Counting equations to predict solutions. Two equations in two unknowns often give one solution, but not always: parallel lines give none and repeated lines give infinitely many. The structure of the equations, not their count, decides.

Try it

Solve the system x + 2y = 7, 3x - 2y = 5 by elimination, then write its augmented matrix.

Worked answer. Add the two equations: (x + 2y) + (3x - 2y) = 7 + 5. The +2y and -2y cancel, leaving 4x = 12, so x = 3. Substitute into the first equation: 3 + 2y = 7, so 2y = 4 and y = 2. The solution is (3, 2). Check: 3 + 2*2 = 7 and 3*3 - 2*2 = 9 - 4 = 5. Both pass. The augmented matrix is [1 2 | 7 ; 3 -2 | 5].

Recap

• A linear equation is a sum of number-times-variable terms set equal to a constant; first powers only.
• A system's solution satisfies every equation simultaneously; geometrically, it is where lines or planes intersect.
• The augmented matrix records the system as pure numbers: rows are equations, columns are variables, last column is constants.
• Exactly three outcomes exist: one solution, no solution, or infinitely many. Two distinct solutions force infinitely many.

Next we turn elimination, the trick that made y vanish, into a complete algorithm that runs on the augmented matrix itself: Gaussian elimination.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 1.1: Systems of linear equations. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 2.4: Solution sets. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net
4. Boyd, S., & Vandenberghe, L. (n.d.). Introduction to applied linear algebra: Vectors, matrices, and least squares. Stanford University. web.stanford.edu
5. Beezer, R. (n.d.). A first course in linear algebra. American Institute of Mathematics Open Textbook Initiative. textbooks.aimath.org
6. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu

Gaussian elimination is a systematic procedure that solves any linear system, no matter its size, by simplifying its augmented matrix step by step. In the last lesson we added two equations and a variable vanished. Gaussian elimination turns that one-off trick into a complete algorithm with a guaranteed finish line. It uses three elementary row operations, each of which changes the matrix without changing its solution set:

1. Swap two rows.
2. Scale a row by a nonzero constant.
3. Replace a row by itself plus a multiple of another row.

The goal is row echelon form (REF): all zero rows sit at the bottom, and the first nonzero entry of each row, its pivot, lies to the right of the pivot above it, creating a staircase of zeros in the lower-left. Once the matrix looks like a staircase, the last equation involves one variable, which we solve and substitute upward, a process called back-substitution.

Why row operations are safe

Before running the machine, it is worth seeing why these three moves are legal. Each row is an equation, and each operation is something we were always allowed to do to equations. Swapping rows only reorders the list; the clues are the same clues in a different order. Scaling a row multiplies both sides of one equation by the same nonzero number: if x - y = 1 is true, then 2x - 2y = 2 is true for exactly the same pairs, and dividing by 2 brings the original back.

The replacement move is the workhorse. Replacing R2 with R2 + 3*R1 (read: row two plus three times row one) adds equal quantities to both sides of equation two, because whatever the solution is, row one's left side equals its right side there. A true statement plus a true statement is still true. And the move is reversible: subtract 3*R1 again and nothing is lost. Reversibility is the whole point. Because every move can be undone, the new system and the old system are true for exactly the same values, so the solution set never changes.

That reversibility requirement is also why scaling by zero is banned. Multiplying a row by 0 turns an equation into 0 = 0, which erases its information forever; there is no way to divide by zero and recover it. The three legal moves rearrange information without ever destroying it.

Key idea: row operations are reversible restatements of the system, so the simplified matrix has exactly the same solutions as the original.

Worked example, a 3x3 system

Solve the system whose augmented matrix is shown, using the rows R1, R2, R3:

R1: [1 1 1 | 6]   R2: [0 2 5 | -4]   R3: [2 5 -1 | 27]

This came from x + y + z = 6, 2y + 5z = -4, and 2x + 5y - z = 27. The strategy is always the same: clear out one column at a time, left to right, using the pivot in that column to wipe out everything below it.

1. Column 1 has its pivot, the 1 in R1. The only nonzero entry below it is the 2 in R3, so replace R3 with R3 - 2*R1, entry by entry: 2 - 2*1 = 0, then 5 - 2*1 = 3, then -1 - 2*1 = -3, and on the right side 27 - 2*6 = 15. The new R3 is [0 3 -3 | 15].
2. Column 2 has its pivot, the 2 in R2, and a 3 below it in R3. The multiplier is 3/2, so replace R3 with R3 - (3/2)*R2: first entry 0 - 0 = 0, then 3 - (3/2)(2) = 3 - 3 = 0, then -3 - (3/2)(5) = -3 - 7.5 = -10.5, and 15 - (3/2)(-4) = 15 + 6 = 21. The new R3 is [0 0 -10.5 | 21].
3. The matrix is now in echelon form:
   [1 1 1 | 6]
[0 2 5 | -4]
[0 0 -10.5 | 21]

Notice the fraction 3/2 did not stop us. Fractions and decimals are routine guests in elimination; the arithmetic rules are unchanged. Now back-substitute from the bottom. Row 3 says -10.5 z = 21; divide both sides by -10.5 to get z = -2. Row 2 says 2y + 5(-2) = -4, that is 2y - 10 = -4; add 10 to both sides for 2y = 6, so y = 3. Row 1 says x + 3 + (-2) = 6, that is x + 1 = 6, so x = 5.

The unique solution is (5, 3, -2). Verify it in the original third equation, the one we modified most: 2(5) + 5(3) - (-2) = 10 + 15 + 2 = 27. Correct. And in the first: 5 + 3 + (-2) = 6. Correct again. A thirty-second check like this catches most arithmetic slips, and it is worth doing every time.

Reading the staircase: what echelon form promises

Why insist on the staircase shape? Because it makes the system solvable by pure momentum. The bottom nonzero row contains exactly one unknown, so one division finds it. The row above contains that unknown plus one new one, so substitution plus one step of algebra finds the new one. Every row up the staircase introduces exactly one fresh variable. Back-substitution is not a second technique to memorize; it is what any triangular system invites you to do, bottom to top.

Key idea: elimination trades many tangled equations for a staircase in which each row hands you one new variable.

Reduced row echelon form and free variables

If we keep going, scaling each pivot to 1 and clearing the entries above each pivot too, we reach reduced row echelon form (RREF), where the solution can be read off with no back-substitution at all.

Let us finish the worked example. Scale R3 by 1/(-10.5) to get [0 0 1 | -2]. Then clear column 3 upward: replace R2 with R2 - 5*R3, giving [0 2 0 | 6], and R1 with R1 - R3, giving [1 1 0 | 8]. Scale R2 by 1/2 to get [0 1 0 | 3], then replace R1 with R1 - R2: [1 0 0 | 5].

The final matrix is [1 0 0 | 5 ; 0 1 0 | 3 ; 0 0 1 | -2], which spells out x = 5, y = 3, z = -2 directly. RREF costs a few more operations than REF, but the answer sits on the surface, and unlike REF, the RREF of a matrix is unique.

RREF also exposes systems with infinitely many solutions. A column without a pivot corresponds to a free variable, one we may set to any value. For instance, if reduction yields

[1 0 2 | 3]   [0 1 -1 | 4]   [0 0 0 | 0]

then column 3 has no pivot, so z is free. Writing z = t, the first row says x + 2t = 3, so x = 3 - 2t, and the second says y - t = 4, so y = 4 + t. The solution set is the infinite family (3 - 2t, 4 + t, t). Spot-check it: t = 0 gives (3, 4, 0) and t = 1 gives (1, 5, 1), and both satisfy x + 2z = 3.

Finally, a row like [0 0 0 | 5] would say 0 = 5, a contradiction signaling an inconsistent system with no solution. Gaussian elimination thus decides all three cases automatically.

A second full run, with a free variable

Solve x + 2y + z = 4, 2x + 4y + 3z = 9, -x - 2y + z = -2. The augmented matrix is [1 2 1 | 4 ; 2 4 3 | 9 ; -1 -2 1 | -2].

1. Clear column 1. Replace R2 with R2 - 2*R1: 2 - 2 = 0, 4 - 4 = 0, 3 - 2 = 1, 9 - 8 = 1, giving [0 0 1 | 1]. Replace R3 with R3 + R1: -1 + 1 = 0, -2 + 2 = 0, 1 + 1 = 2, -2 + 4 = 2, giving [0 0 2 | 2].
2. Column 2 has no pivot available: both remaining rows have 0 there. That is fine; move to column 3, whose pivot is the 1 in R2. Replace R3 with R3 - 2*R2: 2 - 2 = 0 and 2 - 2 = 0, a zero row.

The echelon form is [1 2 1 | 4 ; 0 0 1 | 1 ; 0 0 0 | 0], with pivots in columns 1 and 3. Column 2 is pivot-free, so y is free: set y = t. Row 2 gives z = 1. Row 1 gives x + 2t + 1 = 4, so x = 3 - 2t. The solutions form the family (3 - 2t, t, 1), a line in space.

Check the middle original equation for every t at once: 2(3 - 2t) + 4t + 3(1) = 6 - 4t + 4t + 3 = 9. The t's cancel, so every member of the family works. For contrast, changing the last equation's constant from -2 to 1 would have produced the row [0 0 0 | 3] instead of a zero row: the false sentence 0 = 3, and no solution at all.

A real use: currents in a circuit

Electrical engineers generate systems like these from two physical laws. At any junction, current in equals current out (charge is conserved), and around any closed loop, the battery's voltage is used up by the resistors (energy is conserved). Suppose a 9-volt battery drives current i1 through a 1-ohm resistor, after which the wire splits into two branches: i2 through a 3-ohm resistor and i3 through a 6-ohm resistor. The laws give three equations: i1 - i2 - i3 = 0, then i1 + 3 i2 = 9, then 3 i2 - 6 i3 = 0.

Eliminate. The augmented matrix is [1 -1 -1 | 0 ; 1 3 0 | 9 ; 0 3 -6 | 0]. Replace R2 with R2 - R1: [0 4 1 | 9]. Replace R3 with R3 - (3/4)*R2: 3 - 3 = 0, -6 - (3/4)(1) = -6.75, 0 - (3/4)(9) = -6.75, giving [0 0 -6.75 | -6.75]. So i3 = 1 amp.

Back-substitute: 4 i2 + 1 = 9 gives i2 = 2 amps, and i1 = i2 + i3 = 3 amps. Check the loop: 3 + 3(2) = 9. The algorithm you now know is, entry for entry, what circuit simulation software performs millions of times per second.

Common wrong turns

• Forgetting the constants column. A row operation applies to the entire row, bar included. Computing R3 - 2*R1 on the coefficients but not on the constant changes the system into a different one with different solutions.
• Scaling a row by zero. It converts an equation to 0 = 0, deleting information irreversibly. Scale only by nonzero numbers.
• Using a row you already changed in this step. Perform one operation at a time, writing the new matrix before the next move. Combining two replacements in your head is where sign errors breed.
• Refusing the swap. If the pivot position holds a 0, as in [0 1 | 2 ; 3 4 | 5], you cannot clear a column with it. Swap rows first to bring a nonzero entry into the pivot seat.
• Expecting echelon form to be unique. Different legal move orders give different REFs, and that is fine; the solution set they describe is identical. Only the fully reduced RREF is unique.
• Letting fractions reroute you. Multipliers like 3/2 are legitimate. If you prefer, scale a row first (for example, double R3 before subtracting 3*R2); reversible moves cost nothing.

Try it

Solve by elimination: 2x + y = 8, 4x + 3y = 18.

Worked answer. The augmented matrix is [2 1 | 8 ; 4 3 | 18]. Replace R2 with R2 - 2*R1: 4 - 4 = 0, 3 - 2 = 1, 18 - 16 = 2, giving [0 1 | 2]. That row says y = 2. Back-substitute: 2x + 2 = 8, so 2x = 6 and x = 3. Solution: (3, 2). Check: 4(3) + 3(2) = 12 + 6 = 18. It holds.

Recap

• Three row operations, all reversible, preserve the solution set: swap, scale by a nonzero constant, and add a multiple of one row to another.
• Forward elimination clears columns left to right, producing a staircase (REF); back-substitution climbs it from the bottom.
• Continuing to RREF puts a lone 1 in each pivot position and reveals the answer, or the free variables, at a glance.
• A pivot-free column means a free variable and an infinite family; a row reading 0 = nonzero means no solution.

With solving under control, the next module treats matrices as objects in their own right, with an arithmetic of addition, multiplication, and, eventually, inverses.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 1.2: Row reduction. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 1.3: Parametric form. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Margalit, D., & Rabinoff, J. (n.d.). 2.4: Solution sets. In Interactive linear algebra. LibreTexts. math.libretexts.org
4. O'Connor, J. J., & Robertson, E. F. (n.d.). Carl Friedrich Gauss (1777-1855). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
5. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net
6. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu

A matrix is a rectangular array of numbers arranged in rows and columns, like a small spreadsheet. A matrix with m rows and n columns is called an m x n matrix (read: m by n), and the number in row i, column j is its (i, j) entry. So in the 2 x 3 matrix M = [4 7 0 ; 2 -1 5], the (2, 3) entry is 5: second row, third column. Rows are always named first, in both the size and the entry address.

Matrices are not only bookkeeping for systems of equations; they are objects with their own arithmetic. You can add them, scale them, and, most importantly, multiply them. This lesson builds that arithmetic one operation at a time, with every computation written out, because matrix multiplication follows a rule you would probably not guess on your own, and there is a good reason behind it.

Addition and scalar multiplication

Two matrices of the same size are added entry by entry: top-left with top-left, and so on across the whole array. A matrix is multiplied by a scalar (a single number) by multiplying every entry. For example, with

A = [1 2 ; 3 4] and B = [5 6 ; 7 8]

the sum adds matching positions: A + B = [1+5 2+6 ; 3+7 4+8] = [6 8 ; 10 12]. Scaling triples every entry: 3A = [3*1 3*2 ; 3*3 3*4] = [3 6 ; 9 12]. Subtraction works the same way, position by position: B - A = [5-1 6-2 ; 7-3 8-4] = [4 4 ; 4 4].

Because each position is handled independently, addition inherits the comfortable habits of ordinary numbers: A + B = B + A, and grouping does not matter. The zero matrix, all entries 0, plays the role of the number zero: adding it changes nothing. One rule has no counterpart for numbers: sizes must match exactly. A 2 x 2 matrix plus a 2 x 3 matrix is undefined, because some entries would have no partner to pair with.

Key idea: addition and scalar multiplication work position by position, and only same-size matrices can be added.

A matrix times a vector

Before multiplying two matrices, let us multiply a matrix by a single column vector, the case that runs all of linear algebra. The rule: the result is a column, and each entry is one row of the matrix dotted with the vector, meaning multiply matching entries and add. Compute [1 2 ; 3 4][5 ; 6] one row at a time. Row 1: 1*5 + 2*6 = 5 + 12 = 17. Row 2: 3*5 + 4*6 = 15 + 24 = 39. So the product is [17 ; 39].

There is a second way to read the same computation, and it is worth knowing both. The product is a linear combination of the columns, weighted by the vector's entries: 5[1 ; 3] + 6[2 ; 4] = [5 ; 15] + [12 ; 24] = [17 ; 39]. Same answer, different story. The row picture computes one output entry at a time; the column picture says the output is built from the matrix's columns. The column picture explains why Ax = b asks whether b can be assembled from the columns of A, a question that returns in Module 4.

Matrix multiplication

Now the full rule. The product AB is defined only when the number of columns of A equals the number of rows of B. If A is m x n and B is n x p, then AB is m x p: the inner sizes must agree and then they cancel, leaving the outer sizes. The entry in row i, column j of AB is the dot product of row i of A with column j of B. Equivalently, column j of AB is the matrix A times column j of B: matrix-vector products, done once per column.

Worked example. Multiply the 2x2 matrices A = [1 2 ; 3 4] and B = [5 6 ; 7 8].

• Top-left: row 1 of A dotted with column 1 of B: 1*5 + 2*7 = 5 + 14 = 19.
• Top-right: row 1 with column 2: 1*6 + 2*8 = 6 + 16 = 22.
• Bottom-left: row 2 with column 1: 3*5 + 4*7 = 15 + 28 = 43.
• Bottom-right: row 2 with column 2: 3*6 + 4*8 = 18 + 32 = 50.

So AB = [19 22 ; 43 50]. A useful habit while learning: point at the row with your left hand and the column with your right, and sweep them together as you multiply and add.

A rectangular example. Let A = [1 0 2 ; -1 3 1] (size 2 x 3) and B = [3 1 ; 2 0 ; 1 4] (size 3 x 2). The inner sizes are both 3, so AB exists and is 2 x 2.

Entry by entry: top-left is 1*3 + 0*2 + 2*1 = 3 + 0 + 2 = 5; top-right is 1*1 + 0*0 + 2*4 = 1 + 0 + 8 = 9; bottom-left is -1*3 + 3*2 + 1*1 = -3 + 6 + 1 = 4; bottom-right is -1*1 + 3*0 + 1*4 = -1 + 0 + 4 = 3. So AB = [5 9 ; 4 3].

Note that BA is also defined here, but it is 3 x 3, a different size entirely. The order of the factors is not a cosmetic detail.

Key idea: match inner sizes, then build each entry as a row-times-column dot product; each column of AB is A applied to a column of B.

Order matters

Unlike ordinary numbers, matrix multiplication is not commutative: in general AB is not equal to BA. Reversing the square product above gives BA = [5*1 + 6*3, 5*2 + 6*4 ; 7*1 + 8*3, 7*2 + 8*4] = [23 34 ; 31 46], a completely different matrix from AB = [19 22 ; 43 50]. The deep reason arrives in Module 6: matrices represent actions, and doing action B then action A rarely matches doing A then B, in the same way that putting on socks then shoes differs from shoes then socks.

The identity matrix I, with 1s on the main diagonal and 0s elsewhere, is the one matrix that behaves like the number 1: AI = IA = A for any square A. Multiplication also distributes over addition, A(B + C) = AB + AC, and is associative, (AB)C = A(BC), so long chains of products can be grouped however is convenient. Only the freedom to swap factors is lost.

Losing commutativity has side effects worth seeing once with real numbers. Two nonzero matrices can multiply to zero: with A = [1 0 ; 0 0] and B = [0 0 ; 0 1], every entry of AB is a product involving a 0, and AB = [0 0 ; 0 0].

Cancellation fails for the same reason: with that same A, both A[1 0 ; 0 1] and A[1 0 ; 0 5] equal [1 0 ; 0 0], so AB = AC does not force B = C. And expanding (A + B)^2 gives A^2 + AB + BA + B^2; the middle terms cannot be merged into 2AB unless A and B happen to commute.

Key idea: keep every product in its written order; the familiar rules of algebra survive except the ones that quietly assume swapping.

Powers and the transpose

Two small pieces of vocabulary complete the toolkit. For a square matrix, A^2 means A*A, with the same row-by-column rule. For A = [1 2 ; 3 4]: top-left 1*1 + 2*3 = 7, top-right 1*2 + 2*4 = 10, bottom-left 3*1 + 4*3 = 15, bottom-right 3*2 + 4*4 = 22, so A^2 = [7 10 ; 15 22].

The transpose A^T flips a matrix across its main diagonal, turning rows into columns: [1 2 ; 3 4]^T = [1 3 ; 2 4]. It will star in the least-squares story of Module 8; for now, know the notation and the reversal rule (AB)^T = B^T A^T.

A real use: counting routes in a network

Here is multiplication earning its keep. An airline serves cities 1, 2, and 3 with one-way flights: 1 to 2, 1 to 3, 2 to 3, and 3 to 1. Record them in an adjacency matrix M whose (i, j) entry is 1 if there is a flight from city i to city j, else 0: M = [0 1 1 ; 0 0 1 ; 1 0 0].

Now compute M^2 row by row. Row 1: (0*0 + 1*0 + 1*1, 0*1 + 1*0 + 1*0, 0*1 + 1*1 + 1*0) = (1, 0, 1). Row 2: (1, 0, 0). Row 3: (0, 1, 1). So M^2 = [1 0 1 ; 1 0 0 ; 0 1 1].

The remarkable part: the (i, j) entry of M^2 counts the two-leg routes from i to j. The 1 in position (1, 3) is the route 1 to 2 to 3; the 1 in position (1, 1) is the round trip 1 to 3 to 1. The dot product sums, over every stopover city k, flight(i to k) times flight(k to j), which is exactly a count of connections. Higher powers count longer routes, which is how network software measures reachability.

Common wrong turns

• Multiplying entry by entry. The tempting rule [1 2 ; 3 4][5 6 ; 7 8] = [5 12 ; 21 32] is not matrix multiplication and breaks the connection to systems and transformations. Rows dot columns, always.
• Ignoring the size test. A 2 x 3 times a 2 x 3 is undefined: the inner numbers read 3 then 2, and they disagree. Write the sizes side by side, (2 x 3)(3 x 2), before computing anything.
• Swapping factors mid-calculation. Replacing AB with BA silently changes the answer, and sometimes even the size of the answer.
• Cancelling matrices like numbers. From AB = AC you may not conclude B = C, and from AB = 0 you may not conclude that A or B is zero; the examples above break both.
• Expanding squares by habit. (A + B)^2 = A^2 + AB + BA + B^2. Writing 2AB assumes commutativity you do not have.

Try it

Let A = [0 1 ; 1 0] and B = [2 3 ; 4 5]. Compute AB and BA.

Worked answer. For AB: top-left 0*2 + 1*4 = 4, top-right 0*3 + 1*5 = 5, bottom-left 1*2 + 0*4 = 2, bottom-right 1*3 + 0*5 = 3, so AB = [4 5 ; 2 3]. For BA: top-left 2*0 + 3*1 = 3, top-right 2*1 + 3*0 = 2, bottom-left 4*0 + 5*1 = 5, bottom-right 4*1 + 5*0 = 4, so BA = [3 2 ; 5 4]. They differ, and there is a pattern worth noticing: multiplying by this A on the left swapped B's rows, while multiplying on the right swapped B's columns.

Recap

• Same-size matrices add entry by entry; scalars multiply every entry; the zero matrix is the additive identity.
• A matrix times a vector is a dot product per row, or equivalently a weighted combination of the columns.
• AB needs columns of A to match rows of B; entry (i, j) is row i of A dotted with column j of B.
• Multiplication is associative and distributive but not commutative, and cancellation can fail.

Next we ask when a matrix can be undone: the inverse, the matrix version of a reciprocal.

Sources

1. Kuttler, K. (n.d.). 2.1: Matrix arithmetic. In A first course in linear algebra. LibreTexts. math.libretexts.org
2. Kuttler, K. (n.d.). 2.2: Multiplication of matrices. In A first course in linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 2.4: Properties of matrix multiplication. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Margalit, D., & Rabinoff, J. (n.d.). 3.4: Matrix multiplication. In Interactive linear algebra. LibreTexts. math.libretexts.org
5. O'Connor, J. J., & Robertson, E. F. (n.d.). Arthur Cayley (1821-1895). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
6. Boyd, S., & Vandenberghe, L. (n.d.). Introduction to applied linear algebra: Vectors, matrices, and least squares. Stanford University. web.stanford.edu

For an ordinary nonzero number a, the reciprocal 1/a undoes multiplication: a * (1/a) = 1. Multiply by 5, then by 1/5, and you are back where you started. The matrix version of this undo button is the inverse. A square matrix A is invertible (or nonsingular) if there is a matrix A^(-1) (read: A inverse) with

A * A^(-1) = A^(-1) * A = I,

where I is the identity matrix, the do-nothing matrix with 1s on the diagonal. Think of A as a machine that scrambles vectors; A^(-1) is the machine that unscrambles them, and running the two in either order lands you back at the start. Not every matrix has an inverse. A matrix that lacks one is called singular. Only square matrices can be invertible, and even then it depends on the entries.

One reassuring fact before computing: a matrix never has two different inverses. If B and C both undid A, then B = B I = B (A C) = (B A) C = I C = C, so the two candidates were the same matrix all along. Each step there uses only associativity and the definition of an inverse, and it means we may speak of the inverse of A.

The 2x2 inverse formula

For a 2x2 matrix there is a clean shortcut. If A = [a b ; c d], define the number det(A) = ad - bc, called the determinant. Then:

A^(-1) = (1 / (ad - bc)) * [d -b ; -c a]

In words: swap the diagonal entries, negate the off-diagonal entries, and divide everything by ad - bc. The formula works if and only if ad - bc is not zero. If ad - bc = 0, the matrix is singular and has no inverse.

Why does swapping and negating work? Multiply the two matrices symbolically, before any division. The top-left entry of [a b ; c d][d -b ; -c a] is ad + b(-c) = ad - bc. The top-right entry is a(-b) + ba = 0. The bottom-left is cd + d(-c) = 0. The bottom-right is c(-b) + da = ad - bc. So the product is (ad - bc) times the identity, and dividing by ad - bc lands exactly on I. The formula is not a trick to memorize on faith; it is the unique matrix that makes the product collapse to the identity.

Worked example. Invert A = [4 7 ; 2 6].

1. Compute the determinant: ad - bc = 4*6 - 7*2 = 24 - 14 = 10. Nonzero, so the inverse exists.
2. Swap and negate: [d -b ; -c a] = [6 -7 ; -2 4].
3. Divide by 10: A^(-1) = [0.6 -0.7 ; -0.2 0.4].

Check every entry of A * A^(-1). Top-left: 4*0.6 + 7*(-0.2) = 2.4 - 1.4 = 1. Top-right: 4*(-0.7) + 7*0.4 = -2.8 + 2.8 = 0. Bottom-left: 2*0.6 + 6*(-0.2) = 1.2 - 1.2 = 0. Bottom-right: 2*(-0.7) + 6*0.4 = -1.4 + 2.4 = 1. The product is [1 0 ; 0 1], as required.

Now a matrix with no inverse: B = [2 4 ; 1 2]. Its determinant is 2*2 - 4*1 = 0. Geometrically, both columns point along the same line (the second is twice the first), so the machine B flattens the whole plane onto that line. Two different inputs get squashed to the same output, and no machine can look at the output and recover which input it came from. That is what singular means in practice: information has been destroyed.

Key idea: the 2x2 inverse is swap, negate, divide by the determinant, and it exists exactly when ad - bc is nonzero.

Inverses by row reduction: the [A | I] method

The swap-and-negate shortcut is special to 2x2 matrices. The general method uses Gaussian elimination. Write the block matrix [A | I], with the identity glued to the right of A, and row-reduce until the left block becomes I. Whatever the right block has become is A^(-1). The logic: each row operation is itself an invertible action, and the sequence of actions that converts A to I is, in total, exactly multiplication by A^(-1). Applying that same sequence to I records the product for you.

Worked example. Invert A = [1 2 ; 3 4] by row reduction, starting from [1 2 | 1 0 ; 3 4 | 0 1].

1. Clear below the first pivot. Replace R2 with R2 - 3*R1: 3 - 3 = 0, 4 - 6 = -2, 0 - 3 = -3, 1 - 0 = 1, giving [0 -2 | -3 1].
2. Scale R2 by -1/2 to make its pivot 1: [0 1 | 1.5 -0.5].
3. Clear above the second pivot. Replace R1 with R1 - 2*R2: 1 - 0 = 1, 2 - 2 = 0, 1 - 3 = -2, 0 - (-1) = 1, giving [1 0 | -2 1].

The left block is now the identity, so A^(-1) = [-2 1 ; 1.5 -0.5]. Cross-check with the formula: det(A) = 1*4 - 2*3 = -2, and (1/-2)[4 -2 ; -3 1] = [-2 1 ; 1.5 -0.5]. The two methods agree, as they must. For a 3x3 or larger matrix, this elimination route is the standard one, and if the left block ever produces a zero row, stop: the matrix is singular and no inverse exists.

Solving Ax = b with the inverse

Any linear system can be written as a single matrix equation Ax = b, where A is the coefficient matrix, x is the column of unknowns, and b is the column of constants. If A is invertible, multiply both sides on the left by A^(-1):

A^(-1) A x = A^(-1) b, so x = A^(-1) b.

For example, to solve 4x + 7y = 1, 2x + 6y = 3, use the inverse computed earlier and multiply: x = A^(-1) b = [0.6 -0.7 ; -0.2 0.4][1 ; 3]. The first entry is 0.6*1 + (-0.7)*3 = 0.6 - 2.1 = -1.5, and the second is -0.2*1 + 0.4*3 = -0.2 + 1.2 = 1. So x = -1.5, y = 1. Check in the first equation: 4(-1.5) + 7(1) = -6 + 7 = 1. It holds.

The real payoff appears when the same coefficients face many different right-hand sides, a daily situation in engineering: same structure, different loads. Solve 4x + 7y = 10, 2x + 6y = 8 with the same inverse: first entry 0.6*10 + (-0.7)*8 = 6 - 5.6 = 0.4, second entry -0.2*10 + 0.4*8 = -2 + 3.2 = 1.2. Two multiplications, no new elimination. One caution for later courses: for large systems, software solves Ax = b by elimination directly rather than forming A^(-1), which is slower and less accurate at scale. The inverse is the right concept; elimination is the right computation.

Properties of inverses

• Double undo: (A^(-1))^(-1) = A. Undoing the undo restores the original action.
• Products reverse order: (AB)^(-1) = B^(-1) A^(-1). To undo "apply B, then A," you must undo A first, then B, the socks-and-shoes rule. Verify: (AB)(B^(-1) A^(-1)) = A (B B^(-1)) A^(-1) = A I A^(-1) = I.
• Transpose commutes with inverse: (A^T)^(-1) = (A^(-1))^T.
• Scalars pull out reciprocally: (kA)^(-1) = (1/k) A^(-1) for nonzero k, since doubling a machine's output is undone by halving.

A real use: encoding and decoding messages

A classic application is the Hill cipher. Turn letters into numbers (A = 1, B = 2, and so on), pack them into vectors, and encode by multiplying with a key matrix. Take the key K = [2 1 ; 1 1], whose determinant is 2*1 - 1*1 = 1, and the message block "HE", the vector [8 ; 5]. Encoding: K[8 ; 5] = [2*8 + 1*5 ; 1*8 + 1*5] = [21 ; 13], the letters "UM". The scrambled block looks like noise.

The receiver decodes with the inverse. Swap and negate with determinant 1: K^(-1) = [1 -1 ; -1 2]. Then K^(-1)[21 ; 13] = [1*21 - 1*13 ; -1*21 + 2*13] = [8 ; 5]: "HE" again, exactly. The same undo idea powers computer graphics, where moving a camera means applying the inverse of the world transformation, and robotics, where a controller inverts the matrix linking joint speeds to hand speeds.

Common wrong turns

• Writing division of matrices. Expressions like B/A are ambiguous because order matters: A^(-1)B and B A^(-1) usually differ. Always write which side the inverse multiplies on.
• Treating 1/A as entrywise. The inverse is not the matrix of reciprocals. For A = [1 2 ; 3 4], the entrywise guess [1 0.5 ; 1/3 0.25] fails the test A A^(-1) = I; the true inverse is [-2 1 ; 1.5 -0.5].
• Inverting a sum term by term. (A + B)^(-1) is not A^(-1) + B^(-1). With A = B = I: the left side is (2I)^(-1) = 0.5 I, while the right side is 2I.
• Forgetting the order reversal. (AB)^(-1) = B^(-1) A^(-1), not A^(-1) B^(-1). Shoes off before socks.
• Inverting a nonsquare matrix. A 2 x 3 matrix maps R^3 to R^2 and must merge inputs along the way; no two-sided inverse can exist.
• Skipping the determinant check. Applying swap-negate-divide to [2 4 ; 1 2] divides by zero. Compute ad - bc first, every time.

Try it

Invert A = [3 5 ; 1 2], then use the inverse to solve 3x + 5y = 11, x + 2y = 4.

Worked answer. Determinant: 3*2 - 5*1 = 6 - 5 = 1. Swap and negate: A^(-1) = (1/1)[2 -5 ; -1 3] = [2 -5 ; -1 3]. Solve with b = [11 ; 4]: first entry 2*11 + (-5)*4 = 22 - 20 = 2, second entry -1*11 + 3*4 = -11 + 12 = 1. So x = 2, y = 1. Check: 3*2 + 5*1 = 11 and 2 + 2 = 4. Both hold.

Recap

• The inverse is the undo matrix: A A^(-1) = A^(-1) A = I, and it is unique when it exists.
• For 2x2: swap the diagonal, negate the off-diagonal, divide by ad - bc; a zero determinant means singular.
• In general, row-reduce [A | I] until the left block is I; the right block is then A^(-1).
• The solution of Ax = b is x = A^(-1) b, and (AB)^(-1) = B^(-1) A^(-1).

The number ad - bc kept deciding everything in this lesson. The next module gives that number its own theory: the determinant.

Sources

1. Kuttler, K. (n.d.). 2.6: The identity and inverses. In A first course in linear algebra. LibreTexts. math.libretexts.org
2. Kuttler, K. (n.d.). 2.7: Finding the inverse of a matrix. In A first course in linear algebra. LibreTexts. math.libretexts.org
3. Margalit, D., & Rabinoff, J. (n.d.). 3.5: Matrix inverses. In Interactive linear algebra. LibreTexts. math.libretexts.org
4. Margalit, D., & Rabinoff, J. (n.d.). 3.6: The invertible matrix theorem. In Interactive linear algebra. LibreTexts. math.libretexts.org
5. Weisstein, E. W. (n.d.). Matrix inverse. In MathWorld. Wolfram Research. mathworld.wolfram.com
6. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

The determinant is a single number attached to a square matrix that captures crucial information: whether the matrix is invertible, and geometrically, how it scales area or volume. We write it det(A) or with vertical bars around the array. One number summarizing a whole grid sounds too good to be true, but the last lesson already showed it in action: the 2x2 inverse exists exactly when ad - bc is nonzero. This lesson learns to compute that number for 2x2 and 3x3 matrices, several ways, and to predict how it responds to row operations.

The 2x2 determinant

For A = [a b ; c d], the determinant is det(A) = ad - bc: the product of the main diagonal minus the product of the anti-diagonal. Read it aloud for [3 1 ; 5 2]: three times two, minus one times five. So det = 6 - 5 = 1. When the determinant is zero the matrix is singular; when it is nonzero the matrix is invertible.

A zero is worth seeing once. For [2 4 ; 1 2], the determinant is 2*2 - 4*1 = 0. Look at the columns: [4 ; 2] is twice [2 ; 1]. The two columns point along one line, so the parallelogram they would normally span has collapsed flat, and zero area is exactly what det = 0 reports. The determinant is already doing geometry; the next lesson makes that official.

Two more quick reads build intuition. The identity has det(I) = 1*1 - 0*0 = 1: the do-nothing matrix scales nothing. A diagonal matrix multiplies along its diagonal: det[3 0 ; 0 2] = 3*2 - 0*0 = 6, the natural guess for a machine that stretches one axis by 3 and the other by 2. Keep these anchors in mind; every method below must agree with them.

Key idea: for 2x2, cross-multiply and subtract; zero means collapsed and singular.

The 3x3 determinant by cofactor expansion

For a 3x3 matrix we expand along the first row. Each entry is multiplied by the 2x2 determinant of the matrix left after deleting that entry's row and column (its minor), with a checkerboard of signs + - +. For

A = [a b c ; d e f ; g h i],

det(A) = a(ei - fh) - b(di - fg) + c(dh - eg).

The sign attached to each position comes from a simple rule: position (i, j) carries the sign of (-1)^(i+j). Row 1 gives plus, minus, plus. The full 3x3 checkerboard reads + - + ; - + - ; + - +. The middle minus in the first row is the one everyone forgets at least once, so say the pattern to yourself as you write: plus, minus, plus.

Worked example. Compute the determinant of

A = [2 1 0 ; 1 3 1 ; 0 2 1].

1. First term: the entry 2, times the minor left after deleting row 1 and column 1. That minor is [3 1 ; 2 1] with determinant 3*1 - 1*2 = 1. Contribution: +2 * 1 = 2.
2. Second term: the entry 1, times the minor [1 1 ; 0 1], whose determinant is 1*1 - 1*0 = 1. The checkerboard says minus: contribution -1 * 1 = -1.
3. Third term: the entry 0, times the minor [1 3 ; 0 2]. Whatever that minor is, zero times it is 0.
4. Add: 2 - 1 + 0 = 1.

So det(A) = 1. Because it is nonzero, this matrix is invertible.

Here is the useful secret: you may expand along any row or any column, keeping each entry's checkerboard sign, and the answer never changes.

Let us redo the same matrix down column 3, whose entries are 0, 1, 1 with signs plus, minus, plus. The 0 contributes nothing. The entry 1 in position (2, 3) takes minor [2 1 ; 0 2], determinant 4 - 0 = 4, with a minus sign: -4. The entry 1 in position (3, 3) takes minor [2 1 ; 1 3], determinant 6 - 1 = 5, with a plus sign: +5. Total: -4 + 5 = 1. Same answer, less work. Strategy: expand along the row or column with the most zeros.

Key idea: cofactor expansion is entry times minor times checkerboard sign, along any single row or column you like.

The diagonal shortcut for 3x3

A quick alternative (which works only for 3x3) sums the three full down-right diagonals and subtracts the three full down-left diagonals: det = aei + bfg + cdh - ceg - bdi - afh. Applying it to the same matrix: down-right gives 2*3*1 + 1*1*0 + 0*1*2 = 6 + 0 + 0 = 6; down-left gives 0*3*0 + 1*1*1 + 2*1*2 = 0 + 1 + 4 = 5; the determinant is 6 - 5 = 1, matching the cofactor result.

Treat this rule (often called the rule of Sarrus) as a 3x3-only convenience. It does not extend to 4x4 or beyond: a 4x4 determinant has 24 signed terms, and the diagonal picture produces only 8 of them. For anything larger than 3x3, use cofactor expansion or the row-reduction method below.

How row operations change a determinant

• Swapping two rows multiplies the determinant by -1.
• Scaling a row by k multiplies the determinant by k.
• Adding a multiple of one row to another leaves the determinant unchanged.

Watch all three rules on A = [3 1 ; 5 2], whose determinant is 1. Swap the rows: [5 2 ; 3 1] has determinant 5*1 - 2*3 = -1. Multiplied by -1, as promised. Double the first row of A: [6 2 ; 5 2] has determinant 12 - 10 = 2. Doubled, as promised. Replace R2 with R2 - R1: [3 1 ; 2 1] has determinant 3 - 2 = 1. Unchanged, as promised. The third rule is the powerful one: the entire forward pass of Gaussian elimination is made of replacements, so elimination does not disturb the determinant at all.

Determinants by triangular reduction

For a triangular matrix (all zeros below the diagonal), every cofactor expansion collapses and the determinant is the product of the diagonal entries. Combine that with the three rules and you get the method professionals use: row-reduce to triangular form, multiply the diagonal, and account for any swaps or scalings along the way.

Worked example. Reduce A = [2 1 0 ; 1 3 1 ; 0 2 1]. Replace R2 with R2 - (1/2)*R1: 1 - 1 = 0, 3 - 0.5 = 2.5, 1 - 0 = 1, giving [0 2.5 1]. R3 already starts with 0.

Now clear column 2: the multiplier is 2/2.5 = 0.8, so replace R3 with R3 - 0.8*R2: 2 - 2 = 0 and 1 - 0.8 = 0.2, giving [0 0 0.2]. The triangular matrix has diagonal 2, 2.5, 0.2. Since we used only replacements (no swaps, no scalings), det(A) = 2 * 2.5 * 0.2 = 1. It agrees with both earlier computations, and for large matrices this route is dramatically faster than cofactors.

Key idea: replacements are free, swaps flip the sign, scalings must be paid back, and a triangular determinant is the product of its diagonal.

A real use: area from coordinates

Surveyors, game engines, and mapping software constantly need the area of a triangle whose corners are known coordinates. Determinants deliver it without any lengths or angles. For a triangle with vertices (1, 1), (4, 2), and (2, 5), build the two edge vectors leaving the first vertex: [4 - 1 ; 2 - 1] = [3 ; 1] and [2 - 1 ; 5 - 1] = [1 ; 4]. The parallelogram they span has area |det[3 1 ; 1 4]| = |3*4 - 1*1| = 11, and the triangle is half of it: 5.5 square units.

The same computation is a collinearity detector. Points (0, 0), (1, 1), (3, 3) give edge vectors [1 ; 1] and [3 ; 3], and det[1 3 ; 1 3] = 3 - 3 = 0: zero area, so the three points sit on one line. Graphics code uses exactly this test to decide whether a corner turns left or right, with the sign of the determinant telling which.

Try reversing the order of the two edge vectors in the triangle example: det[1 3 ; 4 1] = 1 - 12 = -11. Same size, opposite sign. The magnitude is still the area; the sign records which way around the triangle you listed the corners, counterclockwise or clockwise. That orientation bit is free extra information, and the next lesson explains what it means for transformations.

Common wrong turns

• Dropping the middle minus. First-row expansion reads plus, minus, plus. Writing a(ei - fh) + b(di - fg) + c(dh - eg) is the single most common determinant error.
• Using the diagonal shortcut on a 4x4. Sarrus is 3x3-only; larger matrices need cofactors or reduction.
• Scaling the whole matrix like a single row. Doubling every entry of an n x n matrix multiplies the determinant by 2^n, once per row. Check: det[2 0 ; 0 2] = 4, which is 2^2 times det(I) = 1, not 2 times.
• Adding determinants across a sum. det(A + B) is generally not det(A) + det(B): with A = B = I (2x2), the left side is det(2I) = 4 while the right side is 1 + 1 = 2.
• Forgetting the bookkeeping during reduction. Every swap flips the sign and every row scaling scales the determinant; log them as you go, or the final product silently drifts.

Try it

Compute det[1 2 0 ; 3 1 2 ; 0 4 1] by expanding along the first row, then confirm with the diagonal shortcut.

Worked answer. Expansion: first term +1 * det[1 2 ; 4 1] = 1*(1 - 8) = -7. Second term -2 * det[3 2 ; 0 1] = -2*(3 - 0) = -6. Third term +0 * (anything) = 0. Total: -7 - 6 = -13. Shortcut check: down-right diagonals give 1*1*1 + 2*2*0 + 0*3*4 = 1; down-left diagonals give 0*1*0 + 4*2*1 + 1*3*2 = 0 + 8 + 6 = 14; and 1 - 14 = -13. The two methods agree.

Recap

• 2x2: ad - bc. 3x3: cofactor expansion with checkerboard signs, along any row or column, or the Sarrus shortcut.
• Zeros are your friends: expand along the row or column that has the most of them.
• Row swaps flip the sign, row scalings scale the determinant, and replacements change nothing, so triangular reduction computes determinants fast.
• Nonzero determinant means invertible; zero means the matrix collapses space flat.

Next lesson: what the determinant means, its product rule, and why the absolute value measures area and volume.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 4.1: Determinants: Definition. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 4.2: Cofactor expansions. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 3.1: Basic techniques. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 3.3: Finding determinants using row operations. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. Weisstein, E. W. (n.d.). Determinant. In MathWorld. Wolfram Research. mathworld.wolfram.com
6. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu

The determinant is more than a computation; it is a diagnostic that ties together nearly every idea in this course. You now know how to produce the number. This lesson is about what the number is telling you: whether a matrix can be undone, how determinants behave under multiplication, and why the absolute value is literally an area or a volume. Each property below gets a numerical test drive, so nothing rests on faith.

Invertibility test

The single most useful fact: a square matrix A is invertible if and only if det(A) is not zero. A zero determinant signals a singular matrix, a system that is either inconsistent or has infinitely many solutions, and columns that are linearly dependent (a connection you will meet in Module 5). So the determinant is a one-number test for a whole cluster of properties.

Run the test on the matrices from earlier lessons. det[3 1 ; 5 2] = 6 - 5 = 1, nonzero, so [3 1 ; 5 2] is invertible and any system with these coefficients has exactly one solution, whatever the right-hand side. The 3x3 workhorse from last lesson, [2 1 0 ; 1 3 1 ; 0 2 1], had determinant 1: invertible as well.

Now det[2 4 ; 1 2] = 4 - 4 = 0, so that matrix is singular: the system 2x + 4y = b1, x + 2y = b2 either contradicts itself or repeats itself, depending on the constants. Which of the two happens depends on b, and that is worth remembering: a zero determinant does not by itself say "no solution." It says "not exactly one."

Key idea: nonzero determinant means invertible and a unique solution; zero determinant means the one-solution case is off the table.

The multiplicative property

Determinants respect multiplication: for square matrices of the same size,

det(AB) = det(A) * det(B).

Watch it happen. Take A = [1 2 ; 3 4] with det(A) = 4 - 6 = -2, and B = [2 0 ; 1 3] with det(B) = 6 - 0 = 6. The product is AB = [1*2 + 2*1, 1*0 + 2*3 ; 3*2 + 4*1, 3*0 + 4*3] = [4 6 ; 10 12], and det(AB) = 4*12 - 6*10 = 48 - 60 = -12. Sure enough, (-2) * 6 = -12. If each machine scales area by its own factor, running them in sequence scales area by the product; that is the geometric heart of the rule.

Three consequences fall out immediately. First, the inverse rule: since det(A) det(A^(-1)) = det(I) = 1, we get det(A^(-1)) = 1 / det(A); for the A above, det(A^(-1)) = -1/2. Second, powers: det(A^k) = det(A)^k, so det(A^3) = (-2)^3 = -8 with no matrix multiplication at all. Third, the transpose changes nothing: det(A^T) = det(A). Check: A^T = [1 3 ; 2 4] has determinant 4 - 6 = -2, the same as A. This last fact is why every row rule from the previous lesson is also a column rule.

Note what is not on the list: determinants do not add. In general det(A + B) is not det(A) + det(B). With A = B = I (2x2), the left side is det(2I) = 4 while the right side is 1 + 1 = 2. Multiplication is the operation determinants love; addition is not.

The product rule also lets geometry check itself. Apply the stretch B = [2 0 ; 0 3] (area factor 6) and then the shear S = [1 1 ; 0 1] (area factor 1). The combined machine is SB = [1*2 + 1*0, 1*0 + 1*3 ; 0*2 + 1*0, 0*0 + 1*3] = [2 3 ; 0 3], and det(SB) = 2*3 - 3*0 = 6. Exactly 1 * 6: the shear added slant but no area, and the product rule knew it in advance.

Key idea: det multiplies over products, inverts over inverses, survives transposes, and refuses to distribute over sums.

Reading the determinant off the pivots

The row-operation rules from last lesson have a tidy consequence: the forward pass of Gaussian elimination, which uses only replacements, leaves the determinant untouched. So after reducing A to triangular form, the determinant of A is the product of the pivots, adjusted by a factor of -1 per row swap and by any row scalings you performed. In the reduction of [2 1 0 ; 1 3 1 ; 0 2 1], the pivots came out 2, 2.5, and 0.2, and their product is 2 * 2.5 * 0.2 = 1, the determinant.

Mind the bookkeeping when swaps enter. If reaching triangular form required one row swap, negate the pivot product; two swaps cancel back to positive. This pivot route is how software computes determinants of large matrices, since cofactor expansion grows explosively while elimination stays polite. It also explains a fact you will meet again in Module 5: a matrix is invertible exactly when elimination finds a full set of nonzero pivots, because that is precisely when the pivot product, the determinant, avoids zero.

Geometric meaning: area and volume

Place the two columns of a 2x2 matrix as arrows from the origin. They span a parallelogram, and the absolute value of the determinant is that parallelogram's area. For A = [3 0 ; 0 2], the columns are along the axes with lengths 3 and 2, spanning a 3-by-2 rectangle of area 6, and indeed det(A) = 6.

A subtler example: the shear [1 1 ; 0 1] has determinant 1*1 - 1*0 = 1. It tilts the unit square into a slanted parallelogram, yet the area stays exactly 1: base 1, height 1, lean irrelevant. And for [3 1 ; 1 4], the determinant 12 - 1 = 11 matches the parallelogram area computed coordinate by coordinate in the last lesson. The determinant is the area scale factor of the machine: feed it any region of area S, and the output region has area |det| * S.

In three dimensions the absolute value of a 3x3 determinant gives the volume of the parallelepiped (a slanted box) spanned by its three columns. The box with edges [1 ; 0 ; 0], [0 ; 2 ; 0], [0 ; 0 ; 5] has volume det = 1*2*5 = 10, matching length times width times height.

A determinant of zero then means the shape has collapsed flat (zero area or volume), which is exactly why zero-determinant matrices are singular: they squash space into a lower dimension and cannot be undone. The sign of the determinant records orientation, whether the transformation flips space (negative) or preserves its handedness (positive).

Try the sign story on familiar machines. A rotation by 90 degrees has matrix [0 -1 ; 1 0] and determinant 0*0 - (-1)(1) = 1: area preserved, no flip, as a rotation should. A reflection across the x-axis, [1 0 ; 0 -1], has determinant -1: area preserved, but space is mirrored, and any text drawn in the plane would come out backwards. Graphics programmers check exactly this sign to catch accidentally mirrored textures.

Cramer's rule: solving small systems with determinants

Determinants can even solve systems directly. For a 2x2 system Ax = b with det(A) nonzero, Cramer's rule says: to find each unknown, replace that unknown's column of A with b, take the determinant, and divide by det(A).

Worked example. Solve x + 2y = 5, 3x + 4y = 11. The coefficient determinant is D = det[1 2 ; 3 4] = 4 - 6 = -2. For x, replace column 1 with the constants: Dx = det[5 2 ; 11 4] = 20 - 22 = -2, so x = Dx / D = (-2)/(-2) = 1. For y, replace column 2: Dy = det[1 5 ; 3 11] = 11 - 15 = -4, so y = (-4)/(-2) = 2. Check: 1 + 2*2 = 5 and 3*1 + 4*2 = 11. Both hold.

Cramer's rule shines for small symbolic systems and for proving formulas, because it displays the answer as a ratio of determinants. For large numerical systems it is hopelessly slow compared with elimination, so treat it as a precision tool, not a workhorse.

A real use: how transformations scale a design

Imagine a design app scaling a logo with the matrix [2 0 ; 0 3]: twice as wide, three times as tall. Every region of the logo, however curvy, has its area multiplied by det = 6, so the app can update the reported square-footage of a floor plan, or the ink coverage of a print job, without remeasuring anything. A rotation (det = 1) changes no areas; a mirror (det = -1) changes no areas but flips orientation. In calculus, the same idea reappears as the Jacobian determinant, the local area scale factor used whenever you change variables in a double integral.

Common wrong turns

• Adding determinants. det(A + B) = det(A) + det(B) fails even for identity matrices; only products factor.
• Forgetting the row count in det(kA). Scaling an n x n matrix by k scales the determinant by k^n. For 3x3, det(2A) = 8 det(A), not 2 det(A).
• Reading det = 0 as "no solution." It means "not exactly one": the system is inconsistent for some right-hand sides and has infinitely many solutions for others.
• Reporting a negative area. Area is |det|; the sign is orientation information, not a negative quantity of space.
• Asking for the determinant of a nonsquare matrix. A 2 x 3 matrix has no determinant; the concept is defined only for square arrays.

Try it

Suppose A and B are 3x3 with det(A) = 3 and det(B) = -5. Find det(A^2 B), det(2A), and det(B^T).

Worked answer. Products factor: det(A^2 B) = det(A)^2 * det(B) = 9 * (-5) = -45. Scaling a 3x3 by 2 multiplies the determinant by 2^3 = 8: det(2A) = 8 * 3 = 24. Transposing changes nothing: det(B^T) = -5.

Recap

• Invertible exactly when the determinant is nonzero; zero rules out the unique-solution case.
• det(AB) = det(A)det(B), det(A^(-1)) = 1/det(A), det(A^T) = det(A); sums do not cooperate.
• |det| is the area or volume scale factor of the transformation; the sign records whether space is flipped.
• Cramer's rule solves small systems as ratios of determinants.

These properties recur constantly. When you later ask whether a transformation is reversible, whether a basis is valid, or whether eigenvalues make a matrix singular, the determinant is the number you reach for.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 4.3: Determinants and volumes. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Kuttler, K. (n.d.). 3.2: Properties of determinants. In A first course in linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 3.4: Applications of the determinant. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Weisstein, E. W. (n.d.). Cramer's rule. In MathWorld. Wolfram Research. mathworld.wolfram.com
5. O'Connor, J. J., & Robertson, E. F. (n.d.). Gabriel Cramer (1704-1752). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
6. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net
7. Axler, S. (n.d.). Linear algebra done right. linear.axler.net

A vector in R^n (read: R n, the space of lists of n real numbers) is an ordered list of n real numbers, written as a column such as v = [2 ; 3] or w = [1 ; -1 ; 4]. There are two equally good mental pictures, and it pays to hold both. Algebraically, a vector is its coordinates, nothing more. Geometrically, it is an arrow from the origin to the point with those coordinates: [2 ; 3] points 2 right and 3 up.

The list picture is why vectors run modern computing. A weather station's noon report (temperature, humidity, wind speed) is a vector in R^3; your ratings of five movies form a vector in R^5; a grayscale photo is a vector with one entry per pixel. Any object described by several numbers at once is a vector, and linear algebra is the arithmetic of such objects. Vectors are the raw material of the whole subject.

Addition, scaling, and linear combinations

Vectors of the same length add entry by entry, and a scalar multiplies every entry, exactly like one-column matrices. Concretely: [2 ; 3] + [1 ; -1] = [2+1 ; 3+(-1)] = [3 ; 2], and 2[1 ; 2] = [2 ; 4], while -1[1 ; 2] = [-1 ; -2], the same arrow reversed. Geometrically, addition follows the tip-to-tail rule: slide the second arrow so it starts where the first ends, and the sum runs from the original start to the final tip. Scaling stretches or shrinks an arrow along its own line.

Combining the two operations gives the central construction of linear algebra. A linear combination of vectors v1, ..., vk is any expression c1 v1 + c2 v2 + ... + ck vk with scalar weights ci. For example, 2[1 ; 0] + 3[0 ; 1] = [2 ; 3]: two steps east plus three steps north.

Another, with v = [1 ; 2] and w = [3 ; -1]: the combination 2v - w is [2 ; 4] - [3 ; -1] = [2 - 3 ; 4 + 1] = [-1 ; 5]. Matrix-vector products, spans, bases, everything ahead is built from weighted sums like these.

Key idea: add entry by entry, scale every entry, and call any weighted sum a linear combination.

The dot product, length, and angle

The dot product of two equal-length vectors multiplies matching entries and adds: v . w = v1 w1 + v2 w2 + ... + vn wn (read: v dot w), producing a single number. For v = [3 ; 4] and w = [2 ; 1], v . w = 3*2 + 4*1 = 6 + 4 = 10. The dot product measures both size and alignment:

• The length (or norm) of v is ||v|| = sqrt(v . v). For v = [3 ; 4], ||v|| = sqrt(9 + 16) = sqrt(25) = 5.
• The angle between vectors satisfies v . w = ||v|| ||w|| cos(theta).
• Two vectors are orthogonal (perpendicular) exactly when v . w = 0.

The length formula is the Pythagorean theorem wearing new clothes: the arrow [3 ; 4] is the hypotenuse of a right triangle with legs 3 and 4, so its length must be 5. Dividing a vector by its own length produces a unit vector, length exactly 1, pointing the same way: [3 ; 4] / 5 = [0.6 ; 0.8], and indeed sqrt(0.36 + 0.64) = sqrt(1) = 1. Unit vectors answer "which direction?" with no opinion about "how far?"

Length also measures distance between points: the distance from (1, 2) to (4, 6) is the length of the difference vector [4 - 1 ; 6 - 2] = [3 ; 4], which is 5. And the angle formula turns arithmetic into geometry. Take v = [1 ; 0] and w = [1 ; 1]: the dot product is 1, the lengths are 1 and sqrt(2), so cos(theta) = 1 / sqrt(2) = 0.707 approximately, giving theta = 45 degrees, exactly what a sketch shows for the diagonal of a square.

The sign of the dot product is a quick alignment gauge even without computing angles: positive means the vectors point broadly the same way (angle under 90 degrees), negative means broadly opposite (over 90 degrees), and zero means perpendicular. Test perpendicularity on [2 ; 1] and [-1 ; 2]: the dot product is 2*(-1) + 1*2 = -2 + 2 = 0, so they are orthogonal. For instance [1 ; 0] and [0 ; 1] have dot product 0, confirming the axes are perpendicular.

Two famous inequalities keep these quantities honest. The Cauchy-Schwarz inequality says |v . w| can never exceed ||v|| ||w||, which is why the cosine formula always produces a value between -1 and 1. And the triangle inequality says ||v + w|| <= ||v|| + ||w||: a detour is never shorter than the straight route. Check the second with v = [3 ; 0], w = [0 ; 4]: the sum [3 ; 4] has length 5, comfortably below 3 + 4 = 7.

Key idea: the dot product is one number scoring alignment; with itself it gives squared length, and zero means perpendicular.

Span

The span of a set of vectors is the collection of all their linear combinations, that is, every vector you can build from them by scaling and adding. The span of a single nonzero vector in R^2 is the line through the origin along that vector: all multiples t[1 ; 2], one point for each real t. The span of two vectors that point in different directions is the entire plane R^2.

Asking whether a particular vector lies in a given span is the same as asking whether a certain linear system is consistent, which is why Gaussian elimination from Module 1 quietly underlies this geometric language.

Worked example. Is [5 ; 1] in the span of v = [1 ; 2] and w = [1 ; -1]? We need weights with c1 v + c2 w = [5 ; 1]. Reading the two coordinates separately gives a system: top entries say c1 + c2 = 5, bottom entries say 2 c1 - c2 = 1. Add the equations: 3 c1 = 6, so c1 = 2, and then c2 = 3.

Confirm the recipe: 2[1 ; 2] + 3[1 ; -1] = [2 + 3 ; 4 - 3] = [5 ; 1]. Yes: the target is reachable, with weights 2 and 3. Because those two spanning vectors point in genuinely different directions, every vector of R^2 is reachable the same way; their span is the whole plane.

Contrast that with [1 ; 2] and [2 ; 4]. The second is twice the first, so any combination c1[1 ; 2] + c2[2 ; 4] equals (c1 + 2 c2)[1 ; 2], a multiple of one direction. The span is only the line through [1 ; 2], described parametrically as all points (t, 2t).

A vector like [5 ; 1] is off that line, since a point on the line must have second coordinate double the first, and 1 is not 2*5. Two vectors do not automatically span a plane; they must contribute different directions. Span is how we describe, precisely, the reach of a set of vectors.

Key idea: span is everything buildable from the given vectors, and membership questions are systems of equations in the weights.

A real use: similarity scores from dot products

Streaming services ask a linear-algebra question every night: which users have similar taste? Store each user's ratings of three films as a vector. Say u = [5 ; 1 ; 4], v = [4 ; 1 ; 5], and w = [1 ; 5 ; 1]. Compare with dot products: u . v = 5*4 + 1*1 + 4*5 = 20 + 1 + 20 = 41, while u . w = 5*1 + 1*5 + 4*1 = 5 + 5 + 4 = 14.

The higher score says u and v lean the same way: they liked and disliked the same films, so recommending v's favorites to u is a sensible bet.

In practice engines refine this by dividing by the lengths, which turns the score into the cosine of the angle between taste vectors, so that one user's habit of rating everything high does not fake agreement. Physics uses the same additions the other way around: two forces acting on one crate combine by vector addition, tip to tail, and the dot product of force with displacement computes work done.

Common wrong turns

• Expecting the dot product to be a vector. It is a single number. If your "dot product" has brackets, entries were multiplied without the final adding.
• Adding lengths across a sum. ||v + w|| is usually less than ||v|| + ||w||: with v = [3 ; 0] and w = [0 ; 4], the sum [3 ; 4] has length 5, not 7. Walking a bent path is longer than the straight line.
• Forgetting the square root. v . v is the squared length. The length of [3 ; 4] is 5, not 25.
• Thinking span needs positive weights. Weights may be any real numbers, negative and zero included; the span of [1 ; 0] contains [-7 ; 0].
• Forgetting the origin. Every span contains the zero vector (take all weights 0), so a description of a span that misses the origin is wrong.

Try it

Let u = [2 ; -1 ; 3] and v = [1 ; 4 ; 2]. Compute u . v and ||u||, and decide whether the vectors are orthogonal.

Worked answer. Dot product: u . v = 2*1 + (-1)*4 + 3*2 = 2 - 4 + 6 = 4. Length: ||u|| = sqrt(2^2 + (-1)^2 + 3^2) = sqrt(4 + 1 + 9) = sqrt(14), about 3.74. Since u . v = 4 is not zero, the vectors are not orthogonal; the positive sign says the angle between them is less than 90 degrees.

Recap

• A vector is a list of numbers and an arrow from the origin; both pictures are correct and useful.
• Addition is entrywise and tip-to-tail; scaling stretches; linear combinations are weighted sums.
• v . w scores alignment; ||v|| = sqrt(v . v); zero dot product means orthogonal.
• The span of a set is everything its combinations can reach, and membership is a solvable system.

Next we ask what makes a collection of vectors, with these two operations, into a self-contained world: the vector space.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 2.1: Vectors. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 2.2: Vector equations and spans. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 4.4: Length of a vector. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 4.7: The dot product. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. Boyd, S., & Vandenberghe, L. (n.d.). Introduction to applied linear algebra: Vectors, matrices, and least squares. Stanford University. web.stanford.edu
6. Strang, G. (n.d.). 18.06SC Linear algebra [Course materials with lecture videos, Fall 2011]. MIT OpenCourseWare. ocw.mit.edu

So far vectors have been columns of numbers. The power of linear algebra comes from noticing that many other objects, such as polynomials, functions, and matrices, obey the very same arithmetic rules: you can add them, you can scale them, and the familiar algebra of those operations goes through unchanged. Rather than proving every theorem separately for columns, for polynomials, and for matrices, mathematics captures the shared pattern once, with the abstract idea of a vector space. Anything that satisfies the definition inherits every theorem for free.

What a vector space is

A vector space is a set V of objects called vectors, together with an addition and a scalar multiplication, that obey the familiar algebraic rules. Concretely, addition must be commutative and associative, there must be a zero vector that adds to leave things unchanged, every vector must have a negative, and scalar multiplication must distribute over addition in both ways, among a total of eight axioms. The point is not to memorize all eight but to recognize the pattern: whenever a set lets you add and scale in the ordinary way, calculus done there behaves like R^n. The space R^n itself is the model example.

For reference, the eight axioms sort into two families:

• Addition rules: u + v = v + u; grouping does not matter, (u + v) + w = u + (v + w); a zero vector exists with v + 0 = v; and every v has a negative with v + (-v) = 0.
• Scaling rules: c(u + v) = cu + cv; also (c + d)v = cv + dv; nested scalings multiply, c(dv) = (cd)v; and the scalar 1 changes nothing, 1v = v.

It helps to see the club's other members. The set P2 of polynomials of degree at most 2 is a vector space: adding 1 + 2x and 3 - x + x^2 gives 4 + x + x^2, still degree at most 2; scaling 1 + 2x by 3 gives 3 + 6x; the zero polynomial plays the role of the zero vector. In fact a + bx + cx^2 behaves exactly like the column [a ; b ; c].

The set of all 2x2 matrices qualifies too, with the entrywise addition and scaling of Module 2. Even audio signals form a vector space: mixing two tracks is vector addition, and a volume knob is scalar multiplication.

Non-members are equally instructive. The positive numbers with ordinary operations fail: scaling 5 by the scalar -1 gives -5, which has left the set. The first quadrant of R^2 (both coordinates nonnegative) is closed under addition, but scaling [1 ; 1] by -1 gives [-1 ; -1], outside the quadrant. A vector space must tolerate every scalar, negatives included.

Key idea: a vector space is any world where addition and scaling behave like they do in R^n; columns, polynomials, matrices, and signals all qualify.

Subspaces

Usually we care about a piece of a vector space that is itself a vector space. A subspace of V is a nonempty subset that is closed under the vector-space operations. To check that a subset W is a subspace, you only need three things:

1. The zero vector is in W.
2. W is closed under addition: if u and v are in W, so is u + v.
3. W is closed under scalar multiplication: if v is in W and c is any scalar, then cv is in W.

If all three hold, W is automatically a vector space in its own right; if any fails, it is not a subspace. Why do three tests replace eight axioms? Because the arithmetic laws (commutativity, associativity, distribution) are inherited from the big space: they hold for all vectors of V, so in particular for the ones in W. The only genuine questions are whether the operations ever escape the subset, and whether the zero vector is present.

Note that condition 1 is really a convenience: taking c = 0 in condition 3 already forces 0 into W. It is listed first because it is the fastest way to disqualify a candidate.

Worked example

Is the line W = { [t ; 2t] : t any real } a subspace of R^2?

• Zero: setting t = 0 gives [0 ; 0], so the zero vector is in W. Good.
• Addition: [a ; 2a] + [b ; 2b] = [a+b ; 2(a+b)], again of the form [t ; 2t]. Closed.
• Scaling: c[a ; 2a] = [ca ; 2(ca)], still of the same form. Closed.

All three pass, so this line through the origin is a subspace. By contrast, the line { [t ; 2t + 1] } is not a subspace, because it misses the origin: no value of t gives [0 ; 0].

Two more candidates sharpen the eye. The set { [x ; y] : x + y = 0 } passes: the zero vector satisfies 0 + 0 = 0; if u = [x1 ; y1] and v = [x2 ; y2] each have coordinates summing to zero, the sum has (x1 + x2) + (y1 + y2) = 0 + 0 = 0; and scaling gives cx + cy = c(x + y) = 0. It is the line through the origin with slope -1.

The unit circle x^2 + y^2 = 1 fails immediately: it does not contain [0 ; 0], and it is not closed anyway, since [1 ; 0] + [0 ; 1] = [1 ; 1] has length sqrt(2), off the circle.

One subtle failure: the union of the two coordinate axes. It contains the origin, and any scalar multiple of an axis vector stays on its axis. But addition escapes: [1 ; 0] + [0 ; 1] = [1 ; 1] lies on neither axis. Closure under addition and closure under scaling are separate demands, and a subspace needs both.

Key idea: to certify a subspace, check zero, sums, and scalar multiples for arbitrary members; to refute one, exhibit a single escape.

The catalog for R^2 and R^3, and two builders

The subspaces of R^2 are exactly the zero vector alone, every line through the origin, and all of R^2; in R^3 they are the origin, lines through the origin, planes through the origin, and all of R^3. Passing through the origin is the non-negotiable requirement: any subspace must contain the zero vector. Nothing in between occurs: no curves, no half-planes, no shifted lines. Flat, through the origin, closed under the operations: that is the whole inventory.

Two constructions manufacture subspaces automatically. First, the span of any set of vectors is always a subspace: sums of combinations are combinations, and scalar multiples of combinations are combinations, so closure is built in. Second, the solution set of a homogeneous system, { x : Ax = 0 }, is always a subspace, called the null space of A.

The check uses linearity of matrix multiplication: if Au = 0 and Av = 0, then A(u + v) = Au + Av = 0 + 0 = 0, and A(cu) = c(Au) = 0. These two builders, spans and null spaces, produce nearly every subspace you will meet in this course.

The same tests work far from R^n. Inside the space of 2x2 matrices, consider the symmetric ones, those with A^T = A. The zero matrix is symmetric. If A and B are symmetric, then (A + B)^T = A^T + B^T = A + B, so the sum is symmetric, and (cA)^T = c A^T = cA handles scaling. Three checks, and the symmetric matrices stand revealed as a subspace, no arrows in sight.

A real use: why superposition works

Engineers lean on subspaces daily under the name superposition. The signals that a linear circuit, or a vibrating string, can produce with no external input form the solution set of a homogeneous linear equation, so they form a subspace: add two valid vibrations and the result is a valid vibration; double one and it remains valid.

That is why a guitar string can carry its fundamental tone and several overtones at once, and why noise-cancelling headphones can add an inverted copy of ambient sound to silence it. When an engineer says "solutions superpose," a mathematician hears "the solution set is closed under addition and scaling," which is precisely the subspace property.

The contrast case completes the picture: solutions of a nonhomogeneous equation Ax = b with b nonzero do not form a subspace. Adding two of them gives A(u + v) = b + b = 2b, not b, and the zero vector is not a solution at all. Geometrically that solution set is a flat shifted off the origin, the same shape as the line y = 2x + 1 that failed our test earlier.

Common wrong turns

• Checking only the zero vector. Containing the origin is necessary, not sufficient. The unit disk contains [0 ; 0], yet scaling [1 ; 0] by 10 escapes it.
• Calling every line a subspace. Only lines through the origin qualify; y = 2x + 1 fails the zero test before anything else.
• Testing with one convenient example. Closure must hold for all members and all scalars. The two-point set containing only [0 ; 0] and [1 ; 1] contains zero, yet [1 ; 1] + [1 ; 1] = [2 ; 2] is outside.
• Restricting scalars to positives. Subspace scaling includes negative and zero scalars; forgetting them is how the first quadrant sneaks past a careless check.
• Assuming vectors must be arrows. Polynomials, matrices, and signals form vector spaces too; the subspace test reads the same in every one of them.

Try it

Is W = { [x ; y] : y = x^2 }, the parabola through the origin, a subspace of R^2?

Worked answer. The zero vector passes, since 0 = 0^2. Now test addition: [1 ; 1] and [2 ; 4] are both on the parabola, but their sum [3 ; 5] is not, because 3^2 = 9 and 5 is not 9. One escape settles it: W is not a subspace. Scaling fails too: 2[1 ; 1] = [2 ; 2], and 2^2 = 4, not 2. Curves bend; subspaces are flat.

Recap

• A vector space is any set with well-behaved addition and scaling; R^n, polynomials, matrices, and signals all qualify.
• A subspace is a subset passing three tests: contains zero, closed under addition, closed under scaling.
• Subspaces of R^2 and R^3 are flats through the origin; spans and null spaces are subspaces automatically.
• To refute a candidate, one concrete escape is enough.

Subspaces can be big or small. The next lesson builds the measuring tools: independence, basis, and dimension.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 2.6: Subspaces. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Kuttler, K. (n.d.). 9.1: Algebraic considerations. In A first course in linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 9.4: Subspaces and basis. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Axler, S. (n.d.). Linear algebra done right. linear.axler.net
5. O'Connor, J. J., & Robertson, E. F. (n.d.). Giuseppe Peano (1858-1932). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
6. O'Connor, J. J., & Robertson, E. F. (n.d.). Hermann Grassmann (1809-1877). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
7. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

Some vectors in a set carry new information; others are redundant, expressible from the rest. Imagine giving directions with three compass words, east, north, and northeast: the third adds nothing, because northeast is east plus north. Linear independence makes this notion of redundancy precise, and it leads to the single most important idea in the theory of vector spaces: the basis, a set of directions with no waste and no gaps.

Linear independence

Vectors v1, ..., vk are linearly independent if the only way to write the zero vector as a linear combination c1 v1 + ... + ck vk = 0 is with every weight zero (c1 = ... = ck = 0). If some nonzero weights also produce zero, the vectors are linearly dependent, meaning at least one is a combination of the others and adds nothing new.

Why is a zero combination the right test? Because if, say, 2 v1 - v2 = 0, then v2 = 2 v1: a hidden zero combination is exactly a hidden recipe for one vector in terms of the rest.

Worked example. Are v1 = [1 ; 2] and v2 = [2 ; 4] independent? Notice v2 = 2 v1, so 2 v1 - v2 = 0 is a zero combination with nonzero weights. They are dependent. By contrast [1 ; 0] and [0 ; 1] are independent: c1[1 ; 0] + c2[0 ; 1] = [c1 ; c2] = [0 ; 0] forces c1 = c2 = 0.

A reliable test for any number of vectors: form a matrix with the vectors as columns and row-reduce. The vectors are independent exactly when every column has a pivot (no free variables). For square matrices, this is the same as a nonzero determinant: det[1 2 ; 1 3] = 3 - 2 = 1, so [1 ; 1] and [2 ; 3] are independent.

A full 3-vector run. Are v1 = [1 ; 0 ; 1], v2 = [0 ; 1 ; 1], v3 = [1 ; 1 ; 2] independent? Load them as columns: A = [1 0 1 ; 0 1 1 ; 1 1 2], and solve Ac = 0. Replace R3 with R3 - R1: [0 1 1]. Replace R3 with R3 - R2: [0 0 0]. The echelon form [1 0 1 ; 0 1 1 ; 0 0 0] has pivots in columns 1 and 2 only, so column 3 is free and the vectors are dependent.

Better yet, elimination hands us the recipe. With c3 = 1 free, row 2 says c2 + c3 = 0, so c2 = -1; row 1 says c1 + c3 = 0, so c1 = -1. Then -v1 - v2 + v3 = 0, in other words v3 = v1 + v2. Check: [1 ; 0 ; 1] + [0 ; 1 ; 1] = [1 ; 1 ; 2]. Confirmed: the third vector was the first two in disguise, and geometrically all three lie in one plane through the origin.

Three quick facts save time. Any set containing the zero vector is dependent, since 1 * 0 = 0 is already a nonzero-weight combination. Two vectors are dependent exactly when one is a scalar multiple of the other. And more than n vectors in R^n are always dependent: the matrix has more columns than rows, so pivots (at most one per row) cannot cover every column, and a free variable is guaranteed.

The geometry is worth carrying with you. One nonzero vector is always independent: a single direction. Two vectors are dependent exactly when they are collinear, squeezed onto one line. Three vectors in R^3 are dependent exactly when they are coplanar, as our worked trio was. Dependence always means the set lives in a smaller flat than its headcount suggests, and dimension, next lesson, will make "smaller flat" a number.

Key idea: independence means the only zero combination is the trivial one; pivots in every column certify it, and a free column exposes the dependency recipe.

Basis

A basis of a vector space (or subspace) is a set of vectors that is both

• linearly independent (no redundancy), and
• spanning (their linear combinations produce the whole space).

A basis is thus a minimal, complete set of building blocks: enough to reach everything, with nothing wasted. Every vector in the space can be written as a linear combination of basis vectors in exactly one way, and those unique weights are the vector's coordinates in that basis.

Why exactly one way? Suppose a vector had two recipes in the same basis. Subtracting them gives a zero combination of the basis vectors whose weights are the differences of the two recipes. Independence forces every difference to be zero, so the recipes were identical after all. Spanning guarantees at least one recipe; independence guarantees at most one. A basis delivers both, which is what makes coordinates trustworthy.

Worked example. The vectors [1 ; 2] and [1 ; -1] form a basis of R^2: they are independent (neither is a multiple of the other; determinant 1*(-1) - 1*2 = -3, nonzero) and they span the plane. In Lesson 7 of this course's vector story we found [5 ; 1] = 2[1 ; 2] + 3[1 ; -1], so the coordinates of [5 ; 1] in this basis are (2, 3). Same arrow, new address system.

The standard basis

The most familiar basis of R^n is the standard basis of unit vectors along the axes. In R^2 it is e1 = [1 ; 0] and e2 = [0 ; 1]; in R^3 it adds e3 = [0 ; 0 ; 1]. Any vector [a ; b] is a*e1 + b*e2, so its coordinates in the standard basis are exactly its ordinary entries: read [7 ; -3] and you have read its address, 7 steps of e1 and -3 steps of e2. That is what makes this basis "standard": the address system and the raw data coincide.

But other bases exist too, and choosing one is like rotating your graph paper. Take [1 ; 1] and [1 ; -1], a basis since det[1 1 ; 1 -1] = -1 - 1 = -2 is nonzero.

To find the coordinates of [4 ; 2] here, solve c1[1 ; 1] + c2[1 ; -1] = [4 ; 2]: the top entries give c1 + c2 = 4, the bottom give c1 - c2 = 2. Adding, 2 c1 = 6, so c1 = 3 and c2 = 1. Check: 3[1 ; 1] + 1[1 ; -1] = [3 + 1 ; 3 - 1] = [4 ; 2]. The coordinates are (3, 1). Choosing a clever basis is often the key to simplifying a problem, a theme that returns powerfully with eigenvectors in Module 7.

A labor-saving fact for R^n: if you have exactly n vectors there, you do not need to check independence and spanning separately. Load them as the columns of an n x n matrix; a nonzero determinant certifies both at once, and a zero determinant sinks both at once. The two properties are a package deal for square counts, which is why the determinant keeps appearing as a basis detector.

Key idea: a basis is an address system for a space; independence makes addresses unique, spanning makes every point reachable.

A real use: color is coordinates in a basis

Every color your screen shows is a linear combination of three basis colors: color = r*red + g*green + b*blue. The triple (r, g, b) is nothing but a coordinate vector in the RGB basis; pure yellow, for instance, has coordinates (255, 255, 0).

Printers use a different basis for the same color space, built on cyan, magenta, and yellow inks, so sending a photo to print is a change of basis: the same colors, re-expressed in new building blocks. Independence matters physically: if one ink were mixable from the other two, it would be dead weight in the cartridge, a dependent vector adding cost and no new colors. The same accounting appears in audio (a mix bus built from independent stems) and in chemistry, where a reaction basis spans every achievable mixture; whenever engineers say "channels" or "degrees of freedom," they are counting basis vectors.

Common wrong turns

• Confusing independent with perpendicular. [1 ; 0] and [1 ; 1] are independent (neither is a multiple of the other) yet not orthogonal, since their dot product is 1. Orthogonal nonzero vectors are always independent, but not the other way around.
• Testing dependence only in pairs. [1 ; 0], [0 ; 1], [1 ; 1] has no parallel pair, yet the trio is dependent because the third is the sum of the first two. Dependence is a property of the whole set.
• Letting the zero vector in. Any set containing 0 is automatically dependent, however innocent the other members.
• Expecting "the" basis. A space has many bases; all of them share only their size. The standard basis is convenient, not mandatory.
• Forgetting to check both requirements. Three vectors in R^2 may span but cannot be independent; one vector in R^2 may be independent but cannot span. A basis must do both jobs at once.

Try it

Decide whether v1 = [1 ; 2 ; 0], v2 = [2 ; 1 ; 3], v3 = [3 ; 3 ; 3] are linearly independent; if not, find the dependency.

Worked answer. Columns into a matrix: [1 2 3 ; 2 1 3 ; 0 3 3]. Replace R2 with R2 - 2*R1: [0 -3 -3]. Replace R3 with R3 + R2 (the new R2): 3 + (-3) = 0 and 3 + (-3) = 0, giving [0 0 0]. Pivots sit in columns 1 and 2 only, so the set is dependent.

Now extract the recipe. Set c3 = 1: row 2 says -3 c2 - 3 = 0, so c2 = -1; row 1 says c1 + 2(-1) + 3 = 0, so c1 = -1. The relation is v3 = v1 + v2. Check: [1 ; 2 ; 0] + [2 ; 1 ; 3] = [3 ; 3 ; 3]. Confirmed.

Recap

• Independent means no vector is buildable from the rest; the only zero combination is all-zero weights.
• Row-reduce the columns: pivots everywhere means independent; a free column yields the dependency recipe.
• A basis is independent and spanning, giving every vector exactly one coordinate address.
• The standard basis reads coordinates directly; other bases trade familiarity for problem-solving power.

Bases raise a counting question: do all bases of a space have the same size? Yes, and that number, the dimension, is next.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 2.5: Linear independence. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 2.7: Basis and dimension. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Margalit, D., & Rabinoff, J. (n.d.). 2.8: Bases as coordinate systems. In Interactive linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 4.10: Spanning, linear independence and basis in R^n. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net
6. Beezer, R. (n.d.). A first course in linear algebra. American Institute of Mathematics Open Textbook Initiative. textbooks.aimath.org

How big is a vector space? Not how many vectors it holds (usually infinitely many), but how many independent directions it has. That count is its dimension, and for matrices the closely related number is the rank. Between them they answer the practical questions of this module: how much room a subspace occupies, and how much genuine information a matrix carries.

Dimension

A remarkable theorem guarantees that every basis of a given vector space has the same number of vectors. Whatever building blocks you choose, the count comes out identical, so the count is a property of the space itself.

That common number is the dimension of the space. So R^2 has dimension 2 (its bases have two vectors), R^3 has dimension 3, and R^n has dimension n. A line through the origin has dimension 1; a plane through the origin has dimension 2; the subspace containing only the zero vector has dimension 0, since its basis is the empty set. Dimension makes the intuitive notion of "how many degrees of freedom" exact.

Worked example. What is the dimension of the plane W = { [x ; y ; z] : x + y + z = 0 } inside R^3? Solve the single equation: x = -y - z, with y = s and z = t free. Every member is [-s - t ; s ; t] = s[-1 ; 1 ; 0] + t[-1 ; 0 ; 1]. The two vectors [-1 ; 1 ; 0] and [-1 ; 0 ; 1] span W, and neither is a multiple of the other, so they are independent: a basis of size 2.

The plane has dimension 2, matching the geometry: one linear condition spent one of the three degrees of freedom.

Dimension also works beyond columns. The polynomial space P2 of degree at most 2 has basis 1, x, x^2, hence dimension 3, which is why quadratics behave like R^3 in disguise.

Dimension is also a bookkeeping ally. In an n-dimensional space, any set of n independent vectors is automatically a basis, any spanning set has at least n members, and any independent set has at most n. Knowing the dimension in advance tells you when to stop hunting: two independent vectors in our plane W, and the search is over.

Key idea: dimension is the size of any basis, and each independent linear condition imposed on a space removes one dimension.

Rank

The rank of a matrix is the number of pivots in its row echelon form, equivalently the number of linearly independent rows, which always equals the number of linearly independent columns. It measures how much genuinely independent information the matrix carries.

Worked example. Find the rank of

A = [1 2 1 ; 2 4 3 ; 1 2 2].

1. Clear column 1. Replace R2 with R2 - 2*R1: 2 - 2 = 0, 4 - 4 = 0, 3 - 2 = 1, giving [0 0 1]. Replace R3 with R3 - R1: 1 - 1 = 0, 2 - 2 = 0, 2 - 1 = 1, giving [0 0 1].
2. The two new rows are identical, so replace R3 with R3 - R2: [0 0 0]. The echelon form is [1 2 1 ; 0 0 1 ; 0 0 0].
3. There are pivots in columns 1 and 3, so rank(A) = 2.

The rank is 2, not 3, because the columns are dependent: the second column [2 ; 4 ; 2] is twice the first [1 ; 2 ; 1]. A 3x3 matrix of rank less than 3 is singular, with determinant zero, tying rank back to invertibility.

Rank has two subspace readings. The column space (the span of the columns, the range of the machine) has dimension equal to the rank, and a basis for it is the set of pivot columns of the original matrix: here, columns 1 and 3, namely [1 ; 2 ; 1] and [1 ; 3 ; 2]. The row space (the span of the rows) also has dimension equal to the rank, which is the quiet miracle of the subject: independent-row count and independent-column count agree for every matrix, however lopsided its shape.

The rank-nullity theorem

The columns without pivots correspond to free variables, and the space of solutions to Ax = 0 (the null space) has dimension equal to the number of free columns, called the nullity. These two counts must add up to the total number of columns:

rank(A) + nullity(A) = number of columns of A.

Watch the theorem verify itself on the example. Solve Ax = 0 from the echelon form [1 2 1 ; 0 0 1 ; 0 0 0]. Row 2 says x3 = 0. Column 2 is pivot-free, so x2 = t is free. Row 1 says x1 + 2t + 0 = 0, so x1 = -2t. The null space is the line { t[-2 ; 1 ; 0] }, with basis [-2 ; 1 ; 0] and dimension 1.

Sanity check the basis vector: A[-2 ; 1 ; 0] has first entry -2 + 2 + 0 = 0, second -4 + 4 + 0 = 0, third -2 + 2 + 0 = 0. Indeed zero.

So rank = 2 and nullity = 1, and 2 + 1 = 3 columns. This rank-nullity theorem is a precise conservation law: every column is either a pivot column (contributing to rank) or a free column (contributing to nullity), never both and never neither. It links the size of a matrix's range to the size of the space it collapses to zero, and it will reappear as we study linear transformations next.

Key idea: columns split into pivot columns and free columns; rank counts the first, nullity the second, and the split is exhaustive.

Rank decides solvability

Rank also answers "does Ax = b have a solution?" with one comparison: the system is consistent exactly when rank(A) = rank([A | b]), where [A | b] is the augmented matrix. Appending b can only keep the rank or raise it by one, and a raise means b brought a direction the columns of A cannot build.

Worked example. Let A = [1 2 ; 2 4], which reduces to [1 2 ; 0 0]: rank 1. Try b = [3 ; 6]: the augmented matrix [1 2 | 3 ; 2 4 | 6] reduces by R2 - 2*R1 to [1 2 | 3 ; 0 0 | 0], still rank 1, so the system is consistent (infinitely many solutions, one free variable).

Now try b = [3 ; 7]: reduction gives [1 2 | 3 ; 0 0 | 1], rank 2 against rank 1, and the bottom row reads 0 = 1. Inconsistent. Same coefficients, different verdicts, and the rank comparison saw it coming.

A real use: rank is information content

A grayscale image is a matrix of brightness values, say 1000 x 1000, a million numbers. If every row happened to be a multiple of one pattern row, the matrix would have rank 1, and you could store it as a single column times a single row: about 2000 numbers instead of a million. Real photographs are not rank 1, but their action concentrates in surprisingly few independent directions, so storing a rank-50 approximation of a rank-1000 image can preserve most of the picture at a fraction of the size.

This low-rank compression idea, made rigorous by the singular value decomposition in later courses, also powers recommendation engines, which model a huge user-by-product ratings matrix as approximately low rank because tastes reduce to a few underlying factors.

Engineers read rank diagnostically too. If six sensors produce measurement vectors forming a matrix of rank 4, then two sensors are echoing combinations of the others, and the system truly measures only four independent quantities. The extremes anchor the scale: the zero matrix has rank 0, carrying no information at all, while an invertible n x n matrix has full rank n, every column pulling its own weight.

Common wrong turns

• Counting nonzero entries or nonzero rows of the original matrix. Rank counts pivots after reduction. The matrix [1 3 ; 2 6] has four nonzero entries and two nonzero rows, yet rank 1.
• Taking column-space basis vectors from the echelon form. Row operations change the column space. Identify which columns hold pivots, then take those columns from the original matrix.
• Expecting rank to exceed the smaller side. Rank is at most the minimum of the row count and column count; a 2 x 5 matrix cannot have rank 3.
• Computing nullity from zero rows. Nullity counts free columns. A 3x3 of rank 2 has one free column and nullity 1, whether or not you notice its zero row.
• Confusing the dimension of a subspace with its home. The plane x + y + z = 0 lives inside R^3 but has dimension 2; where a space sits and how big it is are different questions.

Try it

Find the rank and nullity of A = [1 1 2 ; 1 2 3 ; 2 3 5], and give a basis for its null space.

Worked answer. Replace R2 with R2 - R1: [0 1 1]. Replace R3 with R3 - 2*R1: [0 1 1]. Replace R3 with R3 - R2: [0 0 0]. Echelon form: [1 1 2 ; 0 1 1 ; 0 0 0], pivots in columns 1 and 2, so rank = 2 and nullity = 3 - 2 = 1.

For the null space, set x3 = t: row 2 gives x2 = -t, and row 1 gives x1 = -x2 - 2 x3 = t - 2t = -t. Basis: [-1 ; -1 ; 1]. Check: each row of A dotted with it gives -1 - 1 + 2 = 0, -1 - 2 + 3 = 0, -2 - 3 + 5 = 0. All zero, as required.

Recap

• Dimension is the size of any basis; every independent constraint costs one dimension.
• Rank is the pivot count, equal to the dimensions of both the column space and the row space.
• Rank + nullity = number of columns: pivot columns power the range, free columns power the null space.
• Consistency of Ax = b is the comparison rank(A) = rank([A | b]), and low rank means compressible information.

Next, matrices stop being tables and become verbs: linear transformations, with rank and nullity describing what they preserve and what they crush.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 2.9: The rank theorem. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 2.7: Basis and dimension. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 9.8: The kernel and image of a linear map. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Strang, G. (1993). The fundamental theorem of linear algebra. The American Mathematical Monthly, 100(9), 848-855. doi.org/10.2307/2324660
5. Weisstein, E. W. (n.d.). Matrix rank. In MathWorld. Wolfram Research. mathworld.wolfram.com
6. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

A matrix is not only a static array; it is a machine that transforms vectors. Multiplying a vector x by a matrix A produces a new vector Ax, so the matrix defines a function from one space to another: vectors in, vectors out. Feed [1 ; 0] into [0 -1 ; 1 0] and out comes [0 ; 1]; the machine turned east into north. The functions that arise this way are exactly the linear transformations, and this lesson is about reading a matrix as an action: rotations, reflections, stretches, shears, and projections, plus the algebra that combines them.

The definition

A function T from R^n to R^m is a linear transformation if it respects the two vector-space operations, for all vectors u, v and every scalar c:

1. Additivity: T(u + v) = T(u) + T(v).
2. Homogeneity: T(c v) = c T(v).

Together these say T preserves linear combinations: T(a u + b v) = a T(u) + b T(v). A consequence is that T must send the zero vector to the zero vector: set c = 0 in homogeneity and T(0) = 0 falls out. Multiplication by a matrix always satisfies both rules, and conversely every linear transformation between coordinate spaces is multiplication by some matrix.

Let us test drive the definition. Is T(x, y) = (x + y, 2x) linear? Try additivity on two general inputs: T((x1, y1) + (x2, y2)) = T(x1 + x2, y1 + y2) = (x1 + x2 + y1 + y2, 2x1 + 2x2), which regroups into (x1 + y1, 2x1) + (x2 + y2, 2x2) = T(x1, y1) + T(x2, y2). Scaling checks the same way. Linear. In practice the two rules are often checked as the single combined condition T(au + bv) = a T(u) + b T(v): one identity, both requirements.

Now T(x, y) = (x + 1, y), a shift right by one. It fails before any algebra: T(0, 0) = (1, 0), not the zero vector, and linear maps must fix the origin. And T(x, y) = (xy, x)? Test homogeneity with numbers: T(2 * (1, 1)) = T(2, 2) = (4, 2), but 2 * T(1, 1) = 2(1, 1) = (2, 2). The mismatch convicts it: products of coordinates break linearity, exactly as products of variables broke linear equations in Module 1.

Key idea: linear means combinations pass through the function untouched, and the origin stays put.

The standard matrix

Because a linear transformation preserves combinations, it is completely determined by what it does to the standard basis vectors. Every input is a combination of them, [x ; y] = x e1 + y e2, so T([x ; y]) = x T(e1) + y T(e2): knowing the two images means knowing everything. The standard matrix of T is the matrix whose columns are the images T(e1), T(e2), ..., T(en). Then T(x) = Ax for every x.

Worked example. Let T rotate the plane 90 degrees counterclockwise. It sends e1 = [1 ; 0] to [0 ; 1] and e2 = [0 ; 1] to [-1 ; 0]. Placing these images as columns gives the standard matrix

A = [0 -1 ; 1 0].

Check on [3 ; 0]: A[3 ; 0] = [0*3 + (-1)*0 ; 1*3 + 0*0] = [0 ; 3], correctly rotating the arrow a quarter turn. For an arbitrary angle theta, the same recipe gives the general rotation matrix: e1 lands at [cos(theta) ; sin(theta)] and e2 at [-sin(theta) ; cos(theta)], so the matrix is [cos(theta) -sin(theta) ; sin(theta) cos(theta)]. Set theta = 90 degrees and the entries collapse to the matrix above.

A second build. Suppose a transformation sends e1 to [2 ; 1] and e2 to [0 ; 3]. Its standard matrix is A = [2 0 ; 1 3]. The image of [1 ; 2] is then 1[2 ; 1] + 2[0 ; 3] = [2 ; 7], and the matrix product agrees: A[1 ; 2] = [2*1 + 0*2 ; 1*1 + 3*2] = [2 ; 7]. Columns are destinations: that is the whole trick.

A gallery of 2D transformations

Transformation	Standard matrix
Scale by factor k	[k 0 ; 0 k]
Reflect across the x-axis	[1 0 ; 0 -1]
Rotate 90 degrees counterclockwise	[0 -1 ; 1 0]
Horizontal shear by k	[1 k ; 0 1]
Project onto the x-axis	[1 0 ; 0 0]

Each row of the gallery is the column recipe in action. The reflection fixes e1 and flips e2 to [0 ; -1]. The shear leaves e1 alone and tilts e2 to [k ; 1], so vertical lines lean while horizontal ones slide. The projection keeps e1 and crushes e2 to zero, flattening the plane onto the x-axis.

The gallery extends easily. A diagonal matrix with different entries, such as [2 0 ; 0 3], stretches the x-direction by 2 and the y-direction by 3, turning squares into rectangles. A vertical shear [1 0 ; k 1] tilts e1 instead of e2. Reading any unfamiliar 2x2 matrix is the same two-step routine: where did the columns send e1 and e2?

Watch the shear [1 1 ; 0 1] move the unit square with corners (0,0), (1,0), (1,1), (0,1). The corners map to (0,0), (1,0), (1*1 + 1*1, 0 + 1) = (2,1), and (1,1): the square leans into a parallelogram. Its base and height are unchanged, so the area is still 1, matching det = 1*1 - 1*0 = 1 from Module 3. The determinant of a transformation's matrix is precisely its area scale factor, and the projection's determinant of 0 announces the collapse.

Composition is matrix multiplication

Composing two transformations corresponds to multiplying their matrices: doing S after T is the matrix product (matrix of S)(matrix of T), with the first action written on the right, nearest the input. This is the deep reason matrix multiplication is defined by the row-by-column rule, and why its order matters: applying transformations in a different order generally gives a different result.

Worked example. Let R = [0 -1 ; 1 0] (rotate 90 degrees counterclockwise) and F = [1 0 ; 0 -1] (reflect across the x-axis). Rotate first, then reflect: FR = [1 0 ; 0 -1][0 -1 ; 1 0]. Entry by entry: top row 1*0 + 0*1 = 0 and 1*(-1) + 0*0 = -1; bottom row 0*0 + (-1)*1 = -1 and 0*(-1) + (-1)*0 = 0. So FR = [0 -1 ; -1 0].

Reflect first, then rotate: RF = [0 -1 ; 1 0][1 0 ; 0 -1] = [0 1 ; 1 0], computed the same way. The two answers differ.

Follow e1 to feel why. Rotate-then-reflect: e1 goes to [0 ; 1], then reflects to [0 ; -1]. Reflect-then-rotate: e1 is fixed by the reflection, then rotates to [0 ; 1]. Different journeys, different destinations, and the matrices knew it first.

Composition with itself is worth one look. Rotating 90 degrees twice should be a half turn, and the algebra agrees: R^2 = [0 -1 ; 1 0][0 -1 ; 1 0] = [-1 0 ; 0 -1], the matrix that negates every vector, which is exactly a 180-degree rotation. When a geometric fact and a matrix product confirm each other like this, both are probably right.

A real use: the graphics pipeline

Every frame of a 3D game repositions millions of vertices, and each repositioning is a matrix product. A character model is built at the origin, scaled, rotated to face its heading, and moved into the scene; the renderer then applies a camera matrix and a perspective matrix. Rather than run five transformations per vertex, the engine multiplies the five matrices once and applies the single product to every vertex, associativity doing the heavy lifting. Skeletal animation chains matrices down each limb, wrist after elbow after shoulder, so a waving hand is a product of joint transformations.

Order is a daily concern there. Stretch S = [2 0 ; 0 1] (double the x-direction) and rotate R = [0 -1 ; 1 0]. Stretch then rotate: RS = [0 -1 ; 2 0], sending e1 to [0 ; 2]: the doubled arrow stands upright. Rotate then stretch: SR = [0 -2 ; 1 0], sending e1 to [0 ; 1]: the arrow turned first and the stretch missed it.

Artists see the difference on screen as a limb that fattens the wrong way. One caution: moving an object sideways, translation, is not linear (it moves the origin), so graphics systems embed the plane in one higher dimension, homogeneous coordinates, where translations become matrices too.

Common wrong turns

• Calling translation linear. T(x) = x + [1 ; 0] moves the origin, and T(0) is not 0. Linear maps pivot around a fixed origin.
• Writing composition in reading order. "Do T, then S" is (matrix of S)(matrix of T); the matrix nearest the vector acts first. Reversing the product usually changes the transformation.
• Building the matrix from rows. The images of e1, e2 are the columns. Row-stacking them transposes the matrix and wrecks every computation after.
• Verifying linearity with one friendly example. A single input pair working proves nothing; the two rules must hold for all inputs. One failing example, however, refutes.
• Assuming every matrix is reversible motion. The projection [1 0 ; 0 0] destroys the y-coordinate permanently; determinant zero warns you no inverse exists.

Try it

Find the standard matrix of "rotate 90 degrees counterclockwise, then project onto the x-axis," and compute the image of [2 ; 1].

Worked answer. The parts are R = [0 -1 ; 1 0] and P = [1 0 ; 0 0], and rotation acts first, so the composite is PR = [1 0 ; 0 0][0 -1 ; 1 0] = [0 -1 ; 0 0]. Then PR[2 ; 1] = [0*2 + (-1)*1 ; 0] = [-1 ; 0]. Check in two steps: R[2 ; 1] = [-1 ; 2], and projecting kills the second coordinate, leaving [-1 ; 0]. The answers match.

Recap

• Linear transformations preserve sums and scalings, and every one between coordinate spaces is multiplication by a matrix.
• The standard matrix stores the images of the basis vectors as its columns.
• Scales, reflections, rotations, shears, and projections are all matrices, and the determinant reports their area effect.
• Composition multiplies matrices right to left, and order genuinely matters.

A transformation also raises two subspace questions: what does it crush to zero, and what can it reach? Those are the kernel and the range, next.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 3.1: Matrix transformations. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 3.3: Linear transformations. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 5.2: The matrix of a linear transformation I. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 5.4: Special linear transformations in R^2. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu
6. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

Two subspaces reveal everything about a linear transformation T(x) = Ax: what it sends to zero, and what it can produce. These are the kernel and the range. Every question you can ask about a linear machine, whether it loses information, whether it can hit every target, whether it can be reversed, reduces to a question about these two subspaces, and both are computable with the elimination skills you already have.

The kernel

The kernel (or null space) of T is the set of all inputs that T crushes to the zero vector: ker(T) = { x : Ax = 0 }. It is a subspace of the input space, and finding it means solving the homogeneous system Ax = 0. Its dimension is the nullity from last lesson: kernel and null space are two names for one object. The kernel measures how much information the transformation destroys: whenever two inputs differ by a kernel vector, the machine maps them to the same output, so their difference is invisible on the far side.

Worked example. For A = [1 2 ; 2 4], solve Ax = 0. Row-reduce: replace R2 with R2 - 2*R1 to get [0 0], leaving the single equation x1 + 2 x2 = 0, so x1 = -2 x2. Setting x2 = t, the kernel is { t[-2 ; 1] : t real }, a line through the origin. Because the kernel contains nonzero vectors, this transformation is not reversible; different inputs collapse to the same output. Concretely, [0 ; 0] and [-2 ; 1] both map to [0 ; 0], and no one downstream can tell them apart.

A wider machine. Let B = [1 2 1 ; 0 1 1], mapping R^3 to R^2. The system Bx = 0 reads x1 + 2 x2 + x3 = 0 and x2 + x3 = 0. The second equation gives x2 = -x3; set x3 = t, so x2 = -t. Substitute into the first: x1 - 2t + t = 0, so x1 = t.

The kernel is the line { t[1 ; -1 ; 1] } in R^3. Check the basis vector: row 1 gives 1 - 2 + 1 = 0, row 2 gives -1 + 1 = 0. A machine from a bigger space to a smaller one must crush something; rank-nullity will say exactly how much.

At the other extreme, the rotation matrix [0 -1 ; 1 0] has kernel containing only the zero vector: rotating an arrow never flattens it, so Ax = 0 forces x = 0. A trivial kernel is the signature of a lossless machine.

Key idea: the kernel is the machine's blind spot, computed by solving Ax = 0; only the zero vector there means nothing is lost.

The range

The range (or column space) of T is the set of all outputs it can actually produce: range(T) = { Ax : x in R^n }. Since Ax is a linear combination of the columns of A with weights from x, the range is exactly the span of the columns of A. Its dimension is the rank of A. For the matrix above, both columns are multiples of [1 ; 2], so the range is the single line spanned by [1 ; 2], and the rank is 1.

The range answers solvability instantly: Ax = b has a solution exactly when b lies in the range. For A = [1 2 ; 2 4], the target b = [3 ; 6] sits on the line through [1 ; 2] (it is 3 times it), so the system is consistent; the target b = [3 ; 7] is off the line, so no input reaches it, matching the rank test of the last lesson.

For the wide matrix B = [1 2 1 ; 0 1 1], reduce to find pivots: the echelon form is already [1 2 1 ; 0 1 1], with pivots in columns 1 and 2. A basis for the range is the corresponding original columns, [1 ; 0] and [2 ; 1], which are independent and span all of R^2. So this machine reaches every target in its codomain: rank 2, range everything.

Key idea: the range is the span of the columns; a target is reachable exactly when it lies in that span.

One-to-one and onto

The kernel and range answer the two basic questions about any function.

• T is one-to-one (injective, distinct inputs give distinct outputs) exactly when its kernel is only the zero vector. Any larger kernel means collisions.
• T is onto (surjective, it reaches every vector in the target space) exactly when its range is the whole target space, that is, when the rank equals the dimension of the codomain.

Why does the kernel control one-to-one? If Au = Av for different inputs, then A(u - v) = 0 with u - v nonzero, a nonzero kernel vector. Conversely a nonzero kernel vector k collides with 0. So collisions and kernel vectors are the same phenomenon, and checking one-to-one for a linear map costs only one homogeneous solve.

Classify the cast so far. A = [1 2 ; 2 4]: kernel is a line (not one-to-one), range is a line inside R^2 (not onto). Neither. The rotation: kernel trivial, range all of R^2: both, hence invertible. The projection [1 0 ; 0 0]: kernel is the y-axis, range is the x-axis: neither.

The wide matrix B: onto but not one-to-one. And the tall matrix C = [1 0 ; 0 1 ; 0 0], mapping R^2 into R^3: its columns are independent, so the kernel is trivial and it is one-to-one, but its range is only the plane of vectors with third coordinate zero: not onto. Every combination occurs, except one.

The exception is where rank-nullity bites. For the singular square matrix A = [1 2 ; 2 4], the audit reads rank + nullity = 1 + 1 = 2 columns, and for the wide matrix B it reads 2 + 1 = 3; in every case, growing the range shrinks the kernel and vice versa. A map from R^3 to R^2 can never be one-to-one (nullity is at least 1), and a map from R^2 to R^3 can never be onto (rank is at most 2). For a square matrix, the ledger balances exactly: one-to-one and onto happen together.

The invertible matrix theorem, a first tour

For an n x n matrix A, the following statements are all equivalent: each one true makes all true, each one false makes all false.

• A is invertible;
• det(A) is nonzero;
• rank(A) = n, with a pivot in every column;
• the kernel is only the zero vector, and the columns are independent;
• T(x) = Ax is one-to-one, and it is onto;
• Ax = b has exactly one solution for every b.

This is the grand unifying statement of the first half of the course: determinants, rank, kernels, and solvability are one story told in four vocabularies. When you need to certify invertibility, you are free to check whichever condition is cheapest, and in practice that is usually the pivot count.

Use it as a switchboard. For [1 2 ; 2 4], one glance at det = 0 flips every switch off at once: not invertible, rank below 2, a nonzero kernel vector waiting, columns dependent, neither one-to-one nor onto, and right-hand sides that fail. One cheap computation, six conclusions.

A real use: why one photo cannot recover depth

A camera performs, to good approximation, a linear projection from 3D space to a 2D image plane. Its kernel direction is the viewing ray: move a point straight toward or away from the lens along that ray and its pixel does not change. All depth information along the ray is crushed, exactly like the y-coordinate under [1 0 ; 0 0]. That is why a single photograph cannot tell a large distant object from a small near one.

Stereo vision fixes the problem with a second camera whose kernel points a different way: the two projections together have only the zero vector in common in their kernels, and depth becomes computable again. Robots, self-driving cars, and your own two eyes all exploit this kernel arithmetic.

Common wrong turns

• Housing the kernel and range in the same space. For a 2 x 3 matrix, kernel vectors live in R^3 (inputs) and range vectors in R^2 (outputs). Keep the addresses straight.
• Saying "the kernel is empty." It never is: A0 = 0 always. The lossless case is a trivial kernel, containing exactly the zero vector.
• Confusing range with codomain. The codomain is where outputs are allowed to land; the range is where they actually land. Onto means the two coincide.
• Taking range basis vectors from the echelon form. Row operations change column spans. Find pivot positions in the reduced matrix, then harvest those columns from the original.
• Expecting shape to be overridable. Wide matrices (more columns than rows) are never one-to-one; tall ones are never onto. Rank-nullity is a law, not a tendency.

Try it

For A = [1 0 -1 ; 2 1 0], find a basis for the kernel, and decide whether the map is onto and whether it is one-to-one.

Worked answer. Solve Ax = 0: the equations are x1 - x3 = 0 and 2 x1 + x2 = 0. From the first, x1 = x3; set x3 = t, so x1 = t, and the second gives x2 = -2t. Kernel basis: [1 ; -2 ; 1].

Check: row 1 gives 1 - 1 = 0; row 2 gives 2 - 2 + 0 = 0. The two columns [1 ; 2] and [0 ; 1] are already independent, so the rank is 2, equal to the dimension of R^2: the map is onto. The kernel is a line, so it is not one-to-one, and rank-nullity confirms the audit: 2 + 1 = 3 columns.

Recap

• Kernel: solutions of Ax = 0, a subspace of the input space; it collects everything the machine cannot distinguish from zero.
• Range: the span of the columns, a subspace of the output space; reachable targets and nothing more.
• One-to-one means trivial kernel; onto means range equals codomain; square matrices get both or neither.
• The invertible matrix theorem ties determinant, rank, kernel, and solvability into one switch.

Next, we hunt for the directions a matrix does not turn at all: eigenvectors, and the stretch factors that come with them.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 3.2: One-to-one and onto transformations. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 3.6: The invertible matrix theorem. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 5.7: The kernel and image of a linear map. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Strang, G. (1993). The fundamental theorem of linear algebra. The American Mathematical Monthly, 100(9), 848-855. doi.org/10.2307/2324660
5. Axler, S. (n.d.). Linear algebra done right. linear.axler.net
6. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

Most vectors change direction when a matrix acts on them. A precious few do not: they are only stretched or shrunk, keeping their line. These special directions are the eigenvectors, and the stretch factors are the eigenvalues. They expose the deepest structure of a matrix, and this lesson builds the complete recipe for finding them: set up the characteristic equation, solve for the eigenvalues, then recover an eigenvector for each.

The eigenvalue equation

A nonzero vector v is an eigenvector of a square matrix A, with eigenvalue lambda (the Greek letter, read: LAM-duh), if

A v = lambda v.

In words, multiplying by A has the same effect on v as scaling it by the number lambda, nothing more. The direction is preserved; only the length (and possibly the sign) changes. We require v to be nonzero, since A * 0 = 0 always holds trivially and would tell us nothing.

See the phenomenon before the machinery. Let A = [2 1 ; 1 2]. Feed it [1 ; 0]: out comes [2 ; 1], a genuinely different direction; this input turned. Now feed it [1 ; 1]: out comes [2 + 1 ; 1 + 2] = [3 ; 3], which is exactly 3[1 ; 1]. The diagonal direction did not turn at all; it was stretched by 3. So [1 ; 1] is an eigenvector of this matrix with eigenvalue 3, and the question becomes: how do we find such directions systematically instead of by lucky guessing?

The characteristic equation

Rewrite A v = lambda v as A v - lambda v = 0, or (A - lambda I) v = 0. Subtracting lambda I subtracts lambda from each diagonal entry only; the identity keeps the bookkeeping honest. For a nonzero v to solve this homogeneous system, the matrix A - lambda I must have a nontrivial kernel, which by the invertible matrix theorem means it is singular, which means its determinant is zero. That gives the characteristic equation:

det(A - lambda I) = 0.

Its solutions are the eigenvalues. For a 2x2 matrix this is a quadratic in lambda, so there are at most two eigenvalues. A convenient 2x2 shortcut: the quadratic always takes the form lambda^2 - (trace) lambda + det = 0, where the trace is the sum of the diagonal entries. Consequently the two eigenvalues always add up to the trace and multiply to the determinant, a free error check we will use constantly.

Key idea: eigenvalues are exactly the numbers lambda that make A - lambda I singular; the eigenvectors are the kernel vectors that appear at those special values.

Worked example

Find the eigenvalues and eigenvectors of A = [2 1 ; 1 2].

1. Form A - lambda I = [2 - lambda, 1 ; 1, 2 - lambda].
2. Its determinant is (2 - lambda)^2 - (1)(1) = lambda^2 - 4 lambda + 4 - 1 = lambda^2 - 4 lambda + 3.
3. Set it to zero and factor: lambda^2 - 4 lambda + 3 = (lambda - 1)(lambda - 3) = 0. So lambda = 1 and lambda = 3.

Before hunting eigenvectors, audit: the trace is 2 + 2 = 4, and 1 + 3 = 4. The determinant is 4 - 1 = 3, and 1 * 3 = 3. Both books balance. Now find an eigenvector for each eigenvalue.

• For lambda = 3: solve (A - 3I)v = 0, that is [-1 1 ; 1 -1] v = 0. The equation is -v1 + v2 = 0, so v1 = v2. An eigenvector is [1 ; 1]. Check: A[1 ; 1] = [2*1 + 1*1 ; 1*1 + 2*1] = [3 ; 3] = 3[1 ; 1]. Correct.
• For lambda = 1: solve (A - I)v = 0, that is [1 1 ; 1 1] v = 0. The equation is v1 + v2 = 0, so v1 = -v2. An eigenvector is [1 ; -1]. Check: A[1 ; -1] = [2 - 1 ; 1 - 2] = [1 ; -1] = 1[1 ; -1]. Correct.

Each eigenvector spans a line that the matrix leaves in place, stretching it by its eigenvalue. Eigenvectors are only determined up to a scalar, so any nonzero multiple of [1 ; 1] is also an eigenvector for lambda = 3; the set of all of them, plus the zero vector, is the eigenspace for that eigenvalue. Notice too that the rows of A - 3I became dependent, one the negative of the other. They had to: we chose lambda precisely to make the matrix singular, so a free variable is expected, not alarming.

A second example, start to finish

Find the eigen-data of A = [4 1 ; 2 3].

1. A - lambda I = [4 - lambda, 1 ; 2, 3 - lambda], so the determinant is (4 - lambda)(3 - lambda) - (1)(2).
2. Expand: 12 - 4 lambda - 3 lambda + lambda^2 - 2 = lambda^2 - 7 lambda + 10.
3. Factor: lambda^2 - 7 lambda + 10 = (lambda - 2)(lambda - 5) = 0, so lambda = 2 and lambda = 5. Audit: trace 4 + 3 = 7 = 2 + 5; determinant 12 - 2 = 10 = 2 * 5.
4. lambda = 2: (A - 2I)v = [2 1 ; 2 1]v = 0 says 2 v1 + v2 = 0, so v2 = -2 v1: take [1 ; -2]. Check: A[1 ; -2] = [4 - 2 ; 2 - 6] = [2 ; -4] = 2[1 ; -2].
5. lambda = 5: (A - 5I)v = [-1 1 ; 2 -2]v = 0 says v1 = v2: take [1 ; 1]. Check: A[1 ; 1] = [4 + 1 ; 2 + 3] = [5 ; 5] = 5[1 ; 1].

The full procedure fits in five lines, and two of them are checks. Different matrix, same rhythm.

Special cases worth knowing

Triangular matrices hand you the answer. For [3 5 ; 0 7], the characteristic determinant is (3 - lambda)(7 - lambda) - 0, so the eigenvalues are the diagonal entries 3 and 7. This covers diagonal matrices too.

A repeated eigenvalue can run short of directions. The shear [1 1 ; 0 1] has characteristic polynomial (1 - lambda)^2, so lambda = 1 twice. But (A - I)v = [0 1 ; 0 0]v = 0 forces only v2 = 0: the eigenvectors form a single line, spanned by [1 ; 0]. A double eigenvalue with only one independent direction will block diagonalization next lesson.

Some real matrices have no real eigenvalues. The rotation [0 -1 ; 1 0] gives lambda^2 + 1 = 0, with no real solutions, which matches the geometry: a quarter-turn leaves no direction unturned. (Allowing complex numbers, the roots exist and are imaginary; that door opens in later courses.)

Eigenvalue zero means singular. If lambda = 0 solves the characteristic equation, then det(A) = 0, and the eigenvectors for 0 are exactly the nonzero kernel vectors: the directions the matrix flattens.

A real use: ranking pages by importance

Early web search faced a circular definition: a page is important if important pages link to it. Linear algebra breaks the circle. Put the link structure in a matrix M whose entry (i, j) is the fraction of page j's endorsement flowing to page i, and seek an importance vector v that reproduces itself under the flow: M v = v. That is an eigenvector equation with lambda = 1.

Tiny worked web. Two pages. Page A sends all its weight to B; page B splits its weight evenly between A and itself. Then M = [0 0.5 ; 1 0.5]. Solve Mv = v, that is (M - I)v = 0: the first row says -v1 + 0.5 v2 = 0, so v2 = 2 v1. Take v = [1 ; 2] and check: Mv = [0*1 + 0.5*2 ; 1*1 + 0.5*2] = [1 ; 2].

The self-consistent ranking gives B twice the importance of A, sensible since B receives all of A's endorsement plus half of its own. At web scale the same eigenvector is found by repeated multiplication (the power method), and it was the seed of Google's PageRank.

Common wrong turns

• Subtracting lambda from every entry. A - lambda I touches only the diagonal. Off-diagonal entries keep their values.
• Losing the cross term. The 2x2 determinant is (a - lambda)(d - lambda) - bc; forgetting the -bc changes every root. The trace and determinant audit catches this immediately.
• Accepting v = 0. The zero vector solves (A - lambda I)v = 0 for every lambda and certifies nothing. Eigenvectors are the nonzero solutions.
• Expecting a unique eigenvector. Any nonzero scalar multiple works; report a clean representative and remember it stands for a whole line.
• Getting a unique solution from (A - lambda I)v = 0. If elimination leaves no free variable, the lambda you used is not an eigenvalue; recheck the characteristic equation before recomputing.

Try it

Find the eigenvalues and an eigenvector for each, for A = [3 2 ; 1 2].

Worked answer. The characteristic polynomial is (3 - lambda)(2 - lambda) - 2 = lambda^2 - 5 lambda + 6 - 2 = lambda^2 - 5 lambda + 4 = (lambda - 1)(lambda - 4), so lambda = 1 and lambda = 4. Audit: trace 5 = 1 + 4; determinant 6 - 2 = 4 = 1*4.

For lambda = 1: [2 2 ; 1 1]v = 0 gives v1 = -v2, eigenvector [1 ; -1]; check A[1 ; -1] = [3 - 2 ; 1 - 2] = [1 ; -1]. For lambda = 4: [-1 2 ; 1 -2]v = 0 gives v1 = 2 v2, eigenvector [2 ; 1]; check A[2 ; 1] = [6 + 2 ; 2 + 2] = [8 ; 4] = 4[2 ; 1]. Both verified.

Recap

• An eigenvector keeps its line under the matrix; the eigenvalue is the stretch factor: Av = lambda v, v nonzero.
• Eigenvalues solve det(A - lambda I) = 0; for 2x2 that is lambda^2 - (trace)lambda + det = 0.
• For each eigenvalue, eigenvectors are the kernel of A - lambda I, found by elimination, and free variables are guaranteed.
• Triangular matrices show eigenvalues on the diagonal; rotations may have no real ones; zero as an eigenvalue means singular.

Eigenvectors were found one at a time here. Next lesson assembles them into a basis and rebuilds the whole matrix around them: diagonalization.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 5.1: Eigenvalues and eigenvectors. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 5.2: The characteristic polynomial. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Kuttler, K. (n.d.). 7.1: Eigenvalues and eigenvectors of a matrix. In A first course in linear algebra. LibreTexts. math.libretexts.org
4. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1-7), 107-117. doi.org/10.1016/S0169-7552(98)00110-X
5. Weisstein, E. W. (n.d.). Eigenvector. In MathWorld. Wolfram Research. mathworld.wolfram.com
6. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu
7. Hefferon, J. (n.d.). Linear algebra [Open textbook]. hefferon.net

Eigenvectors are useful because, in the basis they provide, a complicated matrix becomes a simple diagonal one. This process, diagonalization, turns hard matrix computations into easy scalar ones. The last lesson found the special directions one at a time; this lesson assembles them into a coordinate system in which the matrix does nothing but stretch the axes.

What diagonalization means

A square matrix A is diagonalizable if it can be written as

A = P D P^(-1),

where D is a diagonal matrix and P is invertible. The recipe is built directly from eigen-data: the columns of P are eigenvectors of A, and D is the diagonal matrix of the corresponding eigenvalues in the same order. An n x n matrix is diagonalizable exactly when it has n linearly independent eigenvectors, enough to form a basis; independence is what makes P invertible.

Read the formula as a three-step journey, right to left. Multiplying by P^(-1) translates a vector into eigenvector coordinates: how much of each special direction it contains. Multiplying by D then scales each coordinate by its eigenvalue, the only thing the matrix ever does in this basis. Finally P translates back to standard coordinates. Same transformation, better coordinate system: that is the whole idea, and it is the payoff promised in the basis lesson.

One theorem does most of the eligibility checking: eigenvectors belonging to distinct eigenvalues are automatically independent. So an n x n matrix with n different eigenvalues is always diagonalizable. Repeated eigenvalues are the only risky case.

Key idea: diagonalization rewrites A in its eigenvector basis, where it is purely a per-axis stretch.

Worked example

Diagonalize A = [2 1 ; 1 2], using the eigen-data from the previous lesson: eigenvalue 3 with eigenvector [1 ; 1], and eigenvalue 1 with eigenvector [1 ; -1].

1. Put the eigenvectors as columns of P: P = [1 1 ; 1 -1].
2. Put the matching eigenvalues on the diagonal of D in the same order: D = [3 0 ; 0 1].
3. Then A = P D P^(-1). Here det(P) = 1*(-1) - 1*1 = -2, so P^(-1) = (1/-2)[-1 -1 ; -1 1] = [0.5 0.5 ; 0.5 -0.5].

Claims deserve verification, so multiply the three factors. First P D = [1 1 ; 1 -1][3 0 ; 0 1]: row by column gives [3 1 ; 3 -1] (each column of P picked up its eigenvalue). Then (P D) P^(-1) = [3 1 ; 3 -1][0.5 0.5 ; 0.5 -0.5]. Top-left: 3*0.5 + 1*0.5 = 2. Top-right: 3*0.5 + 1*(-0.5) = 1. Bottom-left: 3*0.5 + (-1)*0.5 = 1. Bottom-right: 3*0.5 + (-1)*(-0.5) = 2. The product is [2 1 ; 1 2] = A, exactly as claimed.

The two independent eigenvectors guarantee this works. In the eigenvector basis, A does nothing except scale one axis by 3 and the other by 1, which is exactly what D encodes.

The factorization also puts the matrix's vital statistics on display. The determinant is the product down D's diagonal, 3 * 1 = 3, matching det(A) = 4 - 1 = 3, and the trace is its sum, 3 + 1 = 4, matching 2 + 2. Once a matrix is diagonalized, its determinant, trace, powers, and long-run behavior are all readable at a glance.

Why it matters: matrix powers

Diagonalization makes powers effortless. Because the inner P^(-1) P factors cancel,

A^k = P D^k P^(-1),

and raising a diagonal matrix to a power raises each diagonal entry to that power, nothing else: D^k = [3^k 0 ; 0 1^k]. So instead of multiplying A by itself k times, you compute 3^k and 1^k and reassemble.

Worked check. Compute A^3 both ways. Via the formula: D^3 = [27 0 ; 0 1], so P D^3 = [27 1 ; 27 -1], and multiplying by P^(-1): top-left 27*0.5 + 1*0.5 = 14, top-right 27*0.5 - 0.5 = 13, bottom-left 27*0.5 - 0.5 = 13, bottom-right 27*0.5 + 0.5 = 14. So A^3 = [14 13 ; 13 14].

The long way: A^2 = [2*2 + 1*1, 2*1 + 1*2 ; 1*2 + 2*1, 1*1 + 2*2] = [5 4 ; 4 5], and A^3 = A^2 A = [5*2 + 4*1, 5*1 + 4*2 ; 4*2 + 5*1, 4*1 + 5*2] = [14 13 ; 13 14]. The two routes agree, and for k = 100 only one of them is humane: A^100 = P [3^100 0 ; 0 1] P^(-1), two scalar powers and two small multiplications.

This is the engine behind long-run predictions: in a Markov chain that models how a population moves between states each step, the eigenvalues govern the steady state, and in models of population growth or discrete dynamical systems, the largest eigenvalue sets the long-term growth rate. The eigenvalue with the biggest absolute value dominates as k grows, so the system lines up with its leading eigenvector.

When diagonalization fails

Not every matrix cooperates. The shear [1 1 ; 0 1] has eigenvalue 1 twice but, as the last lesson showed, only one independent eigen-direction, the line through [1 ; 0]. Building P requires two independent eigenvector columns, and there is no second one to use; any attempted P is singular. Such matrices are called defective, and they are the price of repeated eigenvalues gone wrong. (A repeated eigenvalue is not always fatal: the identity matrix has eigenvalue 1 twice and is already diagonal.)

One glowing special case: symmetric matrices, those with A^T = A, are always diagonalizable, never defective, and their eigenvectors for different eigenvalues come out perpendicular. Our worked example is symmetric, and sure enough its eigenvectors satisfy [1 ; 1] . [1 ; -1] = 1 - 1 = 0. This spectral theorem, proved in later courses, is why symmetric matrices are the best-behaved citizens of linear algebra.

A real use: will it rain in the long run?

A simple weather model: if today is sunny, tomorrow is sunny with probability 0.9; if today is rainy, tomorrow is sunny with probability 0.5. Stack the state as a vector [sunny ; rainy] and the transition matrix is M = [0.9 0.5 ; 0.1 0.5], each column summing to 1. Tomorrow's outlook is M times today's, so a forecast k days out is M^k times today's: a matrix power problem, and eigenvalues are the right tool.

The characteristic polynomial is lambda^2 - 1.4 lambda + (0.45 - 0.05) = lambda^2 - 1.4 lambda + 0.4 = (lambda - 1)(lambda - 0.4), so the eigenvalues are 1 and 0.4. The steady state is the eigenvector for 1: solve (M - I)v = 0, that is [-0.1 0.5 ; 0.1 -0.5]v = 0, giving 0.1 v1 = 0.5 v2, so v1 = 5 v2: proportions [5 ; 1], meaning 5/6 of days sunny in the long run, about 83 percent, regardless of today.

The second eigenvalue 0.4 is the forgetting rate: the influence of today's weather shrinks by a factor 0.4 each day, since 0.4^k collapses toward zero: after five days, 0.4^5 = 0.01 approximately, one percent influence left.

Watch the convergence from a certainly-sunny start [1 ; 0]: one step gives M[1 ; 0] = [0.9 ; 0.1]; two steps give M[0.9 ; 0.1] = [0.81 + 0.05 ; 0.09 + 0.05] = [0.86 ; 0.14], already sliding toward [0.833 ; 0.167]. Ecologists run the same mathematics on population stage matrices, where the dominant eigenvalue above or below 1 decides growth or decline, and financial analysts use it on credit-rating transition tables. Whenever a system hops between states with fixed probabilities, the eigenvalue 1 holds its destiny and the second eigenvalue sets the pace of arrival.

Common wrong turns

• Mismatching the order. Column j of P must pair with entry j of D. Swapping the eigenvalues without swapping the eigenvector columns builds a different, wrong matrix.
• Reversing the sandwich. With eigenvectors in P, the formula is A = P D P^(-1), not P^(-1) D P. Verify by multiplying once; the check costs a minute.
• Powering entries directly. Entrywise powering is not matrix powering: [1 1 ; 0 1]^2 = [1 2 ; 0 1], not [1 1 ; 0 1]. Only diagonal matrices power entry by entry, which is the entire point of D.
• Assuming diagonalizability. Defective matrices like the shear have too few eigenvectors. Count independent eigenvectors before writing P.
• Using a singular P. If your eigenvector columns are dependent, you double-collected from one eigenspace. Different eigenvalues never cause this; repeated ones can.

Try it

Diagonalize A = [1 2 ; 2 1], and use the factorization to compute A^2.

Worked answer. Characteristic polynomial: lambda^2 - 2 lambda + (1 - 4) = lambda^2 - 2 lambda - 3 = (lambda - 3)(lambda + 1), so the eigenvalues are 3 and -1. For lambda = 3: [-2 2 ; 2 -2]v = 0 gives v1 = v2, eigenvector [1 ; 1]. For lambda = -1: [2 2 ; 2 2]v = 0 gives v1 = -v2, eigenvector [1 ; -1]. So P = [1 1 ; 1 -1], D = [3 0 ; 0 -1], and P^(-1) = [0.5 0.5 ; 0.5 -0.5] as before.

Then D^2 = [9 0 ; 0 1], so P D^2 = [9 1 ; 9 -1], and multiplying by P^(-1): top-left 4.5 + 0.5 = 5, top-right 4.5 - 0.5 = 4, bottom-left 4.5 - 0.5 = 4, bottom-right 4.5 + 0.5 = 5: A^2 = [5 4 ; 4 5]. Direct check: A^2 = [1 + 4, 2 + 2 ; 2 + 2, 4 + 1] = [5 4 ; 4 5]. Agreement, and note the negative eigenvalue: the machine flips its second eigen-direction, but squaring turns the flip into (-1)^2 = 1.

Recap

• A = P D P^(-1): eigenvectors in the columns of P, matching eigenvalues on the diagonal of D.
• The factorization is a change of coordinates in which A only stretches axes; verify it by multiplying out once.
• Powers collapse to A^k = P D^k P^(-1), powering only the diagonal, which unlocks long-run predictions.
• Diagonalization needs n independent eigenvectors: distinct eigenvalues guarantee it, symmetric matrices always have it, defective matrices never do.

Eigenvalues thus predict long-term behavior that would be invisible in the raw matrix, which is why they matter far beyond pure mathematics. The final module adds the last geometric tool, perpendicularity, and puts the whole course to work.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 5.4: Diagonalization. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 5.3: Similarity. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Margalit, D., & Rabinoff, J. (n.d.). 5.6: Stochastic matrices. In Interactive linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 7.2: Diagonalization. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. Kuttler, K. (n.d.). 7.3: Applications of spectral theory. In A first course in linear algebra. LibreTexts. math.libretexts.org
6. Axler, S. (n.d.). Linear algebra done right. linear.axler.net
7. Strang, G. (n.d.). 18.06SC Linear algebra [Course materials with lecture videos, Fall 2011]. MIT OpenCourseWare. ocw.mit.edu

Orthogonality, the linear-algebra word for perpendicularity, brings geometry into full force. It makes bases easy to use, it splits vectors into independent parts, and it underlies the most important application of the whole subject, least-squares fitting. The tools are modest: the dot product from Module 4 and one new formula. The consequences run from audio compression to satellite navigation.

Orthogonal and orthonormal sets

Recall that two vectors are orthogonal when their dot product is zero. A set of vectors is an orthogonal set if every pair in it is orthogonal. If, in addition, every vector has length 1, the set is orthonormal. The standard basis {[1 ; 0], [0 ; 1]} is orthonormal: the two vectors are perpendicular and each has length 1. The set {[1 ; 1], [1 ; -1]} is orthogonal, since 1*1 + 1*(-1) = 0, but not orthonormal, because each vector has length sqrt(2). Dividing each by its length repairs that: {[1/sqrt(2) ; 1/sqrt(2)], [1/sqrt(2) ; -1/sqrt(2)]} is orthonormal.

Why prize such bases? Because coordinates in them come from dot products, with no system to solve. If {u1, u2} is orthogonal, the weight of a vector b along u1 is (u1 . b)/(u1 . u1), and likewise for u2.

Express b = [4 ; 2] in the basis {[1 ; 1], [1 ; -1]}: the first weight is ([1 ; 1] . [4 ; 2]) / ([1 ; 1] . [1 ; 1]) = (4 + 2)/(1 + 1) = 3, and the second is (4 - 2)/2 = 1. So b = 3[1 ; 1] + 1[1 ; -1], the same coordinates (3, 1) that Module 5 found by solving a system, now produced by two dot products.

In an orthogonal basis, every direction can be measured independently, because no basis vector overlaps any other. That is the working meaning of zero dot product: what happens along one basis direction is invisible to the rest, so each coordinate can be computed, stored, or discarded on its own.

Key idea: orthogonal means no overlap; in an orthogonal basis, each coordinate is a private dot-product measurement.

Projecting one vector onto another

Given a vector b and a direction a, the projection of b onto a is the shadow b casts along the line through a. It is the point on that line closest to b, and the formula is

proj_a(b) = ( (a . b) / (a . a) ) a.

Where does the formula come from? The shadow is some multiple c a of the direction, chosen so that the leftover b - c a is perpendicular to a. Write that requirement with dot products: (b - c a) . a = 0, so a . b - c (a . a) = 0, and solving for the scalar gives c = (a . b)/(a . a). The formula is the perpendicularity condition, solved.

Worked example. Project b = [3 ; 4] onto a = [1 ; 0].

1. Numerator: a . b = 1*3 + 0*4 = 3.
2. Denominator: a . a = 1*1 + 0*0 = 1.
3. Projection: (3/1)[1 ; 0] = [3 ; 0].

The shadow of [3 ; 4] onto the x-axis is [3 ; 0], exactly as geometry predicts. The leftover piece, b - proj_a(b) = [0 ; 4], is perpendicular to a; every vector splits uniquely into a part along a and a part orthogonal to it.

A tilted example. Project b = [3 ; 4] onto a = [1 ; 1]. Numerator: a . b = 3 + 4 = 7. Denominator: a . a = 1 + 1 = 2. Projection: (7/2)[1 ; 1] = [3.5 ; 3.5]. Residual: b - [3.5 ; 3.5] = [-0.5 ; 0.5], and its dot with a is -0.5 + 0.5 = 0: perpendicular, as designed. The residual's length, sqrt(0.25 + 0.25) = sqrt(0.5), about 0.71, is the distance from the point (3, 4) to the line y = x: projection is also a distance-measuring tool.

Because the two pieces are perpendicular, Pythagoras must hold, and it does: ||proj||^2 = 3.5^2 + 3.5^2 = 24.5, the residual contributes 0.25 + 0.25 = 0.5, and the total is 24.5 + 0.5 = 25 = ||b||^2. This split of any vector into a part along a chosen direction and a part perpendicular to it is called the orthogonal decomposition, and it is unique: no other pair of along-and-across pieces rebuilds b.

Why projections matter

Projection answers the question "what is the closest vector in a subspace to a given target?" The gap between the target and its projection is as small as possible and is orthogonal to the subspace. That optimality is the key to fitting models to imperfect data, the subject of the next lesson. Physics has used the same split forever: gravity pulling on a cart sitting on a ramp decomposes into a component along the ramp, which makes the cart roll, and a component perpendicular to it, which the ramp itself resists. Computing either component is one projection formula.

A related procedure, Gram-Schmidt, uses repeated projections to convert any basis into an orthonormal one by subtracting off overlaps, turning a tilted coordinate system into a clean perpendicular one. Watch it once, in full. Start with the basis v1 = [1 ; 1], v2 = [1 ; 0], which is not orthogonal since v1 . v2 = 1.

1. Keep the first vector: u1 = v1 = [1 ; 1].
2. Remove from v2 its shadow along u1: the coefficient is (u1 . v2)/(u1 . u1) = 1/2, so u2 = v2 - (1/2)u1 = [1 - 0.5 ; 0 - 0.5] = [0.5 ; -0.5].
3. Check: u1 . u2 = 0.5 - 0.5 = 0. Orthogonal.
4. Normalize both: ||u1|| = sqrt(2) gives q1 = [1/sqrt(2) ; 1/sqrt(2)], and ||u2|| = sqrt(0.5) gives q2 = [1/sqrt(2) ; -1/sqrt(2)].

The result is an orthonormal basis spanning the same plane. The recipe scales to any number of vectors: each new vector surrenders its shadows on all the directions already built, and what survives is the genuinely new perpendicular part. Numerical software runs this procedure, in a stabilized form called the QR factorization, as a workhorse for solving the least-squares problems of the next lesson.

Orthogonal matrices: rigid motions in matrix form

Pack an orthonormal set as the columns of a matrix Q and something clean happens: Q^T Q = I, because each dot product of columns lands a 1 on the diagonal (unit lengths) and 0 elsewhere (perpendicularity). Such a Q is called an orthogonal matrix, and its inverse is free of charge: Q^(-1) = Q^T. Rotations and reflections are the prime examples; check the rotation [0.6 -0.8 ; 0.8 0.6]: column lengths sqrt(0.36 + 0.64) = 1 each, and their dot product 0.6*(-0.8) + 0.8*0.6 = 0.

See the preservation concretely: apply Q = [0.6 -0.8 ; 0.8 0.6] to [3 ; 4]. The output is [0.6*3 - 0.8*4 ; 0.8*3 + 0.6*4] = [-1.4 ; 4.8], whose squared length is 1.96 + 23.04 = 25: still length 5. The arrow turned; nothing stretched.

Multiplying by an orthogonal matrix preserves every length and every angle, which is why graphics engines and physics simulations love them: a rotation may spin a model, but it never distorts it. And when a formula needs an inverse, the transpose stands in at no computational cost. That bargain matters at scale: inverting a large matrix is expensive and error-prone, while transposing is free, so algorithms are often designed to route every step they can through orthogonal matrices.

A real use: compression by projection

JPEG images and MP3 audio both rest on the coordinate formula of this lesson. The encoder holds a fixed orthogonal basis of patterns: smooth waves, medium wiggles, fine ripples (cosine patterns, in practice). A block of image or sound is a vector, and its coordinate along each pattern is a single dot product, exactly the (u . b)/(u . u) recipe, cheap precisely because the basis is orthogonal. Most photos and songs concentrate their energy in the smooth patterns, so the encoder keeps those coordinates, rounds the tiny high-frequency ones toward zero, and stores a fraction of the numbers.

Decoding is reassembly: multiply each kept coordinate by its basis pattern and add. The picture survives because what was discarded was a nearly invisible perpendicular component.

Common wrong turns

• Dividing by the length instead of the squared length. The denominator is a . a, which is ||a||^2. Writing (a . b)/||a|| gives a scalar with the wrong size.
• Treating projection as symmetric. Projecting [3 ; 4] onto [1 ; 0] gives [3 ; 0]; projecting [1 ; 0] onto [3 ; 4] gives (3/25)[3 ; 4] = [0.36 ; 0.48]. Order matters: one vector supplies the direction, the other casts the shadow.
• Reporting the scalar as the projection. The coefficient (a . b)/(a . a) is a number; the projection is that number times a, a vector.
• Claiming independent vectors must be orthogonal. [1 ; 0] and [1 ; 1] are independent yet not perpendicular. The true implication runs the other way: orthogonal nonzero vectors are always independent.
• Calling a set orthonormal after checking only angles. Orthonormal requires unit lengths too; {[1 ; 1], [1 ; -1]} is orthogonal but not orthonormal.

Try it

Project b = [5 ; 1] onto a = [2 ; 1], find the residual, and verify the residual is orthogonal to a.

Worked answer. Numerator: a . b = 2*5 + 1*1 = 11. Denominator: a . a = 4 + 1 = 5. Projection: (11/5)[2 ; 1] = [4.4 ; 2.2]. Residual: b - [4.4 ; 2.2] = [0.6 ; -1.2]. Orthogonality check: [0.6 ; -1.2] . [2 ; 1] = 1.2 - 1.2 = 0. The shadow along a plus a perpendicular leftover rebuilds b exactly.

Recap

• Orthogonal sets have pairwise zero dot products; orthonormal adds unit lengths, and coordinates become dot products.
• proj_a(b) = ((a . b)/(a . a)) a is the closest point on the line through a, and the residual is perpendicular.
• Gram-Schmidt subtracts shadows to orthonormalize any basis; orthogonal matrices satisfy Q^(-1) = Q^T and preserve shape.
• Projection is the mathematics of best approximation, and compression is projection with small parts discarded.

The final lesson points this machinery at data: when no exact solution exists, project onto what is reachable and take the closest thing, the method of least squares.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 6.1: Dot products and orthogonality. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Margalit, D., & Rabinoff, J. (n.d.). 6.2: Orthogonal complements. In Interactive linear algebra. LibreTexts. math.libretexts.org
3. Margalit, D., & Rabinoff, J. (n.d.). 6.3: Orthogonal projection. In Interactive linear algebra. LibreTexts. math.libretexts.org
4. Kuttler, K. (n.d.). 4.11: Orthogonality. In A first course in linear algebra. LibreTexts. math.libretexts.org
5. O'Connor, J. J., & Robertson, E. F. (n.d.). Jorgen Gram (1850-1916). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
6. O'Connor, J. J., & Robertson, E. F. (n.d.). Erhard Schmidt (1876-1959). In MacTutor history of mathematics archive. University of St Andrews. mathshistory.st-andrews.ac.uk
7. Boyd, S., & Vandenberghe, L. (n.d.). Introduction to applied linear algebra: Vectors, matrices, and least squares. Stanford University. web.stanford.edu

Real data rarely fits a model exactly, so the system Ax = b is often inconsistent, with no exact solution: three data points, two parameters, and no line through all three. Linear algebra offers the next best thing: the vector x that comes as close as possible. This is the least-squares method, it is projection in action, and it is running somewhere near you right now, in every trendline, forecast, and fitted model.

The least-squares idea

If Ax = b has no solution, then b lies outside the column space of A: the machine cannot manufacture it. The best we can do is replace b by its projection onto the column space, the closest reachable vector, and solve exactly for that. The resulting x-hat (read: x hat, the hat marking an estimate) minimizes the length of the error ||Ax - b||, which is the same as minimizing the sum of squared errors, hence the name.

The last lesson showed that closest means perpendicular residual. Here that principle becomes an equation. The residual b - A x-hat must be orthogonal to every column of A, and dotting a vector with every column of A at once is exactly multiplication by A^T, the transpose (rows and columns swapped). So orthogonality says A^T (b - A x-hat) = 0. Distribute and rearrange:

A^T A x-hat = A^T b.

These are the normal equations (normal is the geometry word for perpendicular). They always have a solution, and it is the least-squares answer. Notice the sizes: if A is n x 2, then A^T A is a tiny 2 x 2, however large n grows. A million data points still reduce to a two-equation solve, which is why this method scales to real datasets.

Key idea: unable to reach b, aim for its shadow on the column space; the normal equations are the perpendicularity condition written in matrix form.

Worked example: fitting a line

Suppose we want the best-fit line y = m x + c through the three points (0, 1), (1, 2), and (2, 2). Each point demands c + m x = y, giving an overdetermined system with

A = [1 0 ; 1 1 ; 1 2], unknowns [c ; m], and b = [1 ; 2 ; 2].

The three rows say: the line should pass through all three points. It cannot (the points are not collinear), so we build the normal equations.

First the transpose, concretely: flipping A across its diagonal turns its three rows into three columns, so the 3 x 2 matrix becomes the 2 x 3 matrix A^T = [1 1 1 ; 0 1 2]. Multiply out A^T A entry by entry: top-left 1 + 1 + 1 = 3; top-right 0 + 1 + 2 = 3; bottom-left the same 3 by symmetry; bottom-right 0 + 1 + 4 = 5.

Next the right-hand side, A^T b: first entry 1 + 2 + 2 = 5, second entry 0*1 + 1*2 + 2*2 = 6. In sums language, the matrix collected the count, sum x, and sum x^2, while the right side collected sum y and sum xy. That pattern holds for any number of points, which is why calculators and spreadsheets can fit lines from five running sums without ever storing the raw data twice.

Solve [3 3 ; 3 5][c ; m] = [5 ; 6]. Replace R2 with R2 - R1: [0 2 | 1], so 2m = 1 and m = 0.5. Back-substitute: 3c + 3(0.5) = 5, so 3c = 3.5 and c = 7/6. The best-fit line is y = 0.5 x + 7/6.

Audit the fit. Predictions: at x = 0, y = 7/6, missing the data value 1 by -1/6; at x = 1, y = 10/6 = 5/3, missing 2 by +1/3; at x = 2, y = 13/6, missing 2 by -1/6. The squared misses total 1/36 + 4/36 + 1/36 = 6/36 = 1/6, and no line does better.

Orthogonality is visible too: the residual vector is r = [-1/6 ; 1/3 ; -1/6], and A^T r has entries -1/6 + 1/3 - 1/6 = 0 and 0(-1/6) + 1(1/3) + 2(-1/6) = 0. The error really is perpendicular to both columns, and the model can now predict: at x = 3, expect y = 0.5(3) + 7/6 = 8/3, about 2.67.

Key idea: build A from the model, form A^T A and A^T b, solve the small system, then read predictions off the fitted formula.

When do the normal equations pin down one answer?

The matrix A^T A is always square and symmetric, and it is invertible exactly when the columns of A are linearly independent. In the line-fitting example the columns were [1 ; 1 ; 1] and [0 ; 1 ; 2], not multiples of each other, so the solution (c, m) was unique. Dependence happens in degenerate designs, for instance fitting a line when every data point has the same x-value: a vertical stack of points cannot decide a slope, and the algebra signals it honestly with a singular A^T A.

Even then the projection of b is unique; only its description in parameters is not. Statisticians call this collinearity, and the cure is a better set of columns, meaning genuinely different measurements.

One boundary case rounds out the picture. If the system happens to be consistent after all, the projection of b is b itself, every residual is zero, and the normal equations hand back the exact solution. Least squares is a strict generalization of solving: it agrees with elimination whenever elimination has an answer, and it keeps working when elimination gives up.

Applications across fields

The tools of this course power an enormous range of technology:

• Computer graphics and games move and rotate objects with transformation matrices, exactly the ones from Module 6.
• Search engines historically ranked web pages using the dominant eigenvector of a huge link matrix, a direct use of eigenvalues.
• Data science and statistics compress and analyze data by projecting onto the directions of greatest variance, an eigenvalue technique.
• Engineering and physics model vibrations, circuits, and quantum states as eigenvalue problems, where eigenvalues are natural frequencies or energy levels.
• Machine learning trains many models by solving least-squares and related linear systems at massive scale.

Least squares itself deserves the closing spotlight, because it wears so many costumes. In statistics it is linear regression: the trendline your spreadsheet draws through a scatter plot is the solution of exactly the normal equations you computed by hand above, with columns for whatever explanatory variables you include. Predicting house prices from size, age, and distance to downtown means an A with a column of 1s and three feature columns; the normal equations become 4 x 4, still tiny, and the fitted coefficients report how much each feature moves the price.

In navigation, a GPS receiver hears slightly inconsistent distance equations from several satellites (clocks jitter, signals bend) and takes the least-squares position, the point that disagrees least with all of them; a few extra satellites overdetermine the fix and the redundancy improves it. In science, calibration curves, reaction-rate fits, and astronomical orbit determinations are all the same computation; Gauss invented the method around 1800 to recover the orbit of the lost asteroid Ceres from a handful of imperfect telescope sightings, and it worked.

Modern machine learning scales the idea rather than replacing it: a neural network minimizes a sum of errors over millions of examples, often by iterative methods rather than one matrix solve, but the criterion, make the total squared miss small, and the geometry, project the data onto what the model can express, are the ones you now own. When a course on neural networks later mentions a loss function or a residual, you will recognize this lesson wearing a new vocabulary.

Common wrong turns

• Reaching for A^(-1). A 3 x 2 matrix has no inverse, so x = A^(-1) b is meaningless for overdetermined systems. The invertible object is A^T A.
• Transposing one side only. The normal equations apply A^T to both sides: A^T A x-hat = A^T b. Writing A^T A x-hat = b mismatches sizes and answers.
• Expecting the line to hit the points. The fit minimizes total squared vertical miss; usually no residual is zero. A perfect hit happens only when the system was consistent, in which case least squares quietly returns the exact solution.
• Reading the errors as perpendicular distances. Standard least squares measures vertical gaps (in y), not shortest distances to the line; fitting perpendicular offsets is a different, harder problem.
• Forgetting the independence check. If the columns of A are dependent, A^T A is singular and parameters are not unique; the model needs better-chosen columns.

Try it

Find the least-squares line y = m x + c through (0, 0), (1, 1), and (2, 3).

Worked answer. Here A = [1 0 ; 1 1 ; 1 2] and b = [0 ; 1 ; 3]. Then A^T A = [3 3 ; 3 5] as before, and A^T b = [0 + 1 + 3 ; 0*0 + 1*1 + 2*3] = [4 ; 7]. Solve [3 3 ; 3 5][c ; m] = [4 ; 7]: replace R2 with R2 - R1 to get [0 2 | 3], so m = 1.5; then 3c + 3(1.5) = 4 gives 3c = -0.5, so c = -1/6. The line is y = 1.5 x - 1/6.

Residuals: 0 - (-1/6) = 1/6, then 1 - 4/3 = -1/3, then 3 - 17/6 = 1/6; they sum to zero and their x-weighted sum 0(1/6) + 1(-1/3) + 2(1/6) = 0, confirming perpendicularity to both columns.

Recap

• Inconsistent systems get closest-fit solutions: project b onto the column space and solve for the projection.
• The normal equations A^T A x-hat = A^T b encode the perpendicular-residual condition and stay small even for big data.
• Fitting a line is the flagship case: A holds a column of 1s and a column of x-values, and the solve is 2 x 2.
• Unique parameters require independent columns; regression, GPS, and model training are this one computation at scale.

Every one of these applications rests on the same handful of ideas you have now learned: systems and elimination, matrices and determinants, vector spaces and bases, eigenvalues, and orthogonality. Linear algebra is the quiet engine beneath modern computation, and now you have seen how it runs.

Sources

1. Margalit, D., & Rabinoff, J. (n.d.). 6.5: The method of least squares. In Interactive linear algebra. LibreTexts. math.libretexts.org
2. Boyd, S., & Vandenberghe, L. (n.d.). Introduction to applied linear algebra: Vectors, matrices, and least squares. Stanford University. web.stanford.edu
3. Golub, G. H., & Reinsch, C. (1970). Singular value decomposition and least squares solutions. Numerische Mathematik, 14(5), 403-420. doi.org/10.1007/BF02163027
4. Kalman, D. (1996). A singularly valuable decomposition: The SVD of a matrix. The College Mathematics Journal, 27(1), 2-23. doi.org/10.2307/2687269
5. Leontief, W. W. (1936). Quantitative input and output relations in the economic systems of the United States. The Review of Economics and Statistics, 18(3), 105. doi.org/10.2307/1927837
6. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71-86. doi:10.1162/jocn.1991.3.1.71 find source ↗
7. Strang, G. (n.d.). 18.06 Linear algebra [Course materials, Spring 2010]. MIT OpenCourseWare. ocw.mit.edu