wiki / math

linear algebra

Spans & independence, linear systems & row reduction, matrices & linear maps, determinants, eigenvalues & diagonalization.

span & bases · linear systems · matrix algebra · determinants · linear transformations · change of basis · eigenvalues & diagonalization

Linear combinations, span & bases

Vectors in \(\R^n\). Column vectors with \(n\) real components (row vectors via transpose \({}^T\)); add and scale componentwise; points are identified with vectors based at the origin. A linear combination of \(\vv{v}_1,\ldots,\vv{v}_k\) is \(c_1\vv{v}_1+\cdots+c_k\vv{v}_k\). Dot & cross products and 3D line/plane computations: see vectors.

Span. \(\operatorname{span}(\vv{v}_1,\ldots,\vv{v}_k)\) = all linear combinations. One nonzero vector: a line through the origin; two nonparallel vectors: a plane through the origin (in \(\R^3\), equation \((\vv{v}_1\times\vv{v}_2)\cdot\vv{x}=0\)); three vectors not in a common plane: all of \(\R^3\); \(\operatorname{span}(\vv{0})=\{\vv{0}\}\).

Independence. A nontrivial linear relation \(c_1\vv{v}_1+\cdots+c_k\vv{v}_k=\vv{0}\) (not all \(c_i=0\)) exists iff some \(\vv{v}_i\) is a linear combination of the others — that \(\vv{v}_i\) is redundant: dropping it leaves the span unchanged. No nontrivial relation: linearly independent.

Basis. \(n\) vectors with \(\operatorname{span}(\vv{v}_1,\ldots,\vv{v}_n)=\R^n\); every \(\vv{w}\) is then a unique linear combination of them. Standard basis \(\vv{e}_1,\ldots,\vv{e}_n\) (a single 1, rest 0). Tests: basis \(\iff\det(\vv{v}_1\ \cdots\ \vv{v}_n)\ne0\iff\operatorname{rref}(\vv{v}_1\ \cdots\ \vv{v}_n)=I\); in \(\R^2\) any nonparallel pair; in \(\R^3\), \((\vv{v}_1\times\vv{v}_2)\cdot\vv{v}_3\ne0\).

Coordinates. If \(B=(\vv{v}_1,\ldots,\vv{v}_n)\) is a basis and \(\vv{w}=q_1\vv{v}_1+\cdots+q_n\vv{v}_n\), the \(q_i\) are the coordinates \([\vv{w}]_B\) of \(\vv{w}\) with respect to \(B\) — found by solving a linear system.

Linear systems & row reduction

Three guises. A system of linear equations \(a_{i1}x_1+\cdots+a_{in}x_n=b_i\) ≡ the vector equation \(x_1\vv{v}_1+\cdots+x_n\vv{v}_n=\vv{b}\) (\(\vv{v}_j\) the columns) ≡ the matrix equation \(A\vv{x}=\vv{b}\); condense as the augmented matrix \([A\,|\,\vv{b}]\).

Elementary row operations. Add a multiple of one row to another; multiply a row by a nonzero constant; swap two rows. Each is reversible, so the solution set is exactly preserved (unlike, say, squaring an equation).

Gauss–Jordan. Row by row: find the leading (leftmost nonzero) coefficient, divide the row to make it a leading 1, clear the rest of its column; finally order rows so leading 1s step right and zero rows sit at the bottom. The left block becomes \(\operatorname{rref}(A)\) — determined by \(A\) alone, independent of \(\vv{b}\).

Reading off solutions. Leading 1 in every column (\(\operatorname{rref}(A)=I\)): unique solution. A row reading \(0=c\ne0\): inconsistent, no solution. Columns without a leading 1: free variables — assign them arbitrarily and solve for the rest; infinitely many solutions (a parameterized family).

Homogeneous systems. \(A\vv{x}=\vv{0}\) always has \(\vv{x}=\vv{0}\); a free variable yields a nontrivial relation among the columns, so the columns are not a basis.

Matrix algebra

Operations. \(A+B\) entrywise (same size); \(kA\) entrywise; transpose \(A^T\) swaps rows and columns, \((AB)^T=B^TA^T\).

Matrix × vector. Two equivalent views: \(A\vv{x}=x_1\vv{v}_1+\cdots+x_n\vv{v}_n\), a linear combination of the columns of \(A\) with coefficients \(x_i\); or entry \(i\) = (row \(i\))\(\,\cdot\,\vv{x}\). In particular \(\vv{v}^T\vv{w}=\vv{v}\cdot\vv{w}\).

Matrix product. \((m\times n)(n\times p)=m\times p\), defined only when the inner sizes match; entry \(ij\) = (row \(i\) of \(A\)) \(\cdot\) (column \(j\) of \(B\)); columnwise \(AB=(A\vv{u}_1\ \cdots\ A\vv{u}_p)\) for columns \(\vv{u}_j\) of \(B\). Associative and distributive, \(k(AB)=(kA)B=A(kB)\), but not commutative: in general \(AB\ne BA\). Zero matrix: \(A+0=A\), \(A0=0A=0\); identity \(I\) (1s on the diagonal): \(AI=IA=A\).

Inverses. \(A^{-1}A=AA^{-1}=I\); then \(A\vv{x}=\vv{b}\iff\vv{x}=A^{-1}\vv{b}\), the unique solution. Compute by row-reducing \([A\,|\,I]\to[I\,|\,A^{-1}]\). For \(2\times2\): \(\bigl(\begin{smallmatrix}a&b\\c&d\end{smallmatrix}\bigr)^{-1}=\dfrac1{ad-bc}\bigl(\begin{smallmatrix}d&-b\\-c&a\end{smallmatrix}\bigr)\), needing \(ad-bc\ne0\) (larger inverses also divide by \(\det A\), but Gauss–Jordan is more efficient). \((AB)^{-1}=B^{-1}A^{-1}\).

Invertibility. Equivalent for an \(n\times n\) matrix \(A\): \(A\) is invertible; \(A\vv{x}=\vv{b}\) has exactly one solution for every \(\vv{b}\); \(A\vv{x}=\vv{0}\) only for \(\vv{x}=\vv{0}\); \(\operatorname{rref}(A)=I\); the columns of \(A\) are a basis of \(\R^n\); \(\det A\ne0\).

Determinants

Formulas. \(\det\bigl(\begin{smallmatrix}a&b\\c&d\end{smallmatrix}\bigr)=ad-bc\); \(3\times3\) by cofactor expansion along the top row (erase the entry's row and column, alternate signs):

Geometry. \(\det(\vv{v}\ \vv{w})=\pm\operatorname{Area}\) of the parallelogram spanned; \(\det(\vv{u}\ \vv{v}\ \vv{w})=\vv{u}\cdot(\vv{v}\times\vv{w})=\pm\operatorname{Volume}\) of the parallelepiped (the triple product identity). Sign + iff the tuple is positively oriented: in 2D, \(\vv{w}\) lies counterclockwise of \(\vv{v}\); in 3D, the triple is right-handed (obeys the right-hand rule, like the standard basis).

Properties. \(\det A^T=\det A\) — rows and columns are interchangeable; swapping two rows (or columns) flips the sign, so the order of vectors matters and area/volume formulas need the absolute value; \(\det A\ne0\iff A\) invertible \(\iff\) columns a basis.

Linear transformations

Matrices as maps. \(\vv{x}\mapsto A\vv{x}\) transforms space; \(A\vv{e}_i\) is the \(i\)th column, so the columns are the images of the standard basis and \(A\vv{x}=x_1(A\vv{e}_1)+\cdots+x_n(A\vv{e}_n)\): the unit grid maps to the grid generated by the columns — grid lines stay straight, parallel and evenly spaced, and the origin stays fixed.

Linearity. \(A(c_1\vv{v}_1+\cdots+c_k\vv{v}_k)=c_1A\vv{v}_1+\cdots+c_kA\vv{v}_k\) — the defining property; transformations without it (e.g. componentwise squaring) warp the grid and are nonlinear.

Composition. Apply \(B\), then \(A\): \(A(B\vv{x})=(AB)\vv{x}\) — matrix multiplication is composition of maps (which is why it is defined as it is); \(A^{-1}\) is the inverse map, undoing \(A\).

Matrix from a basis. Knowing the images of any basis determines the matrix: \(F(\vv{v}_1\ \cdots\ \vv{v}_n)=(F\vv{v}_1\ \cdots\ F\vv{v}_n)\), so \(F=(F\vv{v}_1\ \cdots\ F\vv{v}_n)(\vv{v}_1\ \cdots\ \vv{v}_n)^{-1}\).

Rotations & reflections in 2D. Counterclockwise rotation by \(\theta\), and reflection across the line at angle \(\theta\):

\(R_\theta R_\varphi=R_{\theta+\varphi}\) (≡ the angle-addition formulas), \(R_\theta^{-1}=R_{-\theta}\); \(F_\theta^2=I\), \(F_\theta^{-1}=F_\theta\); the product of two reflections is a rotation.

Projections. Onto the line spanned by a unit vector \(\uvec{v}\): \(P=\uvec{v}\uvec{v}^T\), \(P\vv{x}=(\uvec{v}\cdot\vv{x})\uvec{v}\); \(P^2=P\) and \(P\) is not invertible (the whole grid collapses onto the line). Onto the plane with unit normal \(\uvec{n}\): \(Q=I-\uvec{n}\uvec{n}^T\).

Reflections. If \(P\) projects onto a line or plane, the reflection across it is \(F=2P-I\) (the projection is the midpoint of \(\vv{x}\) and \(F\vv{x}\)); \(F^2=I\). Across a plane with unit normal \(\uvec{n}\): \(F=I-2\uvec{n}\uvec{n}^T\).

Orthogonal matrices. Columns form an orthonormal basis \(\iff A^TA=I\iff A^{-1}=A^T\). They preserve dot products, \((A\vv{x})\cdot(A\vv{y})=\vv{x}\cdot\vv{y}\), hence all lengths and angles; \(\det A=\pm1\); products of orthogonal matrices are orthogonal. Rotations and reflections are orthogonal.

Rotation about an axis (3D). About the \(x\)-axis: \(R=\Bigl(\begin{smallmatrix}1&0&0\\0&\cos\theta&-\sin\theta\\0&\sin\theta&\cos\theta\end{smallmatrix}\Bigr)\). About a unit axis \(\uvec{a}\): complete a right-handed orthonormal basis \((\uvec{a},\uvec{b},\uvec{c})\) with \(\uvec{c}=\uvec{a}\times\uvec{b}\); then the rotation is \(\tilde R=CRC^{-1}\), where \(C=(\uvec{a}\ \uvec{b}\ \uvec{c})\) and \(C^{-1}=C^T\).

Change of basis

Similar matrices. \(A=C\tilde AC^{-1}\) with \(C\) invertible. Taking \(C=(\vv{v}_1\ \cdots\ \vv{v}_n)\) for a basis \(B\) (so \(C\vv{e}_i=\vv{v}_i\)), \(\tilde A=M_B(A)\) is the matrix of \(A\) with respect to \(B\): \(A\) transforms the \(\vv{v}_i\) exactly as \(M_B(A)\) transforms the \(\vv{e}_i\). Careful — coefficients land transposed: the expansion \(A\vv{v}_j=\sum_i\tilde a_{ij}\vv{v}_i\) fills column \(j\).

Coordinates transform. \([\vv{w}]_B=C^{-1}\vv{w}\) and \([A\vv{w}]_B=M_B(A)[\vv{w}]_B\). Read \(A=CM_B(A)C^{-1}\) right to left: convert to \(B\)-coordinates, act there, convert back — pick \(B\) adapted to the map (mirror line, rotation axis, eigenvectors) so \(M_B(A)\) is simple.

Eigenvalues & diagonalization

Definition. \(A\vv{v}=\lambda\vv{v}\) with \(\vv{v}\ne\vv{0}\): eigenvector \(\vv{v}\) with eigenvalue \(\lambda\) — \(A\vv{v}\) is parallel to \(\vv{v}\). Eigenvectors come in whole lines (any \(t\vv{v}\), \(t\ne0\), has the same eigenvalue); \(\lambda=0\) eigenvectors (null vectors) exist iff \(A\) is not invertible.

Characteristic equation. Nontrivial solutions of \((A-\lambda I)\vv{x}=\vv{0}\) require \(\det(A-\lambda I)=0\); the left side is the characteristic polynomial, degree \(n\), whose roots are the eigenvalues. For each root solve \((A-\lambda I)\vv{x}=\vv{0}\) for the eigenvectors — an eigenspace (line, plane, …). The product of the eigenvalues equals \(\det A\).

Diagonalization. An eigenbasis (a basis of eigenvectors) diagonalizes:

Distinct eigenvalues give independent eigenvectors, hence an eigenbasis.

Powers & functions. \(D^n=\operatorname{diag}(\lambda_i^n)\), so \(A^n=CD^nC^{-1}\) — matrix powers without pattern-hunting. Likewise roots (\(G=CD^{1/3}C^{-1}\Rightarrow G^3=A\)) and the matrix exponential \(e^A=\sum_{n=0}^\infty A^n/n!=Ce^DC^{-1}\) with \(e^D=\operatorname{diag}(e^{\lambda_i})\).

When it fails. \(R_\theta\) (\(0\lt\theta\lt\pi\)): characteristic roots are complex — no real eigenvectors. Shear \(\bigl(\begin{smallmatrix}1&1\\0&1\end{smallmatrix}\bigr)\): repeated root \(\lambda=1\) with only one line of eigenvectors — not diagonalizable. A repeated (degenerate) eigenvalue may or may not carry enough eigenvectors.

Complex eigenvalues. For a real matrix, complex eigenvalues and eigenvectors come in conjugate pairs (\(\bar\lambda\), \(\bar{\vv{v}}\)); with distinct roots, \(A=CDC^{-1}\) still holds over \(\C\). A real \(2\times2\) with eigenvalue \(\lambda=p+iq\) is similar to the rotation–scaling matrix \(\bigl(\begin{smallmatrix}p&-q\\q&p\end{smallmatrix}\bigr)\): scale by \(|\lambda|\), rotate by \(\arg\lambda\). A 3D rotation by \(\theta\) about \(\uvec{a}\) has eigenvalues \(1,\,e^{i\theta},\,e^{-i\theta}\) (eigenvector \(\uvec{a}\) for \(1\)).

Spectral theorem. Symmetric \(A^T=A\): there is always a real orthonormal eigenbasis — \(A=UDU^T\) with \(U\) orthogonal, no inverse to compute. Eigenvectors for distinct eigenvalues are automatically orthogonal; within a repeated eigenvalue's eigenspace choose an orthonormal set (cross products help in 3D).