MACHINE LEARNING TECHNIQUES - LASA

183
( A λI) x 0
− = (8.5)
where I is the identity matrix. As shown in Cramer's rule, a linear system of equations has
nontrivial solutions iff the determinant vanishes, so the solutions of equation (8.5) are given by
( A λI)
det − = 0
(8.6)
This equation is known as the characteristic equation of A , and the left-hand side is known as the
characteristic polynomial.
For example, for a 2× 2 matrix, the eigenvalues are
1
λ ( ) ( ) 2
±
=
⎡⎡
a11 + a22 ± 4a12a21 + a11 −a
⎤⎤
22
2 ⎢⎢⎣⎣
⎥⎥⎦⎦
(8.7)
which arises as the solutions of the characteristic equation.
( ) ( )
2
x x a11 a22 a11a22 a12a21 0
− + + − = (8.8)
If all k eigenvalues are different, then plugging these back in gives k − 1 independent equations
for the k components of each corresponding eigenvector, and the system is said to be
nondegenerate. If the eigenvalues are n-fold degenerate, then the system is said to be
degenerate and the eigenvectors are not linearly independent. In such cases, the additional
constraint that the eigenvectors be orthogonal,
XX
i j
= Xi X
j
δij
(8.9)
where δij
is the Kronecker delta, can be applied to yield n additional constraints, thus allowing
solution for the eigenvectors.
If all the eigenvectors are linearly independent, then the eigenvalue decomposition
A= XΛ X −1
(8.10)
Where the columns of X are composed of the eigenvectors of A and Λ is a diagonal matrix
composed of the associated eigenvalues.
© A.G.Billard 2004 – Last Update March 2011