MACHINE LEARNING TECHNIQUES - LASA

184
9.1.2 Singular Value Decomposition (SVD)
When the eigenvectors are not linearly independent, then X does not have an inverse (it is thus
singular) and such decomposition does not exist. The eigenvalue decomposition consists then in
finding a similarity transformation such that:
A= UΛ V
(8.11)
With UV , two orthogonal (if real) or unitary (if complex) matrices and Λ a diagonal matrix. Such
decomposition is called singular value decomposition (SVD). SVD are useful in sofar that A
represents the mapping of a n-dimensional space onto itself, where n is the dimension of A.
#
An alternative to SVD is to compute the Moore-Penrose Pseudoinverse A of the non invertible
#
matrix A and then exploit the fact that, for a pair of vectors z and c, z = A c is the shortest length
least-square solution to the problem Az = c . Methods such as PCA that find the optimal (in a
least-square sense) projection of a dataset can be approximated using the pseudoinverse when
the transformation matrix is singular.
9.1.3 Frobenius Norm
The Frobenius norm of an m× nmatrix A is given by:
A
F
m
n
i= 1 j=
1
ij
2
= ∑∑ a
(8.12)
9.2 Recall of basic notions of statistics and probabilities
9.2.1 Probabilities
Consider two variables x and y taking discrete values over the intervals [x 1 ,…, x M ] and [y 1 ,…, y N ]
respectively, then P( x=
x i ) is the probability that the variable x takes value x i
, with:
i) 0 ≤ P( x= x ) ≤1, ∀ i=
1,..., M,
M
∑
i=
1
( x )
ii) P x= = 1.
i
i
( j )
The same two above properties applies to the probabilities P y= y , ∀ j=
1,... N.
Some properties follow from the above:
Let P( x= a)
be the probability that the variable x will take the value a. If P( x a) 1
= = , x is a
constant with value a. If x is an integer and can take any value a between [1, N]
∈• with equal
1
probability, then the probability that x takes value a is Px ( = a)
=
N
© A.G.Billard 2004 – Last Update March 2011