pdfcoffee

Unsupervised Learning
PCA reduces the n–dimensional input data to r–dimensional input data, where
r<n. In the most simple terms, PCA involves translating the origin and performing
rotation of the axis such that one of the axes (principal axis) has the highest variance
with data points. A reduced-dimensions dataset is obtained from the original dataset
by performing this transformation and then dropping (removing) the orthogonal
axes with low variance. Here we employ the SVD method for PCA dimensionality
reduction. Consider X, the n-dimensional data with p points that is, X is a matrix
of size p × n. From linear algebra we know that any real matrix can be decomposed
using singular value decomposition:
XX = UUΣVV TT
Where U and V are orthonormal matrices (that is, U.UT = V.VT = 1) of size p × p
and n × n respectively. Σ is a diagonal matrix of size p × n. The U matrix is called the
left singular matrix, and V the right singular matrix, and Σ , the diagonal matrix,
contains the singular values of X as its diagonal elements. Here we assume that the
X matrix is centered. The columns of the V matrix are the principal components, and
columns of UUΣ are the data transformed by principal components.
Now to reduce the dimensions of the data from n to k (where k < n) we will select the
first k columns of U and the upper-left k × k part of Σ . The product of the two gives
us our reduced-dimensions matrix:
YY kk = UUΣ kk
The data Y thus obtained will be of reduced dimensions. Next we implement PCA
in TensorFlow 2.0.
PCA on the MNIST dataset
Let us now implement PCA in TensorFlow 2.0. We will be definitely using
TensorFlow, we will also need NumPy for some elementary matrix calculation,
and Matplotlib, Matplotlib toolkits, and Seaborn for plotting:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import seaborn as sns
[ 376 ]