pdfcoffee

Unsupervised Learning
We define a class RBM. The class __init_() function initializes the number of
neurons in the visible layer (input_size) and the number of neurons in the hidden
layer (output_size). The function initializes the weights and biases for both hidden
and visible layers. In the following code we have initialized them to zero. You can try
with random initialization as well:
#Class that defines the behavior of the RBM
class RBM(object):
def __init__(self, input_size, output_size, lr=1.0,
batchsize=100):
"""
m: Number of neurons in visible layer
n: number of neurons in hidden layer
"""
# Defining the hyperparameters
self._input_size = input_size # Size of Visible
self._output_size = output_size # Size of outp
self.learning_rate = lr # The step used in gradient descent
self.batchsize = batchsize
# The size of how much data will be used for training
# per sub iteration
# Initializing weights and biases as matrices full of zeroes
self.w = tf.zeros([input_size, output_size], np.float32)
# Creates and initializes the weights with 0
self.hb = tf.zeros([output_size], np.float32)
# Creates and initializes the hidden biases with 0
self.vb = tf.zeros([input_size], np.float32)
# Creates and initializes the visible biases with 0
We define methods to provide the forward and backward passes:
# Forward Pass
def prob_h_given_v(self, visible, w, hb):
# Sigmoid
return tf.nn.sigmoid(tf.matmul(visible, w) + hb)
# Backward Pass
def prob_v_given_h(self, hidden, w, vb):
return tf.nn.sigmoid(tf.matmul(hidden, tf.transpose(w)) + vb)
We create a function to generate random binary values. This is so because both
hidden and visible units are updated using stochastic probability depending upon
the input to each unit in the case of the hidden layer (and top-down input to visible
layers):
[ 394 ]