Advanced Deep Learning with Keras

Chapter 10
Figure 10.6.4 Decoder model
As shown in Figures 10.2.1 to 10.4.1, before building the policy and value networks,
we must first create a function that converts the state to features. This function is
implemented by an encoder of an autoencoder similar to the ones implemented in
Chapter 3, Autoencoders. Figure 10.6.2 shows an autoencoder made of an encoder and
a decoder. In Figure 10.6.3, the encoder is an MLP made of Input(2)-Dense(256,
activation='relu')-Dense(128, activation='relu')-Dense(32). Every
state is converted into a 32-dim feature vector. In Figure 10.6.4, the decoder is also
an MLP but made of Input(32)-Dense(128, activation='relu')-Dense(256,
activation='relu')-Dense(2). The autoencoder is trained for 10 epochs with an
MSE, loss function, and Keras default Adam optimizer. We sampled 220,000 random
states for the train and test dataset and applied 200k/20k train-test split. After training,
the encoder weights are saved for future use in the policy and value networks training.
Listing 10.6.1 shows the methods for building and training the autoencoder.
Listing 10.6.1, policygradient-car-10.1.1.py shows us the methods for building
and training the autoencoder:
# autoencoder to convert states into features
def build_autoencoder(self):
# first build the encoder model
inputs = Input(shape=(self.state_dim, ), name='state')
feature_size = 32
x = Dense(256, activation='relu')(inputs)
x = Dense(128, activation='relu')(x)
feature = Dense(feature_size, name='feature_vector')(x)
# instantiate encoder model
[ 321 ]