pdfcoffee

Chapter 9
Our autoencoder model takes a sequence of GloVe word vectors and learns to
produce another sequence that is similar to the input sequence. The encoder LSTM
compresses the sequence into a fixed-size context vector, which the decoder LSTM
uses to reconstruct the original sequence. A schematic of the network is shown here:
Figure 5: Visualisation of the LSTM network
Because the input is quite large, we will use a generator to produce each batch of
input. Our generator produces batches of tensors of shape (BATCH_SIZE, SEQUENCE_
LEN, EMBED_SIZE). Here BATCH_SIZE is 64, and since we are using 50-dimensional
GloVe vectors, EMBED_SIZE is 50. We shuffle the sentences at the beginning of each
epoch and return batches of 64 sentences. Each sentence is represented as a vector
of GloVe word vectors. If a word in the vocabulary does not have a corresponding
GloVe embedding, it is represented by a zero vector. We construct two instances of
the generator, one for training data and one for test data, consisting of 70% and 30%
of the original dataset respectively:
BATCH_SIZE = 64
def sentence_generator(X, embeddings, batch_size):
while True:
# loop once per epoch
num_recs = X.shape[0]
indices = np.random.permutation(np.arange(num_recs))
num_batches = num_recs // batch_size
for bid in range(num_batches):
sids = indices[bid * batch_size : (bid + 1) * batch_size]
[ 369 ]