pdfcoffee

Chapter 8
Next we define our model. As you can see, the model is fairly straightforward, each
input sentence is a sequence of integers of size max_seqlen (64). This is input into
an Embedding layer that converts each word into a vector given by the size of the
vocabulary + 1. The additional word is to account for the padding integer 0 that was
introduced during the pad_sequences() call above. The vector at each of the 64
time steps are then fed into a bidirectional LSTM layer, which coverts each word to
a vector of size (64,). The output of the LSTM at each time step is fed into a Dense
layer, which produces a vector of size (64,) with ReLU activation. The output of this
Dense layer is then fed into another Dense layer, which outputs a vector of (1,) at
each time step, modulated through a sigmoid activation.
The model is compiled with the binary cross-entropy loss function and the Adam
optimizer, and then trained over 10 epochs:
class SentimentAnalysisModel(tf.keras.Model):
def __init__(self, vocab_size, max_seqlen, **kwargs):
super(SentimentAnalysisModel, self).__init__(**kwargs)
self.embedding = tf.keras.layers.Embedding(
vocab_size, max_seqlen)
self.bilstm = tf.keras.layers.Bidirectional(
tf.keras.layers.LSTM(max_seqlen)
)
self.dense = tf.keras.layers.Dense(64, activation="relu")
self.out = tf.keras.layers.Dense(1, activation="sigmoid")
def call(self, x):
x = self.embedding(x)
x = self.bilstm(x)
x = self.dense(x)
x = self.out(x)
return x
model = SentimentAnalysisModel(vocab_size+1, max_seqlen)
model.build(input_shape=(batch_size, max_seqlen))
model.summary()
# compile
model.compile(
loss="binary_crossentropy",
optimizer="adam",
metrics=["accuracy"]
)
# train
[ 303 ]