pdfcoffee

TensorFlow 1.x and 2.x
However, the recommendation for TensorFlow 2.x is to keep using them if you have
already adopted them, but to use tf.keras if you are starting from scratch. As of
April 2019, Estimators fully support distributed training while tf.keras has limited
support. Given this, a possible workaround is to transform tf.keras models into
Estimators with tf.keras.estimator.model_to_estimator() and then use the
full distributed training support.
Ragged tensors
Continuing our discussion on the benefits of TensorFlow 2.x, we should notice that
TensorFlow 2.x added support for "ragged" tensors, which are a special type of dense
tensor with non-uniformly shaped dimensions. This is particularly useful for dealing
with sequences and other data issues where the dimensions can change across
batches, such as text sentences and hierarchical data. Note that ragged tensors are
more efficient than padding tf.Tensor, since no time or space is wasted:
ragged = tf.ragged.constant([[1, 2, 3], [3, 4], [5, 6, 7, 8]]) ==>
<tf.RaggedTensor [[1, 2, 3], [3, 4], [5, 6, 7, 8]]>
Custom training
TensorFlow can compute gradients on our behalf (automatic differentiation) and this
makes it extremely easy to develop machine learning models. If you use tf.keras,
then you will train your model with fit() and you probably will not need to go into
the details of how the gradients are computed internally. However, custom training
is useful when you want to have finer control over optimization.
There are multiple ways of computing gradients. Let's look at them:
1. tf.GradientTape(): This class records operations for automatic
differentiation. Let's look at an example where we use the parameter
persistent=True (a Boolean controlling whether a persistent gradient tape
is created, which means that multiple calls can be made to the gradient()
method on this object):
import tensorflow as tf
x = tf.constant(4.0)
with tf.GradientTape(persistent=True) as g:
g.watch(x)
y = x * x
z = y * y
dz_dx = g.gradient(z, x) # 256.0 (4*x^3 at x = 4)
dy_dx = g.gradient(y, x) # 8.0
[ 74 ]