pdfcoffee

TensorFlow for Mobile and IoT and TensorFlow.js
TensorFlow Lite
TensorFlow Lite is a lightweight platform designed by TensorFlow. This platform
is focused on mobile and embedded devices such as Android, iOS, and Raspberry
PI. The main goal is to enable machine learning inference directly on the device
by putting a lot of effort in three main characteristics: (1) small binary and model
size to save on memory, (2) low energy consumption to save on the battery, and
(3) low latency for efficiency. It goes without saying that battery and memory are
two important resources for mobile and embedded devices. In order to achieve
these goals, Lite uses a number of techniques such as Quantization, FlatBuffers,
Mobile interpreter, and Mobile converter, which we are going to review briefly in
the following sections.
Quantization
Quantization refers to a set of techniques that constrains an input made of
continuous values (such as real numbers) into a discrete set (such as integers).
The key idea is to reduce the space occupancy of Deep Learning (DL) models by
representing the internal weight with integers instead of real numbers. Of course,
this implies trading space gains for some amount of performance of the model.
However, it has been empirically shown in many situations that a quantized model
does not suffer from a significant decay in performance. TensorFlow Lite is internally
built around a set of core operators supporting both quantized and floating-point
operations.
Model quantization is a toolkit for applying quantization. This operation is applied
to the representations of weights and, optionally, to the activations for both storage
and computation. There are two types of quantization available:
• Post-training quantization quantizes weights and the result of activations
post training.
• Quantization-aware training allows for the training of networks that can be
quantized with minimal accuracy drop (only available for specific CNNs).
Since this is a relatively experimental technique, we are not going to discuss
it in this chapter but the interested reader can find more information in [1].
TensorFlow Lite supports reducing the precision of values from full floats to halfprecision
floats (float16) or 8-bit integers. TensorFlow reports multiple trade-offs in
terms of accuracy, latency, and space for selected CNN models (see Figure 1, source:
https://www.tensorflow.org/lite/performance/model_optimization):
[ 462 ]