pdfcoffee

Chapter 13
Figure 1: Trade-offs for various quantized CNN models
FlatBuffers
FlatBuffers (https://google.github.io/flatbuffers/) is an open source
format optimized to serialize data on mobile and embedded devices. The format
was originally created at Google for game development and other performancecritical
applications. FlatBuffers supports access to serialized data without parsing/
unpacking for fast processing. The format is designed for memory efficiency
and speed by avoiding unnecessary multiple copies in memory. FlatBuffers
works across multiple platforms and languages such as C++, C#, C, Go, Java,
JavaScript, Lobster, Lua, TypeScript, PHP, Python, and Rust.
Mobile converter
A model generated with TensorFlow needs to be converted into a TensorFlow
Lite model. The converter can introduce optimizations for improving the binary
size and performance. For instance, the converter can trim away all the nodes in
a computational graph that are not directly related to inference, but instead were
needed for training.
Mobile optimized interpreter
TensorFlow Lite runs on a highly optimized interpreter that is used to optimize the
underlying computational graphs (see Chapter 2, TensorFlow 1.x and 2.x), which in
turn are used to describe the machine learning models. Internally, the interpreter
uses multiple techniques to optimize the computational graph by inducing a static
graph order and by ensuring better memory allocation. The Interpreter Core takes
~100 kb alone or ~300 kb with all supported kernels.
[ 463 ]