pdfcoffee

Chapter 7
$ python run_classifier.py \
--task_name=COLA|MRPC \
--do_train=true \
--do_eval=true \
--do_predict=true \
--data_dir=${CLASSIFIER_DATA} \
--vocab_file=${BERT_BASE_DIR}/vocab.txt \
--bert_config_file=${BERT_BASE_DIR}/bert_config_file.json \
--init_checkpoint=${BERT_BASE_DIR}/bert_model.ckpt \
--max_seq_length=128 \
--train_batch_size=8 \
--learning_rate=2e-5 \
--num_train_epochs=2.0 \
--output_dir=${TRAINED_CLASSIFIER}
To predict only using a trained network, turn the --do_train and --do_eval flags
to false.
Using BERT as part of your own network
Currently BERT is available on TensorFlow Hub as an estimator, but at the moment
it is not fully compliant with TensorFlow 2.x, in the sense that it is not yet callable
as a tf.hub.KerasLayer. Meanwhile, Zweig demonstrates how to include BERT
in your Keras/TensorFlow 1.x-based network in his blog post [35].
The more popular way to use BERT in your own TensorFlow 2.x code is via the
HuggingFace Transformers library. This library provides convenience classes for
various popular Transformer architectures such as BERT, as well as convenience
classes for fine-tuning on several downstream tasks. It was originally written
for PyTorch, but has since been extended with convenience classes callable from
TensorFlow as well. However, in order to use this library, you must have PyTorch
installed as well.
The Transformers library provides the following classes:
1. A set of Transformer classes for 10 (at the time of writing) different
Transformer architectures that can be instantiated from PyTorch client code.
The naming convention is to append "Model" to the name of the architecture,
for example, BertModel, XLNetModel, and so on. There is also a corresponding
set of classes that can be instantiated from TensorFlow 2.x code; these are
prefixed by "TF", for example, TFBertModel, TFXLNetModel, and so on.
[ 271 ]