Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
{'attention_mask': tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),
'input_ids': tensor([ 101, 1998, 1010, 2061, 2521, 2004, 2027,
2354, 1010, 2027, 2020, 3243, 2157, 1012, 102, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),
'labels': tensor(0)}
Much better! We’re done with our datasets and can move on to the…
Trainer
Even though every pre-trained model on HuggingFace can be fine-tuned in native
PyTorch, as we did in the previous section, the library offers an easy-to-use
interface for training and evaluation: Trainer.
As expected, it takes a model and a training dataset as required arguments, and
that’s it:
from transformers import Trainer
trainer = Trainer(model=bert_cls,
train_dataset=tokenized_train_dataset)
We only need to call the train() method, and our model will be trained! YES, this
train() method actually trains the model! Thank you, HuggingFace :-)
"Awesome! But … does it train for how many epochs? Which
optimizer and learning rate does it use for training?"
We can find it all out by looking at its TrainingArguments:
trainer.args
994 | Chapter 11: Down the Yellow Brick Rabbit Hole