Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
Step Training Loss Validation Loss ...
50 3.587500 3.327199 ...
TrainOutput(global_step=59, training_loss=3.5507330167091498,
metrics={'train_runtime': 22.6958, 'train_samples_per_second': 2.6,
'total_flos': 22554466320384.0, 'epoch': 0.99,
'init_mem_cpu_alloc_delta': 1316954112, 'init_mem_gpu_alloc_delta':
511148032, 'init_mem_cpu_peaked_delta': 465375232,
'init_mem_gpu_peaked_delta': 0, 'train_mem_cpu_alloc_delta':
13103104, 'train_mem_gpu_alloc_delta': 1499219456,
'train_mem_cpu_peaked_delta': 0, 'train_mem_gpu_peaked_delta':
730768896})
trainer.evaluate()
Output
{'eval_loss': 3.320632219314575,
'eval_runtime': 0.9266,
'eval_samples_per_second': 60.438,
'epoch': 0.99,
'eval_mem_cpu_alloc_delta': 151552,
'eval_mem_gpu_alloc_delta': 0,
'eval_mem_cpu_peaked_delta': 0,
'eval_mem_gpu_peaked_delta': 730768896}
There we go: GPT-2 was fine-tuned on Alice’s Adventures in Wonderland for one
epoch.
How good is it at being Lewis Carroll now? Let’s check it out!
Generating Text
The GPT-2 model has a generate() method with plenty of options for generating
text (e.g., greedy decoding, beam search, and more). We won’t be delving into these
details but going the easy way instead: assigning our fine-tuned model and pretrained
tokenizer to a pipeline and using most of its default values.
1014 | Chapter 11: Down the Yellow Brick Rabbit Hole