www.allitebooks.com

Getting the data
The data we will use for this chapter is a set of books from Project Gutenberg at
www.gutenberg.org, which is a repository of public domain literature works.
The books I used for these experiments come from a variety of authors:
• Booth Tarkington (22 titles)
• Charles Dickens (44 titles)
• Edith Nesbit (10 titles)
• Arthur Conan Doyle (51 titles)
• Mark Twain (29 titles)
• Sir Richard Francis Burton (11 titles)
• Emile Gaboriau (10 titles)
Overall, there are 177 documents from 7 authors, giving a significant amount of
text to work with. A full list of the titles, along with download links and a script
to automatically fetch them, is given in the code bundle.
To download these books, we use the requests library to download the files into
our data directory. First, set up the data directory and ensure the following code
links to it:
Chapter 9
import os
import sys
data_folder = os.path.join(os.path.expanduser("~"), "Data", "books")
Next, run the script from the code bundle to download each of the books from
Project Gutenberg. This will place them in the appropriate subfolders of this
data folder.
To run the script, download the getdata.py script from the Chapter 9 folder
in the code bundle. Save it to your notebooks folder and enter the following into a
new cell:
!load getdata.py
Then, from inside your IPython Notebook, press Shift + Enter to run the cell.
This will load the script into the cell. Then click the code again and press
Shift + Enter to run the script itself. This will take a while, but it will print a
message to let you know it is complete.
[ 189 ]