www.allitebooks.com

Chapter 11
This dataset comes from a popular dataset called CIFAR-10. It contains 60,000
images that are 32 pixels wide and 32 pixels high, with each pixel having a
red-green-blue (RGB) value. The dataset is already split into training and testing,
although we will not use the testing dataset until after we complete our training.
The CIFAR-10 dataset is available for download at: http://www.
cs.toronto.edu/~kriz/cifar.html. Download the python
version, which has already been converted to NumPy arrays.
Opening a new IPython Notebook, we can see what the data looks like. First, we set
up the data filenames. We will only worry about the first batch to start with, and
scale up to the full dataset size towards the end;
import os
data_folder = os.path.join(os.path.expanduser("~"), "Data", "cifar-10-
batches-py")
batch1_filename = os.path.join(data_folder, "data_batch_1")
Next, we create a function that can read the data stored in the batches. The batches
have been saved using pickle, which is a python library to save objects. Usually,
we can just call pickle.load on the file to get the object. However, there is a small
issue with this data: it was saved in Python 2, but we need to open it in Python 3. In
order to address this, we set the encoding to latin (even though we are opening it in
byte mode):
import pickle
# Bigfix thanks to: http://stackoverflow.com/questions/11305790/
pickle-incompatability-of-numpy-arrays-between-python-2-and-3
def unpickle(filename):
with open(filename, 'rb') as fo:
return pickle.load(fo, encoding='latin1')
Using this function, we can now load the batch dataset:
batch1 = unpickle(batch1_filename)
This batch is a dictionary, containing the actual data in NumPy arrays, the
corresponding labels and filenames, and finally a note to say which batch it is
(this is training batch 1 of 5, for instance).
We can extract an image by using its index in the batch's data key:
image_index = 100
image = batch1['data'][image_index]
[ 243 ]