www.allitebooks.com

Predicting Sports Winners with Decision Trees
These integers can be fed into the decision tree, but they will still be interpreted
as continuous features by DecisionTreeClassifier. For example, teams may be
allocated integers 0 to 16. The algorithm will see teams 1 and 2 as being similar,
while teams 4 and 10 will be different—but this makes no sense as all. All of the
teams are different from each other—two teams are either the same or they are not!
To fix this inconsistency, we use the OneHotEncoder transformer to encode these
integers into a number of binary features. Each binary feature will be a single value
for the feature. For example, if the NBA team Chicago Bulls is allocated as integer
7 by the LabelEncoder, then the seventh feature returned by the OneHotEncoder
will be a 1 if the team is Chicago Bulls and 0 for all other teams. This is done for every
possible value, resulting in a much larger dataset. The code is as follows:
from sklearn.preprocessing import OneHotEncoder
onehot = OneHotEncoder()
We fit and transform on the same dataset, saving the results:
X_teams_expanded = onehot.fit_transform(X_teams).todense()
Next, we run the decision tree as before on the new dataset:
clf = DecisionTreeClassifier(random_state=14)
scores = cross_val_score(clf, X_teams_expanded, y_true,
scoring='accuracy')
print("Accuracy: {0:.1f}%".format(np.mean(scores) * 100))
This scores an accuracy of 60 percent. The score is better than the baseline, but
not as good as before. It is possible that the larger number of features were not
handled properly by the decision trees. For this reason, we will try changing the
algorithm and see if that helps. Data mining can be an iterative process of trying new
algorithms and features.
Random forests
A single decision tree can learn quite complex functions. However, in many ways
it will be prone to overfitting—learning rules that work only for the training set.
One of the ways that we can adjust for this is to limit the number of rules that it
learns. For instance, we could limit the depth of the tree to just three layers. Such
a tree will learn the best rules for splitting the dataset at a global level, but won't
learn highly specific rules that separate the dataset into highly accurate groups.
This trade-off results in trees that may have a good generalization, but overall
slightly poorer performance.
[ 54 ]