www.allitebooks.com

Chapter 3
This results in an immediate benefit of 60.6 percent, up by 0.6 points by just
swapping the classifier.
Random forests, using subsets of the features, should be able to learn more
effectively with more features than normal decision trees. We can test this by
throwing more features at the algorithm and seeing how it goes:
X_all = np.hstack([X_home_higher, X_teams])
clf = RandomForestClassifier(random_state=14)
scores = cross_val_score(clf, X_all, y_true, scoring='accuracy')
print("Accuracy: {0:.1f}%".format(np.mean(scores) * 100))
This results in 61.1 percent —even better! We can also try some other parameters
using the GridSearchCV class as we introduced in Chapter 2, Classifying with
scikit-learn Estimators:
parameter_space = {
"max_features": [2, 10, 'auto'],
"n_estimators": [100,],
"criterion": ["gini", "entropy"],
"min_samples_leaf": [2, 4, 6],
}
clf = RandomForestClassifier(random_state=14)
grid = GridSearchCV(clf, parameter_space)
grid.fit(X_all, y_true)
print("Accuracy: {0:.1f}%".format(grid.best_score_ * 100))
This has a much better accuracy of 64.2 percent!
If we wanted to see the parameters used, we can print out the best model that was
found in the grid search. The code is as follows:
print(grid.best_estimator_)
The result shows the parameters that were used in the best scoring model:
RandomForestClassifier(bootstrap=True, compute_importances=None,
criterion='entropy', max_depth=None, max_features=2,
max_leaf_nodes=None, min_density=None, min_samples_leaf=6,
min_samples_split=2, n_estimators=100, n_jobs=1,
oob_score=False, random_state=14, verbose=0)
[ 57 ]