www.allitebooks.com

Predicting Sports Winners with Decision Trees
By averaging a large number of decision trees, this variance is greatly reduced.
This results in a model with a higher overall accuracy.
In general, ensembles work on the assumption that errors in prediction are
effectively random and that those errors are quite different from classifier to
classifier. By averaging the results across many models, these random errors are
canceled out—leaving the true prediction. We will see many more ensembles in
action throughout the rest of the book.
Parameters in Random forests
The Random forest implementation in scikit-learn is called
RandomForestClassifier, and it has a number of parameters. As Random forests
use many instances of DecisionTreeClassifier, they share many of the same
parameters such as the criterion (Gini Impurity or Entropy/Information Gain),
max_features, and min_samples_split.
Also, there are some new parameters that are used in the ensemble process:
• n_estimators: This dictates how many decision trees should be built.
A higher value will take longer to run, but will (probably) result in a
higher accuracy.
• oob_score: If true, the method is tested using samples that aren't in the
random subsamples chosen for training the decision trees.
• n_jobs: This specifies the number of cores to use when training the
decision trees in parallel.
The scikit-learn package uses a library called Joblib for in-built parallelization.
This parameter dictates how many cores to use. By default, only a single core is
used—if you have more cores, you can increase this, or set it to -1 to use all cores.
Applying Random forests
Random forests in scikit-learn use the estimator interface, allowing us to use almost
the exact same code as before to do cross fold validation:
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(random_state=14)
scores = cross_val_score(clf, X_teams, y_true, scoring='accuracy')
print("Accuracy: {0:.1f}%".format(np.mean(scores) * 100))
[ 56 ]