www.allitebooks.com

Classifying with scikit-learn Estimators
For a mathematical-based algorithm to compare each of these features, the
differences in the scale, range, and units can be difficult to interpret. If we used
the above features in many algorithms, the weight would probably be the most
influential feature due to only the larger numbers and not anything to do with the
actual effectiveness of the feature.
One of the methods to overcome this is to use a process called preprocessing
to normalize the features so that they all have the same range, or are put into
categories like small, medium and large. Suddenly, the large difference in the
types of features has less of an impact on the algorithm, and can lead to large
increases in the accuracy.
Preprocessing can also be used to choose only the more effective features, create new
features, and so on. Preprocessing in scikit-learn is done through Transformer
objects, which take a dataset in one form and return an altered dataset after some
transformation of the data. These don't have to be numerical, as Transformers are also
used to extract features-however, in this section, we will stick with preprocessing.
An example
We can show an example of the problem by breaking the Ionosphere dataset.
While this is only an example, many real-world datasets have problems of this
form. First, we create a copy of the array so that we do not alter the original dataset:
X_broken = np.array(X)
Next, we break the dataset by dividing every second feature by 10:
X_broken[:,::2] /= 10
In theory, this should not have a great effect on the result. After all, the values
for these features are still relatively the same. The major issue is that the scale has
changed and the odd features are now larger than the even features. We can see the
effect of this by computing the accuracy:
estimator = KNeighborsClassifier()
original_scores = cross_val_score(estimator, X, y,
scoring='accuracy')
print("The original average accuracy for is
{0:.1f}%".format(np.mean(original_scores) * 100))
broken_scores = cross_val_score(estimator, X_broken, y,
scoring='accuracy')
print("The 'broken' average accuracy for is
{0:.1f}%".format(np.mean(broken_scores) * 100))
[ 36 ]