www.allitebooks.com

Chapter 5
Next, we create our transformer using the chi2 function and a SelectKBest
transformer:
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2
transformer = SelectKBest(score_func=chi2, k=3)
Running fit_transform will call fit and then transform with the same dataset.
The result will create a new dataset, choosing only the best three features.
Let's look at the code:
Xt_chi2 = transformer.fit_transform(X, y)
The resulting matrix now only contains three features. We can also get the scores
for each column, allowing us to find out which features were used. Let's look at
the code:
print(transformer.scores_)
The printed results give us these scores:
[ 8.60061182e+03 2.40142178e+03 8.21924671e+07 1.37214589e+06
6.47640900e+03]
The highest values are for the first, third, and fourth columns Correlates to the Age,
Capital-Gain, and Capital-Loss features. Based on a univariate feature selection,
these are the best features to choose.
If you'd like to find out more about the features in the Adult dataset,
take a look at the adult.names file that comes with the dataset and
the academic paper it references.
We could also implement other correlations, such as the Pearson's correlation
coefficient. This is implemented in SciPy, a library used for scientific computing
(scikit-learn uses it as a base).
If scikit-learn is working on your computer, so is SciPy. You do not
need to install anything further to get this sample working.
First, we import the pearsonr function from SciPy:
from scipy.stats import pearsonr
[ 91 ]