6 Lesser-Known Scikit-Learn Features That Will Save You Time

For many people studying data science, Scikit-Learn is often the first machine learning library they encounter. It’s because Scikit-Learn offers various APIs that are useful for model development while still being easy for beginners to use.

As helpful as they may be, many features from Scikit-Learn are rarely explored and have untapped potential. This article will explore six lesser-known features that will save you time.

1. Validation Curve

The first function we will explore is the validation curve function from Scikit-Learn. From the name, you can guess that it performs some kind of validation, but it’s not just simple validation that is performs. The function explores machine learning model performance over various values of specific hyperparameters.

Using the cross-validation method, the validation curve function evaluates training and test performance over the range of hyperparameter values. The process results in two sets of scores that we can compare visually.

Let’s try out the function with sample data and visualize the results. First, let’s load sample data and set up the hyperparameter range we want to explore. In this case, we will explore how the SVC model’s accuracy performance over various gamma hyperparameters.

Once you execute the code above, you will get two scores: train_scores & test_scores.

These are both assigned arrays of scores like below.

We can visualize the validation curve using code similar to the following.

The curve teaches us how the hyperparameters affect the model’s performance. Using the validation curve, we can find the optimal value for the hyperparameter and estimate it better than relying on the simple train-test split.

Try using a validation curve in your model development process to guide you in developing the best model possible and avoid issues such as overfitting.

2. Model Calibration

When we develop a machine learning classifier model, we need to remember that it’s not enough simply to provide correct classification prediction; the probabilities associated with the prediction must also be reliable. The process to ensure that the probabilities are reliable is called calibration.

The calibration process adjusts the model’s probability estimation. The technique pushes the probability to reflect the true likelihood of the prediction so it is not overconfident or underconfident. The uncalibrated model might predict an event with a 90% probability chance, while the actual success rate is much lower, which means the model was overconfident. That’s why we need to calibrate the model.

By calibrating the model, we could improve the trust in model prediction and inform the user of the actual estimation of the actual risk from the model.

Let’s try the calibration process with Scikit-Learn. The library offers a diagnostic function (calibration_curve) and a model calibration class (CalibratedClassifierCV).

We will use breast cancer data and the logistic regression model as the basis. Then, we will compare the original model with the calibrated model for the probability.

We can see that the calibrated logistic regression is closer to the model with perfect calibration than the original. This means that the calibrated model can better estimate the actual risk, although it is still not ideal.

Try using the calibration method to improve the model prediction capability.

3. Permutation Importance

Whenever we work with a machine learning model, we use the data features to provide the prediction result. However, not every feature contributes to the prediction in the same manner.

The permutation_importance() method is for measuring the feature contribution to model performances by randomly permuting (changing) feature values and evaluating the model performance after the permutation. If the model performance degrades, the feature impacts the model; conversely, if the model performance is unchanged, it suggests that the feature might not be that useful for the specific model performance.

The technique is straightforward and intuitive, making it helpful in interpreting any model’s internal decision-making. It’s beneficial for models with no inherent feature importance method embedded inside.

Let’s try out the model with a Python code example. We will use sample data and models similar to our previous example.

With the code above, we have the model and permutation importance result, where we will analyze the feature’s impact on the model. Let’s look at the average and standard deviation results for permutation importance.

The result is as follows.

We can also visualize the result above to understand the model performance better.

The visualization shows that the petal length most impacts the feature performance, while the sepal width has no effect. There is always uncertainty, which is represented by the standard deviation, but we can conclude with the permutation importance technique that petal length has the most impact.

That’s a simple feature importance technique that you can use for your next project.

4. Feature Hasher

Working on features for data science modeling, I often found that high-dimensional features were too memory intensive, which impacted the application’s overall performance. There are many ways to improve performance, such as dimensionality reduction or feature selection. The hashing method is another method that might be rarely used but could be helpful.

Hashing is converting data into a sparse numeric matrix with a fixed size. Applying a hash function to each feature can map the represented feature into a sparse matrix. We will use a hash function via FeatureHasher to compute the matrix column corresponding to a name.

Let’s try it out with the Python code.

You will see the output looks like the one below.

The dataset has been represented into a sparse matrix with 10 different features. You can specify the number of hash features you need to balance the memory usage and information loss.

5. Robust Scaler

Real-world data is rarely clean, and more often than not riddled with outliers. While an outlier is not intrinsically bad and might give information that contributes to the actual insight, there are times when it will skew our model results.

There are many techniques for scaling our outliers, but sometimes they can introduce bias. That’s why robust scaling is important to help preprocess our data. Robust scaling transforms the data by removing the median and scaling them according to the IQR instead of using mean and standard deviation.

The robust scaler is proper, with only a few outliers at extreme positions. By applying it, the dataset is stable and not influenced much by the outliers, which makes it useful for any machine learning model development.

Here is an example of using the robust scaler. Let’s use the Iris data example and introduce an outlier in the dataset.

The above code easily scales the data with our introduced outlier. Try using yourself if you are having a hard time with outliers.

6. Feature Union

Feature union is a Scikit-Learn feature that combines multiple feature transformations within the pipeline. Instead of transforming the same features sequentially, feature union inputs the data into several transformers simultaneously to provide all the transformed features.

It’s a helpful feature where different transformers are required to capture various aspects of data and need to be present in the dataset. One transformer might used for the PCA technique, while the others use robust scaling.

Let’s try it out with the following code below. For example, we can create transformers for both PCA and polynomial features transformers.

The example result of the transformed features is shown in the output below.

The result contains the PCA and polynomial features we have previously transformed.

Try experimenting with several transformations to see if they suit your analysis.

Conclusion

Scikit-Learn is a library that many data scientists use to develop models easily from their data. It’s easy to use and provides many features that are useful for our model work, yet many of those features seem underutilized though they could save you time.

In this article, we have explored six of these underutilized features, from validation curves to feature hashing to a robust scaler that isn’t highly influenced by outliers. Hopefully you found something new and of value herein, and I hope this has helped!