In this article, you will learn how to build an end-to-end sentiment analysis pipeline using Scikit-LLM and open-source large language models served through the Groq API.
Topics we will cover include:
- How Scikit-LLM bridges classical scikit-learn pipelines with modern large language model API calls.
- How to set up Scikit-LLM with a Groq backend and prepare the IMDB Movie Reviews dataset for inference.
- How to build, run, and evaluate a zero-shot sentiment classification pipeline using scikit-learn-compatible syntax.
Building an End-to-End Sentiment Analysis Pipeline with Scikit-LLM
Introduction
Traditional machine learning pipelines for predictive tasks like text classification usually rely on extracting structured, numerical features from raw text — for instance, TF-IDF frequencies or token embeddings — to feed into classical models such as logistic regression, ensembles, or support vector machines.
With the rise of large language models (LLMs), the rules of the game have somewhat changed: it is now possible to leverage zero-shot or few-shot reasoning on existing, pre-trained models for language tasks as part of a machine learning framework. Scikit-LLM is a Python library that addresses this: it bridges the gap between classical machine learning and modern LLM API calls. In this article, we will use Scikit-LLM alongside Groq backend models to build an end-to-end pipeline for sentiment analysis (a domain-specific form of text classification), achieving reasonably fast inference results with open-source models. From preprocessing to inference, we will use a large, realistically-sized dataset — the IMDB movie reviews dataset.
Prerequisites, Setup, and Obtaining the Dataset
To make the code shown in this tutorial work, you’ll need to have installed the Scikit-LLM library:
Once installed, the first step is to set it up and configure API credentials. In other words, we will need to “connect” Scikit-LLM to an endpoint — namely an LLM API repository like Groq. Make sure you register on Groq and generate an API key here: you’ll need to copy and paste it in the code below:
|
from skllm.config import SKLLMConfig
# 1. Pointing to a Groq’s compatible endpoint SKLLMConfig.set_gpt_url(“https://api.groq.com/openai/v1”)
# 2. Set your free Groq API key # Get yours at https://console.groq.com/keys SKLLMConfig.set_openai_key(“YOUR-API-KEY-GOES-HERE”) |
Scikit-LLM uses an endpoint function, set_gpt_url, that is compatible with OpenAI by default; we have routed it to make internal requests to a custom Groq URL: https://api.groq.com/openai/v1.
The next stage of the process is importing the IMDB Movie Reviews dataset — which has about 50K instances — and preparing it for the sentiment analysis pipeline we will build. Instances consist of a text review labeled with a sentiment, which can be positive or negative (this is a binary classification problem, solvable with models like logistic regression, for instance).
For convenience, we read the dataset from a publicly available GitHub repository version in CSV format:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
import pandas as pd from sklearn.model_selection import train_test_split
# Fetching a large, realistic-sized dataset (IMDB Movie Reviews – 50,000 rows) # We will read the data from a public raw CSV for convenience url = “https://raw.githubusercontent.com/Ankit152/IMDB-sentiment-analysis/master/IMDB-Dataset.csv” print(“Downloading dataset…”) df = pd.read_csv(url)
print(f“Total dataset size: {df.shape[0]} rows”)
# In a realistic LLM pipeline using a free-tier API, sending 50,000 requests # will likely trigger quota limits. Thus, we will use 500 rows for demonstrating our pipeline execution. # Feel free to use more data if you have paid API access. df_sampled = df.sample(n=500, random_state=42)
# The IMDB dataset contains HTML tags and formatting noise: that’s perfect for testing our cleaner X = df_sampled[“review”] y = df_sampled[“sentiment”] # Labels are ‘positive’ or ‘negative’
# Splitting into training (for initializing zero-shot labels) and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) |
Note that we fetched 500 rows only for demonstration purposes, as otherwise inference may take long without sufficient computing resources. You can freely change this sample size, n=500, to adapt it to your own needs.
Building the Sentiment Analysis Pipeline
Here comes the most interesting part of the process! A data science pipeline boils down to a series of preprocessing, cleaning, and data preparation steps followed by model setup or training, inference, and evaluation. For a predictive, text-based scenario like ours, preprocessing typically entails cleaning and normalizing the text. Scikit-learn provides an elegant class, FunctionTransformer, to define and encapsulate preprocessing steps based on a custom function:
|
from sklearn.preprocessing import FunctionTransformer
def clean_text_data(texts): “”“Cleans raw text inputs by removing HTML tags and stripping whitespace.”“” series = pd.Series(texts).astype(str) # Remove HTML tags like <br /> cleaned = series.str.replace(r‘<[^>]+>’, ‘ ‘, regex=True) # Remove extra spaces cleaned = cleaned.str.strip().str.replace(r‘\s+’, ‘ ‘, regex=True) return cleaned.tolist()
# Wrapping the cleaning function to enable its use inside a Pipeline object text_cleaner = FunctionTransformer(clean_text_data) |
Now we put together this preprocessing object with a model instance to create the Pipeline. Once defined, this pipeline orchestrates the whole process of preparing the data and passing it to the model at both training and inference stages — even though we use the term “training”, no actual weight-based training will occur, as we are utilizing a pre-trained model from Groq for zero-shot classification. Fitting the model only involves passing it the classification labels to use.
|
from sklearn.pipeline import Pipeline from skllm.models.gpt.classification.zero_shot import ZeroShotGPTClassifier
# Define the end-to-end pipeline sentiment_pipeline = Pipeline([ (“cleaner”, text_cleaner), # Updated to use Groq’s active Llama 3.1 8B model (“llm_classifier”, ZeroShotGPTClassifier(model=“custom_url::llama-3.1-8b-instant”)) ])
# Fit the pipeline # Note: For Zero-Shot classification, fit() doesn’t train the LLM. # It simply registers the unique labels present in ‘y_train’ (positive, negative). print(“Fitting the pipeline…”) sentiment_pipeline.fit(X_train, y_train) |
Once we have run the pipeline to “fit” the model, we use it once more for inference. Both steps use familiar scikit-learn syntax. Besides evaluating the model pipeline’s performance, we also display a few example predictions:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
from sklearn.metrics import classification_report
print(f“Running predictions on {len(X_test)} test samples…”) # Run predictions through the pipeline predictions = sentiment_pipeline.predict(X_test)
# Evaluate the pipeline’s performance on the realistic data print(“\n— Classification Report —“) print(classification_report(y_test, predictions))
# Display a few side-by-side examples print(“\n— Sample Predictions —“) for review, actual, predicted in zip(X_test[:3], y_test[:3], predictions[:3]): # Truncate review for display purposes short_review = review[:100] + “…” print(f“Review: {short_review}”) print(f“Actual: {actual} | Predicted: {predicted}\n”) |
Here’s the detailed output — execution of the above code may take a few minutes to complete:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
—– Classification Report —– precision recall f1–score support
negative 0.95 0.97 0.96 60 positive 0.95 0.93 0.94 40
accuracy 0.95 100 macro avg 0.95 0.95 0.95 100 weighted avg 0.95 0.95 0.95 100
—– Sample Predictions —– Review: I saw mommy...well, she wasn‘t exactly kissing Santa Clause; he has his hand on her thigh and wicked... Actual: negative | Predicted: negative
Review: This entry is certainly interesting for series fans (like myself), but yet it is mostly incomprehens... Actual: negative | Predicted: negative
Review: Ingrid Bergman (Cleo Dulaine) has never been so beautiful. Gary Cooper as “Cleent” so perfectly cast... Actual: positive | Predicted: positive |
Our pipeline is doing a solid job at classifying sentiment in reviews. Well done!
Wrapping Up
This article walked you through defining an end-to-end pipeline for sentiment classification using Scikit-LLM and freely available, pre-trained LLMs from API endpoints like Groq. This is a versatile approach to using classic scikit-learn syntax in novel, LLM-driven machine learning applications.