IndusAI Docs
indusai.co

An end-to-end project

Lesson 62 of 62, about 8 min

This final lesson runs a complete project from raw data to a saved model, using everything from the track. The dataset is the wine quality style data built into scikit-learn (178 wines, 13 chemical measurements, 3 grape varieties). Every step is one you will repeat on real problems.

1. Frame the problem

Given 13 chemical measurements of a wine, predict which of three grape varieties it came from, so that a lab can flag mislabelled bottles. It is a multi-class classification problem. Classes are roughly balanced, so accuracy is a reasonable headline metric, and we will also look at per class recall.

2. Load and look

python
from sklearn.datasets import load_wine
import pandas as pd

data = load_wine()
df = pd.DataFrame(data.data, columns=data.feature_names)
df["target"] = data.target
print(df.shape)
print(df["target"].value_counts().sort_index())
print(df.describe().T[["mean", "std", "min", "max"]].round(2).head(6))

Features are on wildly different scales (proline in the hundreds, hue below 2), so scaling will matter for some models.

3. Split

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
print(len(X_train), "train,", len(X_test), "test")

The test set is now locked away until step 7.

4. A baseline

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.dummy import DummyClassifier

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
baseline = DummyClassifier(strategy="most_frequent")
print("baseline cv accuracy:", cross_val_score(baseline, X_train, y_train, cv=5).mean().round(3))

Anything below about 0.4 is not learning.

5. Compare models with cross-validation

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)

candidates = {
    "logistic":      make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000)),
    "knn":           make_pipeline(StandardScaler(), KNeighborsClassifier()),
    "svm rbf":       make_pipeline(StandardScaler(), SVC()),
    "random forest": RandomForestClassifier(n_estimators=200, random_state=0),
}
for name, model in candidates.items():
    scores = cross_val_score(model, X_train, y_train, cv=cv)
    print(f"{name:14s} {scores.mean():.3f} +/- {scores.std():.3f}")

Several models are close to perfect on this dataset. In a real project you would weigh accuracy against speed, interpretability and how easy the model is to maintain. Here we will tune the SVM.

6. Tune the chosen model

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)

pipe = make_pipeline(StandardScaler(), SVC())
grid = {"svc__C": [0.1, 1, 10, 100], "svc__gamma": ["scale", 0.1, 0.01, 0.001]}
search = GridSearchCV(pipe, grid, cv=cv).fit(X_train, y_train)
print("best settings:", search.best_params_)
print("best cv accuracy:", round(search.best_score_, 3))

7. Final evaluation on the test set

Once. This is the number that goes in the report.

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC
from sklearn.metrics import classification_report, confusion_matrix

X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

final = make_pipeline(StandardScaler(), SVC(C=1, gamma="scale")).fit(X_train, y_train)
pred = final.predict(X_test)
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred, target_names=["class 0", "class 1", "class 2"]))

8. Understand what it learned

Even a good score deserves a sanity check. A quick permutation importance shows which measurements the model relies on; if the top feature makes no chemical sense, investigate before shipping.

python
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC
from sklearn.inspection import permutation_importance

data = load_wine()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2, random_state=42, stratify=data.target)
final = make_pipeline(StandardScaler(), SVC(C=1)).fit(X_train, y_train)

imp = permutation_importance(final, X_test, y_test, n_repeats=10, random_state=0)
ranked = sorted(zip(data.feature_names, imp.importances_mean), key=lambda t: -t[1])
for name, score in ranked[:5]:
    print(f"{name:20s} {score:.3f}")

9. Save and reload

python
import pickle
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC

X, y = load_wine(return_X_y=True)
final = make_pipeline(StandardScaler(), SVC(C=1)).fit(X, y)

blob = pickle.dumps(final)                 # in a real project: pickle.dump(final, open("wine.pkl", "wb"))
restored = pickle.loads(blob)
print("prediction for first wine:", restored.predict(X[:1]))

The pipeline carries its scaler with it, so whoever loads the model does not need to remember any preprocessing. joblib is a faster alternative to pickle for large NumPy based models.

10. Deploy and monitor

In production the model sits behind an API (FastAPI is the common choice), a scheduled batch job, or inside an application. Log every prediction along with its inputs. When you later learn the true labels, compare them with the predictions; if accuracy drifts down, retrain on recent data. Keep the training code, the data snapshot and the model version together so any result can be reproduced.

A checklist for your own projects

  1. One sentence problem statement and a chosen metric.
  2. Look at the data before modelling.
  3. Split first; keep the test set sealed.
  4. Beat a dummy baseline.
  5. Compare simple models with cross-validation, in pipelines.
  6. Tune only the winner.
  7. Evaluate once on the test set.
  8. Check what the model relies on.
  9. Save the whole pipeline.
  10. Monitor and plan to retrain.

Where to go next

  • Rebuild this project on a dataset you care about. Kaggle and the UCI repository have hundreds.
  • Learn one deep learning framework (PyTorch) for image and text problems.
  • Learn SQL and a bit of data engineering; most of the job is getting the data.
  • Read scikit-learn's user guide. It is one of the best pieces of documentation in software.

Practice

  1. Repeat the whole pipeline on the breast cancer dataset, choosing recall on the malignant class as your metric.
  2. Replace the SVM with a random forest and compare the permutation importances. Do the same features come out on top?
  3. Write the deployment step: a function predict_variety(measurements: list[float]) -> int that loads the saved model and returns a prediction.