Skip to main content
MahiDev — Learn. Build. Become AI-ready.

Lesson 11

Feature Scaling

Put features on comparable scales with StandardScaler and MinMaxScaler, fit the scaler on training rows only, and watch nearest neighbours recover.

Intermediate25 min
Feature Scaling hero Module 27 Lesson 11Feature Scaling hero Module 27 Lesson 11

Hours run from 1 to 12 while video minutes run into the thousands. Any model that measures distance or adds up weighted features treats the bigger numbers as more important, even when they mean nothing. Scaling rewrites every feature into a similar range so that each one gets a fair say.

Hours squashed near zero next to widely spread video minutes, then both spread evenly after scalingHours squashed near zero next to widely spread video minutes, then both spread evenly after scaling

Features on different scales

describe makes the problem obvious. A difference of 100 video minutes is tiny, but in raw numbers it outweighs the whole range of hours.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

print(learners[["hours", "practice", "video_minutes"]].describe().loc[["min", "max", "std"]].round(1))

Output:

text
     hours  practice  video_minutes
min    1.1       0.0           16.0
max   12.0      40.0         2979.0
std    3.3      12.2          857.3

StandardScaler

StandardScaler subtracts each column's mean and divides by its standard deviation. Afterwards every column has mean 0 and standard deviation 1. Scalers have fit, transform, and fit_transform, which does both.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.preprocessing import StandardScaler

X = learners[["hours", "practice", "video_minutes"]]
scaled = StandardScaler().fit_transform(X)

print(scaled[:3].round(2))
print("means:", scaled.mean(axis=0).round(2) + 0.0)
print("stds: ", scaled.std(axis=0).round(2))

Output:

text
[[ 0.61 -1.39  1.41]
 [-0.67 -1.64 -1.03]
 [-1.34  0.25  0.97]]
means: [0. 0. 0.]
stds:  [1. 1. 1.]

Adding 0.0 turns any -0.0 from rounding into a plain 0.0.

MinMaxScaler

MinMaxScaler squeezes each column into 0 to 1 instead. It keeps the shape of the data but is pulled around by extreme values.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.preprocessing import MinMaxScaler

X = learners[["hours", "practice", "video_minutes"]]
scaled = MinMaxScaler().fit_transform(X)

print(scaled.min(axis=0), scaled.max(axis=0))
print(scaled[:3].round(2))

Output:

text
[0. 0. 0.] [1. 1. 1.]
[[0.7  0.1  0.92]
 [0.31 0.02 0.22]
 [0.11 0.6  0.79]]

Fit on the training rows only

The scaler learns means and spreads, so it must learn them from the training rows alone. Then transform both sets with those numbers. The test columns come out close to, but not exactly, mean 0.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X = learners[["hours", "practice", "video_minutes"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
scaler = StandardScaler().fit(X_train)
train_scaled = scaler.transform(X_train)
test_scaled = scaler.transform(X_test)

print("learned means:", scaler.mean_.round(1))
print("train means:", train_scaled.mean(axis=0).round(2) + 0.0)
print("test means: ", test_scaled.mean(axis=0).round(2))

Output:

text
learned means: [   6.8   20.2 1564.9]
train means: [0. 0. 0.]
test means:  [-0.13  0.28 -0.13]

Fitting the scaler on all rows would leak facts about the test set into training, which makes scores look better than they will be on new data.

Scaling rescues nearest neighbours

Back to the KNN problem from earlier. The same model on the same columns goes from barely better than a coin flip back to roughly the accuracy it had with hours and practice alone.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler

X = learners[["hours", "practice", "video_minutes"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
raw = KNeighborsClassifier().fit(X_train, y_train)
print("raw:   ", raw.score(X_test, y_test))

scaler = StandardScaler().fit(X_train)
scaled = KNeighborsClassifier().fit(scaler.transform(X_train), y_train)
print("scaled:", scaled.score(scaler.transform(X_test), y_test))

Output:

text
raw:    0.575
scaled: 0.85

Key takeaways

  • Distance-based and weighted models are skewed by features with big numbers

  • StandardScaler gives each column mean 0 and standard deviation 1

  • MinMaxScaler squeezes each column into 0 to 1

  • Fit the scaler on training rows, then transform training and test rows

  • Trees do not need scaling; KNN and many linear models do

  • Next: Encoding Categories

  • Previous: Random Forests

Loading playground…

Check your understanding

Feature Scaling Quiz

Why scale features?

1 / 10

Previous

← Random Forests

Next

Encoding Categories →