Lesson 12

Encoding Categories

Turn text columns into numbers with OneHotEncoder and OrdinalEncoder, handle unseen categories safely, and feed the encoded columns to a model.

Intermediate25 min

Encoding Categories hero Module 27 Lesson 12

Models need numbers, but plenty of useful columns are text: a course track, a city, a level. Replacing Python, Data, and Web with 0, 1, and 2 would invent an order that does not exist. One-hot encoding gives each category its own 0 or 1 column instead, and ordinal encoding is kept for categories that really are ordered.

A track column turned into three columns of zeros and ones

One-hot encoding

OneHotEncoder learns the categories during fit, sorted alphabetically. sparse_output=False returns an ordinary array you can print, and get_feature_names_out names the new columns.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(sparse_output=False)
encoded = encoder.fit_transform(learners[["track"]])

print(learners["track"].head(4).tolist())
print(encoded[:4])
print(encoder.get_feature_names_out())

Output:

text

['Web', 'Data', 'Python', 'Data']
[[0. 0. 1.]
 [1. 0. 0.]
 [0. 1. 0.]
 [1. 0. 0.]]
['track_Data' 'track_Python' 'track_Web']

Categories the encoder never saw

By default a new category at prediction time raises an error. handle_unknown="ignore" writes a row of zeros instead, so one unexpected value does not break the whole batch.

python

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

train = pd.DataFrame({"track": ["Python", "Data", "Web", "Data"]})
fresh = pd.DataFrame({"track": ["Web", "Cloud"]})

strict = OneHotEncoder(sparse_output=False).fit(train)
try:
    strict.transform(fresh)
except ValueError:
    print("strict encoder: ValueError")

lenient = OneHotEncoder(sparse_output=False, handle_unknown="ignore").fit(train)
print(lenient.transform(fresh))

Output:

text

strict encoder: ValueError
[[0. 0. 1.]
 [0. 0. 0.]]

OrdinalEncoder for ordered levels

When the order is real, one column of 0, 1, 2 is fine. Pass the order explicitly, otherwise the encoder sorts alphabetically and advanced would come first.

python

import pandas as pd
from sklearn.preprocessing import OrdinalEncoder

levels = pd.DataFrame({"level": ["beginner", "advanced", "intermediate", "beginner"]})
encoder = OrdinalEncoder(categories=[["beginner", "intermediate", "advanced"]])

print(encoder.fit_transform(levels).ravel())
print(OrdinalEncoder().fit(levels).categories_)

Output:

text

[0. 2. 1. 0.]
[array(['advanced', 'beginner', 'intermediate'], dtype=object)]

pd.get_dummies for a quick look

pandas can one-hot encode too, which is handy for exploring. Prefer the scikit-learn encoder for models, because it remembers the categories from training and applies the same columns to new rows.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

print(pd.get_dummies(learners["track"].head(4), dtype=int))

Output:

text

   Data  Python  Web
0     0       0    1
1     1       0    0
2     0       1    0
3     1       0    0

Use the encoded columns in a model

Fit the encoder on training rows, stack its columns next to the numeric features, and fit as usual. Knowing the track improves the score prediction a little, because tracks shift scores up or down.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder

train, test = train_test_split(learners, test_size=0.2, random_state=42)
encoder = OneHotEncoder(sparse_output=False).fit(train[["track"]])

def features(rows):
    return np.hstack([rows[["hours", "practice"]].to_numpy(), encoder.transform(rows[["track"]])])

plain = LinearRegression().fit(train[["hours", "practice"]], train["score"])
with_track = LinearRegression().fit(features(train), train["score"])

print("without track:", round(plain.score(test[["hours", "practice"]], test["score"]), 3))
print("with track:   ", round(with_track.score(features(test), test["score"]), 3))

Output:

text

without track: 0.706
with track:    0.741

Stacking columns by hand works but is fiddly. The pipelines lesson does the same thing in one object.

Key takeaways

Text columns must be encoded before fitting
OneHotEncoder makes one 0 or 1 column per category
handle_unknown="ignore" turns unseen categories into all zeros
OrdinalEncoder suits truly ordered categories; pass the order explicitly
Fit encoders on training rows, like scalers
Next: Missing Values
Previous: Feature Scaling

Loading playground…

Check your understanding

Encoding Categories Quiz

Why not replace Python, Data, Web with 0, 1, 2?

1 / 10

← Feature Scaling

Missing Values →