Lesson 02

Features and Labels

Split a table into X, the two-dimensional feature table, and y, the answer column, and learn why models need numbers, double brackets, and reshape.

Intermediate25 min

Features and Labels hero Module 27 Lesson 2

Every supervised model needs two things. X holds the features: the columns the model looks at, one row per example. y holds the label: the answer you want it to learn, one value per row. Getting their shapes right is the first thing to check when scikit-learn complains.

The learners table with hours and practice marked as X and passed marked as y

X is a table, y is a column

Select the feature columns with a list inside the brackets and the label with a single name. X has one row per learner and one column per feature. y has one value per learner.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

X = learners[["hours", "practice"]]
y = learners["passed"]

print(X.shape, y.shape)
print(X.head(3))
print(y.head(3).tolist())

Output:

text

(200, 2) (200,)
   hours  practice
0    8.7         4
1    4.5         1
2    2.3        24
[0, 0, 0]

Double brackets keep X two-dimensional

learners["hours"] is a Series with one dimension. learners[["hours"]] is a DataFrame with one column. Models always want the second shape, even for a single feature, and refuse the first.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

print(learners["hours"].shape, learners[["hours"]].shape)

model = LinearRegression()
try:
    model.fit(learners["hours"], learners["score"])
except ValueError as error:
    print("fit refused the Series:", type(error).__name__)

model.fit(learners[["hours"]], learners["score"])
print("fitted on", model.n_features_in_, "feature")

Output:

text

(200,) (200, 1)
fit refused the Series: ValueError
fitted on 1 feature

reshape(-1, 1) for NumPy arrays

With a plain NumPy array there are no double brackets, so reshape it. -1 means work out the number of rows, and 1 asks for one column.

python

import numpy as np

hours = np.array([2.0, 4.5, 7.0, 9.5])
print(hours.shape)

X = hours.reshape(-1, 1)
print(X.shape)
print(X)

Output:

text

(4,)
(4, 1)
[[2. ]
 [4.5]
 [7. ]
 [9.5]]

Features must be numbers

Models do arithmetic, so a text column such as track stops fit straight away. A later lesson turns text into numbers with an encoder. For now, pick numeric columns.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

try:
    LinearRegression().fit(learners[["hours", "track"]], learners["score"])
except ValueError as error:
    print("ValueError:", "could not convert string to float" in str(error))

print(learners.select_dtypes("number").columns.tolist())

Output:

text

ValueError: True
['hours', 'practice', 'video_minutes', 'score', 'passed']

Models remember column names

Fit on a DataFrame and the model stores the feature names. New rows must then arrive as a DataFrame with the same columns, which catches mix-ups such as passing practice where hours was expected.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

X = learners[["hours", "practice"]]
model = LinearRegression().fit(X, learners["score"])

print(model.feature_names_in_)
newcomer = pd.DataFrame({"hours": [5.0], "practice": [20]})
print(model.predict(newcomer).round(1))

Output:

text

['hours' 'practice']
[50.4]

Key takeaways

X is a two-dimensional table of features, one row per example
y is the label column, one answer per row
Use double brackets or reshape(-1, 1) for a single feature
Features must be numeric; text columns need encoding first
A model fitted on a DataFrame expects the same column names later
Next: Training and Test Sets
Previous: What Is Scikit-learn

Loading playground…

Check your understanding

Features and Labels Quiz

What does X usually hold?

1 / 10

← What Is Scikit-learn

Training and Test Sets →