Skip to main content
MahiDev — Learn. Build. Become AI-ready.

Lesson 04

Linear Regression

Predict a number with LinearRegression: fit a line, read coef_ and intercept_, add more features, and check the fit with the R squared score.

Intermediate25 min
Linear Regression hero Module 27 Lesson 4Linear Regression hero Module 27 Lesson 4

Regression predicts a number, such as a score. Linear regression assumes each feature pushes the answer up or down by a fixed amount per unit. With one feature that is a straight line: score equals intercept plus slope times hours. fit chooses the slope and intercept that make the squared misses as small as possible.

Learners' scores plotted against hours with a rising fitted lineLearners' scores plotted against hours with a rising fitted line

One feature, one straight line

After fitting, the line is stored on the model. coef_ holds one slope per feature and intercept_ is the score the line gives at zero hours. Names ending in an underscore are always things the model learned.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

model = LinearRegression().fit(learners[["hours"]], learners["score"])
print("slope:", model.coef_.round(2))
print("intercept:", round(model.intercept_, 2))

Output:

text
slope: [3.4]
intercept: 34.17

Predict new learners

predict plugs each new row into the line. Doing the arithmetic by hand gives the same numbers, which is a good way to see there is no magic.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

model = LinearRegression().fit(learners[["hours"]], learners["score"])
newcomers = pd.DataFrame({"hours": [3.0, 6.0, 9.0]})

print(model.predict(newcomers).round(1))
print((model.intercept_ + model.coef_[0] * newcomers["hours"]).round(1).tolist())

Output:

text
[44.4 54.5 64.7]
[44.4, 54.5, 64.7]

More features, one coefficient each

Add practice and the model learns a weight for each column. Each weight is the change in score for one more unit of that feature while the others stay the same.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

X = learners[["hours", "practice"]]
model = LinearRegression().fit(X, learners["score"])

for name, weight in zip(X.columns, model.coef_):
    print(f"{name}: {weight:.2f} points per unit")
print(f"intercept: {model.intercept_:.2f}")

Output:

text
hours: 3.53 points per unit
practice: 0.56 points per unit
intercept: 21.58

R squared with score

For regressors, score returns R squared: 1.0 is a perfect fit, 0 is no better than always guessing the average. Measure it on the test set.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["score"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LinearRegression().fit(X_train, y_train)

print("train R squared:", round(model.score(X_train, y_train), 3))
print("test R squared: ", round(model.score(X_test, y_test), 3))

Output:

text
train R squared: 0.778
test R squared:  0.706

Train and test are fairly close. A straight line has only three numbers to learn here, so it cannot memorise individual learners the way a deep tree can; the small gap is mostly which rows happened to land in the test set.

A line never stops

The learned rule is only trustworthy inside the range it saw. Ask about 40 hours a week and the line happily predicts a score above 100.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LinearRegression

model = LinearRegression().fit(learners[["hours", "practice"]], learners["score"])
extreme = pd.DataFrame({"hours": [40.0], "practice": [20]})
print(model.predict(extreme).round(1))
print("largest hours seen:", learners["hours"].max())

Output:

text
[174.]
largest hours seen: 12.0

Key takeaways

  • LinearRegression predicts a number as intercept plus weight times each feature

  • coef_ has one weight per feature; intercept_ is the value at all zeros

  • Attributes ending in _ are learned during fit

  • score gives R squared for regressors; check it on test data

  • Predictions outside the training range are guesses

  • Next: Measuring Regression Error

  • Previous: Training and Test Sets

Loading playground…

Check your understanding

Linear Regression Quiz

What does regression predict?

1 / 10

Previous

← Training and Test Sets

Next

Measuring Regression Error →