Skip to main content
MahiDev — Learn. Build. Become AI-ready.

Lesson 06

Logistic Regression

Predict a yes or no answer with LogisticRegression, read probabilities with predict_proba, pick your own threshold, and check accuracy on test rows.

Intermediate25 min
Logistic Regression hero Module 27 Lesson 6Logistic Regression hero Module 27 Lesson 6

Classification predicts a category instead of a number: passed or not, spam or not. Despite its name, logistic regression is a classifier. It combines the features like a line does, then squeezes the result through an S-shaped curve into a probability between 0 and 1. Above 0.5, it answers yes.

An S-shaped probability curve rising with hours, with pass and fail points at 1 and 0An S-shaped probability curve rising with hours, with pass and fail points at 1 and 0

A yes or no answer

Fit on the training rows with passed as the label. classes_ lists the answers the model can give, and predict returns one of them per row.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)

print(model.classes_)
print("predicted:", model.predict(X_test.head(8)).tolist())
print("actual:   ", y_test.head(8).tolist())

Output:

text
[0 1]
predicted: [0, 1, 0, 0, 1, 1, 1, 1]
actual:    [0, 1, 0, 0, 1, 1, 1, 1]

Probabilities with predict_proba

predict_proba returns one column per class, in the order of classes_. Each row adds up to 1. The second column is the chance of passing.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)
newcomers = pd.DataFrame({"hours": [3.0, 6.5, 10.0], "practice": [10, 20, 35]})

print(model.predict_proba(newcomers).round(3))
print(model.predict(newcomers))

Output:

text
[[0.985 0.015]
 [0.606 0.394]
 [0.02  0.98 ]]
[0 0 1]

Choose your own threshold

predict uses 0.5. If a wrong yes is costly, for example flagging a learner as safe when they are not, require a higher probability before answering yes.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)
chance = model.predict_proba(X_test)[:, 1]

print("yes at 0.5:", int((chance >= 0.5).sum()))
print("yes at 0.7:", int((chance >= 0.7).sum()))
print("predict agrees with 0.5:", bool((model.predict(X_test) == (chance >= 0.5)).all()))

Output:

text
yes at 0.5: 20
yes at 0.7: 16
predict agrees with 0.5: True

Accuracy with score

For classifiers, score returns accuracy: the share of rows answered correctly. The next lesson shows why accuracy alone can mislead.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)

print("train accuracy:", round(model.score(X_train, y_train), 3))
print("test accuracy: ", round(model.score(X_test, y_test), 3))

Output:

text
train accuracy: 0.825
test accuracy:  0.825

Coefficients push toward yes

Positive weights raise the chance of the second class. Both features help here, and one hour of study is worth several practice problems.

python
import numpy as np
import pandas as pd

rng = np.random.default_rng(27)
learners = pd.DataFrame({
    "hours": rng.uniform(1, 12, 200).round(1),
    "practice": rng.integers(0, 41, 200),
    "video_minutes": rng.integers(0, 3001, 200),
    "track": rng.choice(["Python", "Data", "Web"], 200),
})
bonus = learners["track"].map({"Python": 0, "Data": 6, "Web": -5})
noise = rng.normal(0, 6, 200)
learners["score"] = (
    22 + learners["hours"] * 3.4 + learners["practice"] * 0.55 + bonus + noise
).clip(0, 100).round()
learners["passed"] = (learners["score"] >= 60).astype(int)

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = learners[["hours", "practice"]]
y = learners["passed"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)

for name, weight in zip(X.columns, model.coef_[0]):
    print(f"{name}: {weight:.2f}")
print(f"intercept: {model.intercept_[0]:.2f}")

Output:

text
hours: 0.77
practice: 0.11
intercept: -7.59

Key takeaways

  • Classification predicts a category; logistic regression is a classifier

  • classes_ lists the possible answers in column order

  • predict_proba gives one probability per class; each row sums to 1

  • predict uses a 0.5 threshold; compare probabilities to use your own

  • score gives accuracy for classifiers

  • Next: Measuring Classification

  • Previous: Measuring Regression Error

Loading playground…

Check your understanding

Logistic Regression Quiz

What does classification predict?

1 / 10

Previous

← Measuring Regression Error

Next

Measuring Classification →