Skip to main content
MahiDev — Learn. Build. Become AI-ready.

Lesson 18

Grouping with groupby

Split a table into groups on one or more columns, then count, total, and average each group with the split, apply, combine pattern.

Intermediate25 min
Grouping with groupby hero Module 24 Lesson 18Grouping with groupby hero Module 24 Lesson 18

groupby splits a table into one group per distinct value, applies a summary to each group, and combines the answers into a new table. Split, apply, combine is the pattern behind most reporting work.

Four learner rows collapsing into one average row per trackFour learner rows collapsing into one average row per track

One number per group

Name the column to group by, pick the column to summarize, and call the summary. The distinct values become the index of the result, in sorted order.

python
import pandas as pd

learners = pd.DataFrame({
    "name": ["Avery", "Jordan", "Casey", "Riley", "Morgan", "Quinn"],
    "track": ["Python", "NumPy", "Python", "pandas", "NumPy", "pandas"],
    "city": ["Northgate", "Eastport", "Northgate", "Westfield", "Eastport", "Westfield"],
    "hours": [42, 31, 55, 24, 47, 38],
    "score": [88, 74, 91, 65, 82, 79],
})
print(learners.groupby("track")["score"].mean())
print(learners.groupby("track")["hours"].sum())
print(learners.groupby("city")["name"].count())

Output:

text
track
NumPy     78.0
Python    89.5
pandas    72.0
Name: score, dtype: float64
track
NumPy     78
Python    97
pandas    62
Name: hours, dtype: int64
city
Eastport     2
Northgate    2
Westfield    2
Name: name, dtype: int64

How big is each group

size() counts the rows in each group, gaps included. count() counts only the real values, so the two differ exactly where data is missing. value_counts() is the shortcut when the size is all you want.

python
import pandas as pd

learners = pd.DataFrame({
    "name": ["Avery", "Jordan", "Casey", "Riley", "Morgan", "Quinn"],
    "track": ["Python", "NumPy", "Python", "pandas", "NumPy", "pandas"],
    "city": ["Northgate", "Eastport", "Northgate", "Westfield", "Eastport", "Westfield"],
    "hours": [42, 31, 55, 24, 47, 38],
    "score": [88, 74, 91, 65, 82, 79],
})
print(learners.groupby("track").size())
print(learners["track"].value_counts())
print(learners.groupby("track")["score"].count())

Output:

text
track
NumPy     2
Python    2
pandas    2
dtype: int64
track
Python    2
NumPy     2
pandas    2
Name: count, dtype: int64
track
NumPy     2
Python    2
pandas    2
Name: score, dtype: int64

Several columns at a time

A list of columns after the group gives a summary table rather than one column. Ask for the numeric columns by name: a mean over a text column has no meaning and pandas will not invent one.

python
import pandas as pd

learners = pd.DataFrame({
    "name": ["Avery", "Jordan", "Casey", "Riley", "Morgan", "Quinn"],
    "track": ["Python", "NumPy", "Python", "pandas", "NumPy", "pandas"],
    "city": ["Northgate", "Eastport", "Northgate", "Westfield", "Eastport", "Westfield"],
    "hours": [42, 31, 55, 24, 47, 38],
    "score": [88, 74, 91, 65, 82, 79],
})
print(learners.groupby("track")[["hours", "score"]].mean().round(2))
print(learners.groupby("track", as_index=False)[["hours"]].sum())

Output:

text
        hours  score
track               
NumPy    39.0   78.0
Python   48.5   89.5
pandas   31.0   72.0
    track  hours
0   NumPy     78
1  Python     97
2  pandas     62

as_index=False keeps the grouping column as a normal column, which is what you want when the result feeds into another step.

Group on two columns

A list of grouping columns makes one group per combination, and the result gets a two-level index. reset_index() flattens it back into ordinary columns.

python
import pandas as pd

learners = pd.DataFrame({
    "name": ["Avery", "Jordan", "Casey", "Riley", "Morgan", "Quinn"],
    "track": ["Python", "NumPy", "Python", "pandas", "NumPy", "pandas"],
    "city": ["Northgate", "Eastport", "Northgate", "Westfield", "Eastport", "Westfield"],
    "hours": [42, 31, 55, 24, 47, 38],
    "score": [88, 74, 91, 65, 82, 79],
})
by_pair = learners.groupby(["city", "track"])["score"].mean()
print(by_pair)
print(by_pair.reset_index())

Output:

text
city       track 
Eastport   NumPy     78.0
Northgate  Python    89.5
Westfield  pandas    72.0
Name: score, dtype: float64
        city   track  score
0   Eastport   NumPy   78.0
1  Northgate  Python   89.5
2  Westfield  pandas   72.0

Look inside the groups

A group object is iterable: each turn gives the group name and the rows that belong to it. get_group fetches one by name, and first takes the leading row of every group.

python
import pandas as pd

learners = pd.DataFrame({
    "name": ["Avery", "Jordan", "Casey", "Riley", "Morgan", "Quinn"],
    "track": ["Python", "NumPy", "Python", "pandas", "NumPy", "pandas"],
    "city": ["Northgate", "Eastport", "Northgate", "Westfield", "Eastport", "Westfield"],
    "hours": [42, 31, 55, 24, 47, 38],
    "score": [88, 74, 91, 65, 82, 79],
})
grouped = learners.groupby("track")
for track, rows in grouped:
    print(track, rows["name"].tolist())

print(grouped.get_group("pandas")[["name", "score"]])
print(grouped.first()[["name", "score"]])

Output:

text
NumPy ['Jordan', 'Morgan']
Python ['Avery', 'Casey']
pandas ['Riley', 'Quinn']
    name  score
3  Riley     65
5  Quinn     79
          name  score
track                
NumPy   Jordan     74
Python   Avery     88
pandas   Riley     65

Key takeaways

  • groupby splits, applies a summary, and combines the answers

  • The grouping values become the index unless you pass as_index=False

  • size counts rows and count counts only the values that are present

  • A list of grouping columns gives one group per combination

  • Next: agg, transform, and filter

  • Previous: Iterating Rows and Columns

Loading playground…

Check your understanding

Grouping with groupby Quiz

What are the three steps behind groupby?

1 / 10

Previous

← Iterating Rows and Columns

Next

agg, transform, and filter →