9. GroupBy

The split-apply-combine pattern: split rows into groups, apply a function, combine results.

flowchart LR
    A["DataFrame (dept, salary)"] -->|"1 · split by dept"| G1["Eng: 90, 95"]
    A -->|"1 · split by dept"| G2["Sales: 60, 65"]
    G1 -->|"2 · apply mean()"| R1["Eng → 92.5"]
    G2 -->|"2 · apply mean()"| R2["Sales → 62.5"]
    R1 -->|"3 · combine"| C["Result — one row per group"]
    R2 -->|"3 · combine"| C

df = pd.DataFrame({
    'dept':   ['Eng','Eng','Sales','Sales','Eng'],
    'gender': ['M','F','M','F','M'],
    'salary': [90, 85, 60, 65, 95],
    'age':    [30, 28, 40, 35, 33]
})

df.groupby('dept')['salary'].mean()
# dept
# Eng      90.0
# Sales    62.5

df.groupby('dept').agg(
    avg_salary=('salary', 'mean'),
    max_age=('age', 'max'),
    headcount=('salary', 'count')
)                                   # NAMED aggregations (cleanest)

df.groupby(['dept','gender'])['salary'].mean()   # multi-key

# multiple aggs on multiple columns
df.groupby('dept').agg({'salary': ['mean','max'], 'age': 'mean'})

transform vs filter vs apply#

# transform: returns same-length result -> great for group-normalization
df['salary_z'] = df.groupby('dept')['salary'].transform(
    lambda s: (s - s.mean()) / s.std())

# filter: keep only groups meeting a condition
df.groupby('dept').filter(lambda g: g['salary'].mean() > 70)

# apply: most flexible (and slowest)
df.groupby('dept').apply(lambda g: g.nlargest(1, 'salary'))

🚀 Best Practice: Use named aggregation agg(new=('col','func')) — it produces flat, clearly-named columns instead of a confusing MultiIndex.

⚠️ Common Mistake: Using apply when transform or agg would do — apply is much slower and can silently change output shape.

⭐ Interview Question: transform vs agg? agg reduces each group to one value; transform returns a value for every row (same length), ideal for adding group-derived columns.

Real business example: df.groupby('customer_id')['order_value'].agg(['sum','count','mean']) → per-customer lifetime value, order count, and average order.