Pandas
1 min read
Updated 4 Aug 2026
16. Performance Tips
df['city'] = df['city'].astype('category') # if few unique values
| Technique | Benefit |
|---|---|
| Categorical dtype for low-cardinality strings | huge memory savings + faster groupby |
Downcast numerics (int32, float32) |
halve memory |
Vectorize — avoid iterrows/apply |
10–100× faster |
query for complex filters |
readable and can be faster |
chunksize for big files |
bounded memory |
eval for arithmetic on large frames |
avoids temporaries |
df['city'] = df['city'].astype('category') # if few unique values
df['id'] = pd.to_numeric(df['id'], downcast='integer')
# ❌ never iterate rows for computation
# for i, row in df.iterrows(): df.at[i,'x'] = row.a + row.b
# ✅ vectorize
df['x'] = df['a'] + df['b']
⚠️ Common Mistake:
df.iterrows()is extremely slow (Python-level row objects). Vectorize or use.apply(axis=1)only as a last resort.
🚀 Best Practice: Converting a high-frequency string column (like country, category) to
categorydtype can cut memory by 90% and speed upgroupbydramatically.
⭐ Interview Question: How to reduce a DataFrame's memory? Downcast numeric dtypes, convert low-cardinality strings to
category, drop unused columns, and read only needed columns withusecols.