Quick Methods to Detect Outliers in Your Data

To detect outliers, flag values outside the IQR fences, Q1 − 1.5 × IQR and Q3 + 1.5 × IQR, or, for roughly normal data, values more than 3 standard deviations from the mean (|z| > 3). Plot the data first and investigate every flagged point before removing it, because an outlier can be a data error or your most important finding.

How do you find outliers with the IQR method?

Compute the first quartile (Q1) and third quartile (Q3), take IQR = Q3 − Q1, and flag anything below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR. Quartiles are not pulled around by extreme values, so the fences stay put however large the outliers are, which makes this the safest default when you don't know the distribution.

# Python
Q1 = df['price'].quantile(0.25)
Q3 = df['price'].quantile(0.75)
IQR = Q3 - Q1

lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR

# Find outliers
outliers = df[(df['price'] < lower_bound) | (df['price'] > upper_bound)]
print(f"Found {len(outliers)} outliers")

Why 1.5? It is John Tukey's convention. On normally distributed data the fences sit about 2.7 standard deviations from the mean and flag roughly 0.7% of values. Use 3 × IQR when you only want the extreme ("far out") points.

Pros: Not affected by extreme outliers. Watch out: on strongly right-skewed data it flags many legitimate tail values (about 8% of a standard lognormal distribution), so for positive, skewed data such as prices or incomes, apply it to log-transformed values.

How do you detect outliers with z-scores?

A z-score is (value − mean) / standard deviation, the number of standard deviations a value sits from the mean; flag values with |z| > 3. In normally distributed data only about 0.3% of values land that far out, so anything beyond is unusual.

# Python
import numpy as np
from scipy import stats

# Add nan_policy='omit' if the column has missing values;
# otherwise every score comes back NaN
z_scores = np.abs(stats.zscore(df['price']))
outliers = df[z_scores > 3]

# Interpretation (for normally distributed data):
# |z| > 3  → extreme: only about 0.3% of values are this far out
# |z| > 2  → unusual: about 5% of values are this far out
# |z| ≤ 2  → normal range

Limitation: Assumes roughly normal data, and the outliers inflate the very mean and standard deviation they are measured against. Use the IQR method for skewed data and the modified z-score below for small samples. The formula is covered in more depth in z-score explained.

How do you flag outliers with percentiles?

Treat values below the 1st percentile or above the 99th percentile as outliers. This always flags about 2% of rows whether or not anything is unusual, so it suits trimming and capping better than deciding whether outliers exist.

# Flag top and bottom 1% as outliers
lower = df['price'].quantile(0.01)
upper = df['price'].quantile(0.99)

outliers = df[(df['price'] < lower) | (df['price'] > upper)]

Flexible: Adjust percentiles based on your needs (0.5%, 2.5%, etc.)

Why can the z-score method miss an obvious outlier?

Because the outlier inflates the mean and standard deviation used to score it, and in small samples the largest possible z-score is capped. With stats.zscore, which divides by the population standard deviation, no value can score above √(n − 1), so with 10 or fewer rows a |z| > 3 rule can never flag anything.

Take ten prices: 12, 14, 15, 15, 16, 17, 18, 19, 21 and 95. The IQR rule gives Q1 = 15 and Q3 = 18.75 (pandas' default linear interpolation), so IQR = 3.75 and the fences are 9.375 and 24.375: 95 is flagged. The z-score rule misses it, because the mean is 24.2, the standard deviation is 23.7 and 95 scores only 2.98. The 1st/99th percentile rule flags 95 but also 12, the lowest ordinary price.

The modified z-score avoids the problem by using the median and the median absolute deviation (MAD), which a few outliers barely move. Iglewicz and Hoaglin's version flags scores above 3.5; for these prices the median is 16.5, the MAD is 2 and 95 scores 26.47.

# Modified z-score: median and MAD instead of mean and SD
median = df['price'].median()
mad = (df['price'] - median).abs().median()
modified_z = 0.6745 * (df['price'] - median) / mad
outliers = df[modified_z.abs() > 3.5]

If more than half the values are identical, the MAD is 0 and every value that differs from the median gets an infinite score, so fall back to the IQR rule for that column.

How do you spot outliers visually?

Plot the data before computing anything. A box plot draws the IQR rule for you: by default its whiskers stop at the furthest points within 1.5 × IQR of the box, and every point drawn beyond them is an outlier by that rule (see box plots explained).

Box Plot - See Outliers Immediately

# Python
import matplotlib.pyplot as plt

df.boxplot(column='price')
plt.show()

# Points beyond the whiskers = outliers

Scatter Plot - Multivariate Outliers

plt.scatter(df['age'], df['salary'])
# Look for points far from the cluster

A scatter plot catches multivariate outliers: points that look normal on each variable alone but not in combination, such as a 22-year-old with a senior executive's salary. With many variables, use Mahalanobis distance or an isolation forest (scikit-learn's IsolationForest) instead of eyeballing every pair.

How should you handle outliers?

Remove an outlier only when you can show it is an error, such as a typo, a test record or a unit mix-up. Genuine extreme values carry information, so keep, cap or transform them instead, and document every row you change.

Option 1: Keep Them

When: Outliers are valid and important

Example: CEO salary in salary analysis (real data point)

Option 2: Remove Them

# Remove outliers
df_clean = df[z_scores <= 3]

When: Data errors or measurement mistakes

Warning: Document why you removed them!

Option 3: Cap Them (Winsorization)

# Cap at 1st and 99th percentiles
lower = df['price'].quantile(0.01)
upper = df['price'].quantile(0.99)

df['price_capped'] = df['price'].clip(lower, upper)

When: Want to reduce impact but keep all data points

Option 4: Transform Them

# Log transformation reduces impact of high outliers
df['log_price'] = np.log(df['price'] + 1)

# Or use robust statistics
median_price = df['price'].median()  # Instead of mean

When: Positive, right-skewed values such as prices or incomes, where a log scale shrinks the extremes and the median summarizes the data better than the mean.

IQR or z-score: which method should you use?

Use the IQR rule by default, and the z-score only for roughly normal data with a reasonable sample size. Whichever you choose, look at a box plot or scatter plot first.

Situation	Method
Normal distribution	Z-score (threshold: 3)
Skewed distribution	IQR method (on log values if strongly skewed)
Small sample (a few dozen rows or fewer)	IQR or modified z-score
Don't know distribution	IQR method (safest)
Need quick check	Box plot visualization
Multiple variables	Scatter plots first

Common Mistakes to Avoid

The costliest mistake is deleting outliers you have not investigated: an extreme value can be a data error, fraud or your best customer, and only errors should be removed.

• Removing outliers without investigating why they exist
• Using z-scores on heavily skewed data
• Using |z| > 3 on 10 or fewer rows, where it cannot flag anything
• Not documenting which outliers you removed
• Removing outliers before visualizing them (you might miss insights)