Normal Distribution Explained — The Bell Curve in Data Analysis

The normal distribution (also called Gaussian distribution or bell curve) is a probability distribution that describes how data values are spread around the mean.

Key Characteristics

Shape: Symmetric bell curve

• Peak at mean: Most data clusters around the center (mean)
• Symmetric: Left and right sides are mirror images
• Tails: Fewer values at extremes (far from mean)

Mathematical Properties:

• Mean = Median = Mode (all at the center)
• Defined by two parameters: μ (mean, center) and σ (standard deviation, spread)
• Total area under curve = 1 (100% probability)

Visual Representation

Why Normal Distribution Matters

1. Natural Phenomena Follow Normal Distribution

• Heights of adult men: μ = 170cm, σ = 10cm (bell curve)
• IQ scores: μ = 100, σ = 15 (by design)
• Blood pressure readings, shoe sizes, birth weights

2. Central Limit Theorem

• Average of ANY distribution (even non-normal) → Normal distribution (with large enough sample)
• This makes normal distribution fundamental to statistics

3. Statistical Tests Assume Normality

• T-tests, ANOVA, linear regression all assume normal distribution
• Confidence intervals, p-values rely on normality

4. Quality Control

• Manufacturing tolerances (part dimensions cluster around target)
• Process control charts (detect deviations from normal behavior)

Real Example: Flipkart Delivery Times

Data: 10,000 deliveries in Mumbai

Business Application: Set delivery promise based on percentiles

• Promise "30-40 minutes" (covers ~68% of cases)
• For 95% reliability, promise "20-45 minutes" (μ ± 2σ)

For any normally distributed data, a fixed percentage of values falls within each standard deviation range from the mean.

The Rule

Practical Applications

Example 1: Student Test Scores

Data:

Apply 68-95-99.7 Rule:

1 SD from mean (68% of students):

2 SD from mean (95% of students):

3 SD from mean (99.7% of students):

Grading Application:

• A grade: Top 2.5% (> μ + 2σ = 95+ points)
• B grade: 84th to 97.5th percentile (85-95 points)
• C grade: 16th to 84th percentile (65-85 points)
• D grade: Bottom 2.5-16% (55-65 points)
• F grade: Bottom 2.5% (< 55 points)

Example 2: Zomato Order Preparation Time

Data:

Quality Control Thresholds:

Within 1 SD (68% of orders):

Within 2 SD (95% of orders):

Beyond 3 SD (0.3% of orders):

Anomaly Detection: Orders >32 min are 3 SD above mean (top 0.15%) → Likely issue (restaurant delay, traffic, driver problem)

Example 3: Website Load Time SLA

Data:

SLA Target: "95% of pages load in less than 2 seconds"

Check if Target is Met:

Action: Optimize to reduce mean OR standard deviation

• Reduce mean: 1.5s → 1.35s (then 95% < 1.95s ✓)
• Reduce SD: 0.3s → 0.25s (then 95% = 2.0s ✓)

Standard Normal Distribution

Definition: Normal distribution with mean = 0, standard deviation = 1.

Notation: Z ~ N(0, 1)

Why it matters: Any normal distribution can be standardized (converted to Z) for easy probability calculations.

Z-Score (Standardization)

Z-score = Number of standard deviations a value is from the mean.

Formula:

Interpretation:

• Z = 0 → Value equals mean
• Z = +1 → Value is 1 SD above mean
• Z = -2 → Value is 2 SD below mean
• Z = +3 → Value is 3 SD above mean (top 0.15%, very unusual)

Using Z-Scores: Real Examples

Example 1: Student Performance Comparison

Context: Compare students from different exams (different means and SDs).

Student A:

Student B:

Conclusion: Student B performed BETTER relative to their class (Z = 1.6 > 1.0), even though raw score (78) is lower than Student A (85).

Example 2: Flipkart Fraud Detection

Context: Detect unusual order amounts (potential fraud).

Customer Order History:

New Order: ₹4,500

Calculate Z-Score:

Interpretation:

• Order is 8.25 SD above mean (extremely unusual)
• Probability: P(Z > 8.25) < 0.0001% (virtually impossible by chance)
• Action: Flag for fraud review (likely compromised account or data entry error)

Threshold for Alerts:

• Z > 3: Unusual, investigate (top 0.15%)
• Z > 4: Very unusual, high priority alert (top 0.003%)
• Z > 5: Almost certainly fraud/error (top 0.00003%)

Example 3: Swiggy Delivery Time Percentile

Context: Customer asks "What percentile is my 25-minute delivery?"

Data:

Calculate Z-Score:

Find Percentile (using Z-table):

Z-Table (Standard Normal Table)

Purpose: Convert Z-score to probability (percentile).

Common Z-Scores:

| Z-Score | Percentile | Interpretation | |---------|------------|----------------| | -3.0 | 0.15% | Bottom 0.15% (extremely low) | | -2.0 | 2.5% | Bottom 2.5% (unusually low) | | -1.0 | 16% | Bottom 16% (below average) | | 0.0 | 50% | Median (exactly average) | | +1.0 | 84% | Top 16% (above average) | | +2.0 | 97.5% | Top 2.5% (unusually high) | | +3.0 | 99.85% | Top 0.15% (extremely high) |

How to Use:

1. Calculate Z-score: Z = (X - μ) / σ
2. Look up Z in table → Get percentile
3. Interpret: "Value is in Xth percentile" (higher/lower than X% of data)

Many statistical tests assume normality. Here's how to check if your data follows a normal distribution.

Method 1: Visual Inspection (Histogram)

Create a histogram and check for bell curve shape.

Python Example:

import pandas as pd
import matplotlib.pyplot as plt

# Sample data: Delivery times
data = pd.Series([28, 30, 31, 32, 32, 33, 34, 34, 35, 36, ...])

# Histogram
plt.hist(data, bins=30, edgecolor='black', alpha=0.7)
plt.xlabel('Delivery Time (minutes)')
plt.ylabel('Frequency')
plt.title('Delivery Time Distribution')
plt.show()

What to Look For:

• ✓ Symmetric bell shape (normal)
• ✗ Right skew (long tail on right — use median, not mean)
• ✗ Left skew (long tail on left — not normal)
• ✗ Bimodal (two peaks — two separate populations)
• ✗ Uniform (flat — all values equally likely, not normal)

Method 2: Q-Q Plot (Quantile-Quantile Plot)

Compare your data's quantiles to theoretical normal distribution quantiles.

Python Example:

import scipy.stats as stats

# Q-Q plot
stats.probplot(data, dist="norm", plot=plt)
plt.title('Q-Q Plot')
plt.show()

Interpretation:

• ✓ Points fall on straight line: Data is normally distributed
• ✗ Points deviate from line: Data is not normal
• S-curve: Heavy tails (more outliers than normal)
• Inverted S: Light tails (fewer outliers than normal)

Method 3: Shapiro-Wilk Test (Statistical Test)

Formal hypothesis test for normality.

Python Example:

from scipy.stats import shapiro

stat, p_value = shapiro(data)
print(f"Shapiro-Wilk statistic: {stat:.4f}")
print(f"P-value: {p_value:.4f}")

if p_value > 0.05:
    print("Data is normally distributed (fail to reject H₀)")
else:
    print("Data is NOT normally distributed (reject H₀)")

Interpretation:

• p > 0.05: Data is consistent with normal distribution (assume normality)
• p ≤ 0.05: Data is NOT normally distributed (use non-parametric tests)

Note: Shapiro-Wilk is sensitive to large samples (n > 5,000 often rejects normality even for "normal-ish" data). For large samples, rely more on visual inspection.

Method 4: Skewness and Kurtosis

Check symmetry (skewness) and tail heaviness (kurtosis).

Python Example:

from scipy.stats import skew, kurtosis

skewness = skew(data)
kurt = kurtosis(data)

print(f"Skewness: {skewness:.2f}")
print(f"Kurtosis: {kurt:.2f}")

# Normal distribution: skewness ≈ 0, kurtosis ≈ 0

Interpretation:

Skewness:

• ≈ 0: Symmetric (normal)
• > 0: Right-skewed (long right tail)
• < 0: Left-skewed (long left tail)
• |skew| > 1: Highly skewed (not normal)

Kurtosis (excess kurtosis):

• ≈ 0: Normal tails (mesokurtic)
• > 0: Heavy tails (leptokurtic — more outliers)
• < 0: Light tails (platykurtic — fewer outliers)
• |kurt| > 3: Very different from normal

When Data is NOT Normal

Options:

1. Transform Data (make it more normal):

• Log transformation: For right-skewed data (income, order values)
• Square root: For moderate right skew
• Box-Cox: Automated optimal transformation

import numpy as np

# Log transformation
data_log = np.log(data + 1)  # +1 to handle zeros

# Check if transformed data is more normal
stats.probplot(data_log, dist="norm", plot=plt)

2. Use Non-Parametric Tests (don't assume normality):

• Mann-Whitney U (instead of t-test)
• Kruskal-Wallis (instead of ANOVA)
• Spearman correlation (instead of Pearson)

3. Use Percentiles (instead of mean ± SD):

• Report median (P50), IQR (P25-P75), P95, P99
• Percentiles work for ANY distribution

4. Central Limit Theorem (for large samples):

• Even if data is not normal, the MEAN of samples is approximately normal (for n > 30)
• Allows t-tests on non-normal data if sample size is large

Application 1: A/B Test Sample Size Calculation

Context: Planning A/B test for Flipkart homepage redesign.

Assumptions:

• Current conversion: 2.5%
• Want to detect 10% relative lift (2.5% → 2.75%)
• Significance: α = 0.05, Power: 80%

Normal Distribution Used: Sample proportions are approximately normally distributed (by CLT).

Formula (simplified):

Action: Run test with 62K users per variant to reliably detect 10% lift.

Application 2: Quality Control (Manufacturing)

Context: Electronics factory produces resistors (target: 100 ohms).

Data:

Calculate Defect Rate:

Control Chart: Monitor daily mean and SD. Alert if:

• Mean shifts (μ ≠ 100 ohms → recalibrate machine)
• SD increases (σ > 1.5 ohms → machine wearing out)

Application 3: Risk Management (Finance)

Context: Portfolio returns normally distributed.

Data:

Risk Metrics:

Value at Risk (VaR) — Maximum loss at 95% confidence:

Expected Shortfall — Average loss in worst 5% scenarios:

Application 4: Grading on a Curve

Context: Professor wants to assign letter grades based on normal distribution.

Data:

Grading Cutoffs (using Z-scores):

Result: Consistent grading regardless of exam difficulty (normalized to class performance).