Detecting Missing Data

What is Missing Data?

Missing data means empty cells in your table. In pandas, missing values show as NaN (Not a Number).

Real data often has missing values because:

Someone forgot to fill a form
A sensor stopped working
Data was lost during transfer

How to Check for Missing Values

code.py

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'Name': ['John', 'Sarah', None, 'Mike'],
    'Age': [25, None, 30, 28],
    'City': ['NYC', 'LA', 'Chicago', None]
})

print(df)

Output:

    Name   Age     City
0   John  25.0      NYC
1  Sarah   NaN       LA
2   None  30.0  Chicago
3   Mike  28.0     None

Check Each Cell: isna()

code.py

print(df.isna())

Output:

    Name    Age   City
0  False  False  False
1  False   True  False
2   True  False  False
3  False  False   True

True = missing, False = has value

Count Missing Values

code.py

# Missing per column
print(df.isna().sum())

Output:

Name    1
Age     1
City    1
dtype: int64

Each column has 1 missing value.

Total Missing Values

code.py

total_missing = df.isna().sum().sum()
print(f"Total missing: {total_missing}")

Output: Total missing: 3

Percentage of Missing

code.py

pct_missing = (df.isna().sum() / len(df)) * 100
print(pct_missing)

Output:

Name    25.0
Age     25.0
City    25.0
dtype: float64

25% of each column is missing.

Quick Summary

code.py

# All info at once
print(df.info())

This shows "non-null count" for each column.

Find Rows with Missing Values

code.py

# Rows that have ANY missing value
rows_with_missing = df[df.isna().any(axis=1)]
print(rows_with_missing)

Key Points

NaN = missing value in pandas
isna() or isnull() checks for missing (both work same)
isna().sum() counts missing per column
info() shows quick summary of data

What's Next?

Now you can find missing data. Next, learn how to remove rows or columns with missing values.