Beginner · Data

Data Preprocessing

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Transforming raw data into a format suitable for model training.

Technical Definition

Transforming raw data into a format suitable for model training.

How it works

Preprocessing handles missing values, scales numeric features, encodes categories, tokenizes text, resizes images, and removes duplicates. Consistent preprocessing between training and inference is critical — even tiny mismatches can silently degrade model accuracy in production.

Related Concepts

• Feature Engineering — The process of creating, selecting, and transforming input variables to improve a machine learning model's performance.
• Data Pipeline — An automated workflow that ingests, transforms, and delivers data to models.
• Data Cleaning — Identifying and fixing or removing errors, inconsistencies, and duplicates in a dataset.