Beginner · Fundamentals

Feature Engineering

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The process of creating, selecting, and transforming input variables to improve a machine learning model's performance.

Technical Definition

The process of creating, selecting, and transforming input variables to improve a machine learning model's performance.

How it works

Feature engineering transforms raw data into informative inputs for ML models. This includes creating new features (e.g., extracting day-of-week from timestamps), normalizing values, encoding categorical variables (one-hot, label encoding), and selecting the most relevant features. Good feature engineering can dramatically improve model performance and often matters more than algorithm choice. Deep learning has partially automated this through learned representations.

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Embedding — A dense vector representation that captures semantic meaning, mapping discrete items like words into continuous mathematical space.
• Deep Learning — A subset of machine learning using neural networks with many layers to learn hierarchical representations from large datasets.