Intermediate · Systems

Model Compression

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Techniques for shrinking models while preserving most of their performance.

Technical Definition

Techniques for shrinking models while preserving most of their performance.

How it works

Compression combines quantization (lower-precision weights), pruning (removing weights), distillation (training a small model to mimic a large one), and weight sharing. Crucial for edge deployment, faster inference, and reduced serving cost.

Related Concepts

• Knowledge Distillation — Compressing a large teacher model into a smaller student model by training the student to mimic the teacher's outputs.
• Quantization — Reducing numerical precision of model weights (e.g., 32-bit to 4-bit) to shrink size and speed up inference.
• Pruning — Removing redundant weights or neurons from a network to reduce size and improve inference speed.