Advanced · Fundamentals

Federated Learning

Visual diagram · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Distributed training where models learn from data on many devices without the data ever leaving those devices.

Technical Definition

Distributed training where models learn from data on many devices without the data ever leaving those devices.

How it works

Each client trains on local data and sends only model updates to a central server for aggregation. Google uses this for keyboard prediction. Challenges include non-IID data, communication efficiency, and privacy guarantees.

Visual Explanation (flowchart)

Central Server → Distribute Model → [Device A, Device B, Device C train locally] → Send Updates → Aggregate → Updated Global Model

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Gradient Descent — An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease of the loss function.
• Deep Learning — A subset of machine learning using neural networks with many layers to learn hierarchical representations from large datasets.