Intermediate · Systems

On-device AI

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. AI models that run directly on edge devices (e.g., smartphones, IoT) rather than relying on cloud servers for inference.

Technical Definition

AI models that run directly on edge devices (e.g., smartphones, IoT) rather than relying on cloud servers for inference.

How it works

On-device AI offers several benefits, including reduced latency, enhanced privacy (as data stays local), and lower reliance on network connectivity. This requires significant model optimization techniques like quantization and pruning to fit powerful AI models onto resource-constrained hardware. It's crucial for applications requiring real-time processing and offline functionality.

Related Concepts

• Federated Learning — Distributed training where models learn from data on many devices without the data ever leaving those devices.
• Edge AI — Running AI models directly on user devices instead of cloud servers.
• Model Quantization — A technique to reduce the memory footprint and computational cost of AI models by representing their weights and activations with lower precision numbers.