Intermediate · Reinforcement Learning

Q-Learning

Visual diagram · (in preparation) · Math · Worked example · 3 difficulty levels.

TL;DR. A model-free RL algorithm that learns action values without knowing environment dynamics.

Technical Definition

A model-free RL algorithm that learns action values without knowing environment dynamics.

How it works

Q-Learning maintains a Q-table estimating expected cumulative reward for each state-action pair. Deep Q-Networks (DQN) replace the table with a neural network for high-dimensional state spaces like Atari games.

Mathematical Notation

Q(s,a) ← Q(s,a) + α[r + γ max_a' Q(s',a') − Q(s,a)]

The Q-value is updated toward immediate reward r plus discounted best future value. α controls update speed.

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Reinforcement Learning — A paradigm where an agent learns to make decisions by receiving rewards or penalties from its environment through trial and error.