Advanced · Reinforcement Learning
Bellman equation
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A formula in reinforcement learning that defines the value of an action based on immediate rewards and future expected rewards, used in Q-learning.
Technical Definition
A formula in reinforcement learning that defines the value of an action based on immediate rewards and future expected rewards, used in Q-learning.
How it works
The Bellman equation is a fundamental concept in reinforcement learning that relates the value of a state-action pair to the immediate reward received and the expected value of subsequent states. It's crucial for algorithms like Q-learning, which use it to iteratively update their estimates of action values.
Related Concepts
- Reinforcement Learning — A paradigm where an agent learns to make decisions by receiving rewards or penalties from its environment through trial and error.
- Q-Learning — A model-free RL algorithm that learns action values without knowing environment dynamics.