Intermediate · Reinforcement Learning

Reward Shaping

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Designing or modifying the reward signal to guide an RL agent toward desired behavior more efficiently.

Technical Definition

Designing or modifying the reward signal to guide an RL agent toward desired behavior more efficiently.

How it works

Reward shaping adds intermediate rewards to sparse environment signals. Poorly designed shaping can lead to reward hacking. Potential-based reward shaping guarantees the optimal policy remains unchanged.

Related Concepts

• Reinforcement Learning — A paradigm where an agent learns to make decisions by receiving rewards or penalties from its environment through trial and error.
• Q-Learning — A model-free RL algorithm that learns action values without knowing environment dynamics.
• Policy Gradient — RL algorithms that directly optimize the policy function by gradient ascent on expected reward.