Intermediate · Fundamentals
Markov chain
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A model where future states depend only on the current state, not on past events, used for sequences of events with probabilistic transitions.
Technical Definition
A model where future states depend only on the current state, not on past events, used for sequences of events with probabilistic transitions.
How it works
A Markov chain is a mathematical system that transitions from one state to another based solely on the current state. The probability of moving to the next state is independent of the sequence of states that preceded it. This property makes them useful for modeling systems that evolve over time, like weather patterns or stock prices.
Related Concepts
- Reinforcement Learning — A paradigm where an agent learns to make decisions by receiving rewards or penalties from its environment through trial and error.