Reinforcement Learning: From Board Games to Real Robots — AlphaGo, AlphaStar, and Robot Manipulation
RL’s mathematical framework is the Markov Decision Process (MDP): an agent observes state, takes action, receives reward, and transitions to the next state. The goal: learn a policy maximizing long-term cumulative reward. Unlike supervised learning’s labeled data requirement, RL only needs a reward function — the agent generates its own training data through environment interaction (self-play or simulation).
AlphaGo and AlphaZero: Board Game Breakthroughs
AlphaGo (DeepMind, 2016) is RL’s most widely known milestone: defeating world champion Lee Sedol 4:1 in Go — a game assumed to require human-like “intuition.” Go’s strategy space (~10^170) far exceeds chess (~10^120), making brute-force search infeasible.
AlphaGo Zero (2017) learned Go purely from self-play without any human game data, surpassing all prior versions. AlphaZero generalized the same algorithm to chess and Shogi, decisively outperforming traditional AI engines — demonstrating the generalization ceiling of self-play RL. Core technical combination: deep neural networks (Value Network + Policy Network) + Monte Carlo Tree Search (MCTS) + self-play RL — widely applied to other complex decision problems since.
AlphaStar: Real-Time Strategy Games
Real-time strategy (RTS) games add extra challenges: imperfect information (fog of war), continuous high-dimensional action space (hundreds of possible actions per frame), and long-horizon strategic planning. AlphaStar (DeepMind, 2019) reached Grandmaster level in StarCraft II, surpassing 99.8% of human players.
Robot Learning: Sim-to-Real Transfer
RL in robotics faces the Sim-to-Real Transfer challenge — policies trained in simulators often fail to transfer directly to the real physical world (robot dynamics differences, sensor noise, contact physics). Boston Dynamics robot locomotion control, and humanoid robot companies Figure and 1X are combining LLMs (high-level task understanding) with RL (low-level motion control). OpenAI’s Dactyl demonstrated purely RL-trained robot hand solving a Rubik’s Cube — a key dexterous manipulation reference.
