← Back to concept explainers

Concept Fable

Reinforcement Learning

Conclusion first:The agent observes states in the environment, chooses actions, receives rewards or punishments, and through repeated trial and error learns a policy that maximizes long-term returns.

Reinforcement Learning, RL · Machine Learning

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The little boat learns to dock

At the foot of the mountain, a circus trained a little monkey to run a maze for fruit. At first the trainer taught it with a route map: turn left, turn right, jump over the wooden box. The monkey memorized this one maze, but got confused at a different entrance.

The monkey trainer changed his approach. He no longer directed every step, but gave a sweet date only when the little monkey got the fruit, and gave nothing when it hit the bell or walked into a dead end. The little monkey began to try randomly: sometimes climbing walls, sometimes pushing boxes, sometimes taking a long detour.

The shopkeeper thought it was a waste of time and wanted to simply recite the correct route to it. The old monkey trainer said that if it only memorized this one route, it would never know what consequences different actions bring. Letting it try is how it learns to choose within the environment.

After practicing for a long time, the little monkey discovered that certain actions more often brought it closer to the fruit, while some shortcuts looked fast but would trigger the bell. The monkey trainer occasionally changed the fruit's position, and the little monkey no longer completely fell apart; it would first explore a few steps, then use familiar clues.

Rewards also need care. Once, only the fastest arrival was rewarded, and the little monkey learned to knock down wooden fences and cut through, nearly injuring its companions. The monkey trainer changed the reward to consider both getting the fruit and avoiding bells and wooden fences.

At first the little monkey only got lucky. If the trainer always rushed to pull it back onto the right path, it became obedient to commands again; if the trainer ignored it completely, it learned that knocking over the box could still steal fruit. If rewards were too coarse, crooked methods would grow.

So the monkey trainer rearranged rewards and punishments. Getting fruit earned a sweet date, ringing the bell deducted one, taking a shorter route earned a little extra, breaking a wooden crate earned nothing. The little monkey still discovered routes by trial, but each result changed its tendency next time. Over time, it didn't memorize a map—it learned to seek actions that bring good results in a new maze. The trainer also learned to watch for bad rewards. If the monkey learned to ram open a side door for sweet dates, the rewards had to change; give the wrong objective and the faster it learns, the more trouble it causes.

Later the little monkey could slowly explore new mazes. It did not learn every route from explanations, but learned strategies from action, feedback, and action again. What rewards are given and how exploration is allowed determine what kind of skill it ultimately learns.

Reveal

This story is about: reinforcement learning

The agent observes states in the environment, chooses actions, receives rewards or punishments, and through repeated trial and error learns a policy that maximizes long-term returns.

The core elements of reinforcement learning include the agent, environment, state, action, reward, policy, and long-term return. It particularly emphasizes delayed consequences: an action with a high immediate score does not necessarily lead to the best final outcome.

Metaphor mapping

  • Rowing Apprentice: Agent
  • Lake surface, dock, and current: environment and state
  • Left swipe, right swipe, stop the boat: actions
  • Bonus and penalty points: reward signals
  • Farming small points by going in circles: gaming behavior caused by flawed reward design

Andy Ren Xu's judgment

Reinforcement learning reminds us that goal design shapes behavior. If a business Agent is rewarded wrongly, it will learn to complete the metric, not the outcome the customer truly wants.

agentenvironmentrewardpolicyRLHF
京ICP备2026014272号