How to Make Your First World Model from Scratch

The CartPole problem is a classic benchmark in reinforcement learning, solvable by a Deep Q-Network (DQN) that learns to balance a pole on a cart. However, this standard approach has significant limitations because the agent's goal is immutably encoded during training through the reward function. Attempting to give a trained agent a new goal, such as parking the cart at a specific location, fails because the agent has no mechanism to understand the new request. This inflexibility demonstrates that changing objectives requires complete, and often costly, retraining from scratch, revealing that the agent learns which actions are rewarding rather than the underlying dynamics of the environment.