Playing Connect Four with Deep Q-Learning

Reinforcement learning is applied to the game of Connect Four using Deep Q-Learning (DQN) to overcome the limitations of tabular methods in complex environments. The approach transitions from on-policy Sarsa to an off-policy, batched training setup with a replay buffer, which improves computational efficiency and stabilizes learning. To increase data throughput, a vectorized environment wrapper is implemented to simulate multiple games in parallel. The resulting DQN agent learns significantly faster than previous methods and can defeat a random policy, but it plateaus and exhibits weak defensive skills against a human player. The agent successfully learns offensive strategies but fails to anticipate and block opponent threats.