This Puzzle Shows Just How Far LLMs Have Progressed in a Little Over a Year

The capabilities of large language models have progressed significantly in a little over a year, as demonstrated by a geometric puzzle. The task involves writing a Python program to find all distinct squares that can be drawn on a cross-shaped grid of dots. While GPT-4o required two hours and over 40 iterations to produce a correct solution in 2024, the newer Claude Sonnet 4.5 generated a complete and correct Python program in just five seconds. Although Sonnet 4.5 initially failed to reason the correct answer directly, its code generation capabilities proved far superior and faster than the previous model's.