I Trained a Tiny Network to Compress Data. It Drew a Pentagon.

A small neural network is trained from scratch in NumPy to reproduce an experiment on model interpretability from an Anthropic paper. The goal is to understand how a network can represent more features than it has dimensions, a concept known as superposition. This phenomenon is proposed as an explanation for polysemanticity, where single neurons respond to multiple unrelated concepts. When compressing five features into two dimensions, the network's learned representations form a perfect pentagon. This geometric structure emerges naturally from gradient descent as the optimal way to pack the features and minimize interference, without being explicitly programmed.