The Reversal Curse: Why a Language Model That Knows “A Is B” Can’t Tell You “B Is A”

Language models exhibit a surprising logical blind spot known as the "Reversal Curse." This means a model trained on a fact like "A is B" can easily recall it but often fails completely when asked to retrieve the same information in the opposite direction, "B is A." To test this, a simple model was built from scratch and trained on fictional facts presented in only one direction. The results were stark: the model showed perfect recall for facts in the trained direction but had zero accuracy when quizzed on their reverse. Strikingly, even significantly increasing the model's size and complexity failed to fix the issue, suggesting the problem is fundamental to how these models learn.