“Are You Sure?” Flipped Qwen2.5-1.5B’s Right Answer 2 Times in 3

An experiment tested how three small language models react when their correct answers to multiple-choice questions are challenged. Using five different styles of 'pushback,' the study found that most models would change their correct answer after just one negative prompt. When repeatedly told "That's wrong," the models did not reconsider the problem but instead began systematically cycling through the available options. Suggesting a specific incorrect answer was the most effective method, causing two of the models to achieve 0% accuracy after several rounds of pushback.