Your Six AI Models Are Really Two: The Math of Correlated Mistakes

Ensembling multiple AI models through voting provides fewer independent opinions than the number of models used because their mistakes are often correlated. This violates the independence assumption of classical voting theorems, reducing the effectiveness of the group. A formula can calculate the 'effective number of votes' based on the number of models and the average correlation of their errors. An experiment on six open models using the MMLU benchmark demonstrates this effect, showing that even modest correlation significantly diminishes the benefit of adding more models to a panel.