Stop Asking if a Model Is Interpretable

The common question of whether a model is interpretable is flawed because interpretability is not an inherent, binary property. Instead, it should be viewed as a set of methods that allow humans to analyze models to answer specific questions. These methods serve three distinct functions: diagnosing model failures during development, validating that a successful model relies on correct reasoning, and extracting new knowledge in scientific domains like medical imaging. By reframing the question to focus on what an explanation should explain, interpretability transforms from a vague requirement into a precise scientific tool.