The Pok\'emon Theorem: Fairness Criteria Can't Catch 'Em All

원문 초록

Debates around algorithmic fairness rarely end. Models considered "fair" by one criterion are soon shown to be unfair by another. We prove that this is unavoidable. Satisfying any finite checklist of fairness criteria never certifies fairness itself, since the model may always violate yet another criterion. We call this the Pok\'emon theorem: you can't catch them all. In fact, trying to catch all criteria recovers the classical impossibility theorems. Since these violations come from differences between the groups' data, a natural remedy would be to make the data fair by learning fair features. Unfortunately, this, too, is impossible: features that scrub out sensitive group information also scrub out all predictive signal. While exact fairness is out of reach, approximate fairness is not. We derive the Pareto frontier between fairness and predictive signal for approximately fair features, and show how to attain it.