Goodhart's Law isn't as useful as you might think
Cedric Chin, writing about the famous adage that says “when a measure becomes a target, it ceases to be a good measure.”
At some level, this is self-evident. Goodhart’s Law is about as pithy and about as practicable as “the only certainty in life is death and taxes” and “hell is other people.” It is descriptive; it tells you of the existence of a phenomenon, but it doesn’t tell you what to do about it or how to solve it.
There are more useful formulations of the law that help solve it. Some organizations use this narrower, more actionable form:
When people are pressured to meet a target value there are three ways they can proceed:
- They can work to improve the system
- They can distort the system
- Or they can distort the data
With that framing in hand, Cedric goes on to describe how you can solve for these three ways and design incentive/measurement systems that work.