Goodhart’s law, perverse incentives, and rats with no tails

1902 was a big year for rats in Hanoi, Vietnam.

The city had spent the past few years building a city-wide sewer system, and toilets with running water became a symbol of progress and status. Of course, all those miles of underground sewage pipe were predator-free real estate for rats, so they multiplied.

When the rats got hungry, they’d follow the sewage lines to the surface. Those high status toilets didn’t seem so desirable once rats started popping out, bringing everything from gross ickyness to bubonic plague.

So the government created a bounty program. Bring in the severed tail of a rat and get paid, simple as that. The Great Hanoi Rat Massacre picked up quickly, and at its peak, citizens killed over 20,000 rats in a single day. So far so good.

But people are greedy. They started cutting the tails off and then releasing the rats back into the wild to breed. More breeding leads to more rats which leads to more tails which leads to more money. Simple economics. Some people even threw together makeshift rat breeding operations.

And just like that, Hanoi’s rat problem was worse than ever.

The rat bounty is an example of a perverse incentive. Whenever you see a reward that ends up encouraging the opposite of the behavior it was aiming for, you know you’ve got a perverse incentive on your hands.

The classic example in the development world is our friend the velocity goal. Misguided managers sometimes set a velocity goal in an effort to increase the productivity and value of the team. Usually it’s something like: “Let’s aim for 25 story points every sprint!” Or even worse, maybe it’s at an individual level: “Everyone aim for 10 story points per sprint!”

Of course people start “breeding rats” to game the system. They’ll over-inflate the estimates. They’ll tackle a ton of easy 1-pointer bug tickets instead of picking up the meatier work. They’ll do things fast instead of right. They’ll ignore their blocked coworkers because they’re too focused on hitting their own goals. This is not the way to increase the productivity and value of the team. A velocity goal is a perverse incentive.

Goodhart’s law is one of my all time favorites. Here’s the simplified version:

When a measure becomes a target, it ceases to be a good measure.

Because remember, velocity itself isn’t bad. It’s useful for forecasting, for example. But the second we turn the velocity measure into a target, it stops being a good measure. People game the system in an attempt to hit the target, and the value of the measure gets lost along the way.

Automated testing code coverage also comes to mind. As a measure, sure, it’s useful. But when a team decides something like “we need 80% code coverage” then it stops being useful. It doesn’t take long for the tail-less rats to start breeding. Then we’re left with tests that aren’t valuable but contribute to the coverage goal.

Those pointless tests take time to run and someone has to maintain them. So instead of making automated testing more useful, we’ve made it less useful. Perverse incentive.

It’s a sticky wicket, Goodhart’s law. Obviously, some things need to be targets, right? What is a company without goals? But how can we make sure we’re not creating perverse incentives?

The silver bullet is a ruthless adherence to outcome-based goals over output-based goals. Outcomes are what the business wants or needs to achieve. Outputs are the actions that help reach an outcome. We want goals to be based around what the business wants, not the steps we think will get us there.

Velocity goals and code testing coverage goals are output based. They’re the steps we think will get us to the outcomes we want. What are the outcomes we want? Let’s build the goals around those.