Bad metrics, bad outcomes
The broader message is that it is easy to set a metric, but the metric becomes the north-star goal toward which people’s behavior and effort get oriented. The person or the organization or the entity setting the metric may just be thinking about reaching the number. But the metric matters so much that it becomes a really hard task to set it straight, and people and organizations can end up doing very weird things to achieve it.
A story. A king of a small town, in some random place, thousands of years past, had a weird problem. After the rainy season the town was suddenly flooded with snakes. It created real problems: the entire population was scared to step out, and the town was suffering.
The king had to do something, and he decided to create an incentive to reduce the number of snakes. He announced: whoever brings one snake gets five gold coins. This incentivized the snake charmers and the people versed in the art of catching snakes to earn some easy money, and things began to stabilize after a while. These people were the heroes of the town, everyone cherished them, they made some money, and the town was getting rid of snakes. Everyone was happy - a classic win-win-win situation. More on that idea in a future post.
But markets have cycles, and this was no outlier. Just like a monopoly is hard to maintain - wherever there is margin, more players enter, margins become thinner, and the market eventually stabilizes at a point where there is just enough profit for the people in it to continue - more and more people started learning the art of catching snakes. Or rather, more people started putting their brains into solving the metric to get more money. And one of the ways they found was to just start breeding snakes.
Every house in the town started breeding snakes and handing them to the king. Breeding was way more profitable than catching, because the reward per snake was fixed. So even after all the wild snakes were actually caught, people had multiple pet snakes at home that they were breeding just to hand over to the king. The entire problem had reversed: it was now in the people’s interest to keep the town full of snakes, and the king was paying for every single one of them.
The point is that all of this happened because a bad metric was set, and people always game the metric to get the reward.
Let’s shift gears and talk about politics. In many countries, politicians offer a certain economic or political benefit to a certain segment of the population to get their favor, get votes, and stay in power or win the next election. It has happened in various forms across many countries - it is not a new thing. The shape is different, but the broad strokes are the same.
India has a reservation system. The government allows people belonging to certain castes - officially called Scheduled Castes and Scheduled Tribes, or SC/ST - to clear a lower bar in the entrance exams to get the same college seat that a person in the General category needs much higher marks to secure. This was historically created because one segment of the population was not progressive, and the government wanted to give them a head start by lowering the minimum scores required, in the hope that over time the overall community would improve. The intention was correct.
The dates: caste-based reservations in education began with the First Amendment in June 1951, which added Article 15(4) after the Supreme Court’s Champakam Dorairajan judgment struck down the earlier quota system. The 27 percent OBC quota reached central government jobs in 1990 under V.P. Singh, and central educational institutions with the 2006 Act, effective from the 2007-08 session - upheld by the Supreme Court in 2008. The 10 percent EWS quota came in January 2019 and survived a 3-2 Supreme Court challenge in 2022.
And the share of seats reserved in central institutions has only moved in one direction: 22.5 percent in 1951, 49.5 percent once OBCs were added, 59.5 percent once EWS was - against a 50 percent ceiling the Supreme Court had fixed in 1992, which the EWS quota was carved around rather than counted under.
Sources: the First Amendment, 1951, the Central Educational Institutions (Reservation in Admission) Act, 2006, the Ashoka Kumar Thakur judgment, 2008, PIB on the EWS quota, and coverage of the 2022 EWS verdict.
But over time this has led to a situation where a student who scored higher does not get the seat, and has to surrender it to someone who scored lower in the same exam - just because of the quota they were born into. The intention of the policy was good, and it does help a certain set of people. But the data tells a harder story. The Justice Rohini Commission found that 97 percent of central OBC-quota jobs and education seats went to just 25 percent of OBC castes (Indian Express). In 2024, a seven-judge Supreme Court bench allowed sub-classification within SC/ST quotas precisely because capture inside the groups had become undeniable, with several judges writing that a creamy-layer exclusion should evolve (Supreme Court Observer). And in August 2026 the Union government told the Supreme Court it opposes exactly that exclusion for SC/STs (The Hindu, Explained). The pattern is consistent: within every beneficiary group, the better-off slice captures most of the benefit - not the poorest members the policy was meant for. The intention behind the policy was not captured by the metric.
There are multiple other examples of the same pattern, in all directions. The point stands: once you have a bad metric, you will not get the intended outcome. It is very important to choose the right metric, correctly.
This is also, at its core, about incentive structures - but that deserves its own post.