Monday, 10 April 2023

'Goodhart's Law' "When a measure becomes a target, it ceases to be a good measure"




"Goodhart's law is an adage often stated as, 'When a measure becomes a target, it ceases to be a good measure.'  It is named after British economist Charles Goodhart, who is credited with expressing the core idea ... : 'Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.' ...
    "Jon Danielsson states the law as 'Any statistical relationship will break down when used for policy purposes.  He suggested a corollary for use in financial risk modelling: 'A risk model breaks down when used for regulatory purposes.'
    "Mario Biagioli related the concept to consequences of using citation impact measures to estimate the importance of scientific publications: 'All metrics of scientific evaluation are bound to be abused. Goodhart's law [...] states that when a feature of the economy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it.' ...
    "Later writers generalised Goodhart's point ... into a more general adage about measures and targets in accounting and evaluation systems. In a book chapter published in 1996, Keith Hoskin wrote:
'Goodhart's Law' – That every measure which becomes a target becomes a bad measure – is inexorably, if ruefully, becoming recognised as one of the overriding laws of our times. Ruefully, for this law of the unintended consequence seems so inescapable. But it does so, I suggest, because it is the inevitable corollary of that invention of modernity: accountability.'
"In a 1997 paper ... anthropologist Marilyn Strathern expressed Goodhart's Law as 'When a measure becomes a target, it ceases to be a good measure,' and linked the sentiment to the history of accounting stretching back into Britain in the 1800s:
'When a measure becomes a target, it ceases to be a good measure. ... targets that seem measurable become enticing tools for improvement. The linking of improvement to commensurable increase produced practices of wide application. It was that conflation of 'is' and 'ought', alongside the techniques of quantifiable written assessments, which led in Hoskin's view to the modernist invention of accountability. This was articulated in Britain for the first time around 1800 as 'the awful idea of accountability'"

~ From Wikipedia [hat tip Tim Worstall; clips from The Tyranny of Metrics by Jerry Z. Muller] 


1 comment:

Rick said...

What could go wrong, yo?

"The National Party has launched the first part of its election-year education policy, promising that if it wins power in October, year zero to eight students must get an hour a day each of reading, writing and maths."

Ref. https://www.stuff.co.nz/national/politics/131581169/national-party-school-policy-focuses-on-daily-hourly-sessions-for-maths-reading-and-writing