From the algorithm that raises your insurance premium to institutional denials over state scandals, it’s a problem with deep roots
I was recently scheduled to present my Cautionary Tales podcast live on stage as a curtainraiser for a podcast conference. Talented voice actors, live sound effects and even a trombone would weave a dramatic soundscape for a full house. There was only one problem. Nobody seemed to have the authority to let me in.
The people issuing conference passes wouldn’t give me one — not unreasonably, since I wasn’t attending the conference proper and hadn’t registered to do so. They advised that I speak to “that lady there”. That lady there seemed puzzled: the conference wouldn’t officially open until tomorrow, so I didn’t need a pass. Just walk in, she said. The security guard had a different view. He had been given strict instructions that nobody gets in without a pass. Talk to the conference team, he told me. Nothing to do with us, they said — talk to security.
I realised I was facing what the writer Dan Davies has named an “accountability sink”, in which it was somehow nobody’s fault. In his recent book The Unaccountability Machine, Davies explains the basic logic of an accountability sink: decision-making power is removed from individuals you might want to shout at, and made instead by an algorithm or some distant committee both ignorant of and immune to your objections.
Everyone I spoke to insisted that they were powerless to act. And if you cannot change a decision, you cannot be held accountable for it. That’s the accountability sink at work.
“Computer says no” first became a comedy catchphrase 20 years ago, so although the accountability sink is a useful new term it describes an older problem. Just ask 440 luckless squirrels who in 1999 arrived at Schiphol airport in the Netherlands, en route from Beijing to Athens. Unfortunately, particularly for the squirrels, they had been shipped without the right paperwork.
What to do? It wasn’t legal to send them on to Athens. It wasn’t possible, apparently, to send them back to Beijing or to fix the paperwork. So the airport ground staff started up an industrial shredder normally used in poultry farms for killing male chicks and threw all 440 of the squirrels into it.
When this debacle became known to the public, it was universally agreed that shredding 440 squirrels had been The Wrong Thing To Do, but it was less clear who should be accountable. The basic problem, Davies explains, was that the decision to destroy paperwork-less animals had been turned into a matter of policy emerging from the Dutch department of agriculture. Policy is policy, not something that gets overruled by manual workers in airport sheds wearing protective gloves and standing next to an industrial shredder and a few crates of squirrels.
The policy itself was probably sensible. The problem was that there was no way to identify and deal with exceptional cases. The question, “What if the policy demands that we hurl hundreds of squirrels into an industrial shredder?” arose too late.
Accountability sinks can have a legitimate purpose. When the pressure is on, it’s tempting to play favourites or take short-cuts, prioritising the noisiest complaints or putting off painful consequences. “Computer says no” may be a punchline, but the computer is a useful way to deflect awkward customers. It also often makes the right choice.
Consider the idea of an independent monetary policy, when an elected government sets an inflation target and then an unelected central bank has the task of trying to hit that target. It’s a sensible response to the fact that democratically elected governments always have a short-term incentive to stoke inflation, reaping the briefest of benefits followed by some serious headaches.
When the Bank of England hurts millions by raising interest rates, it can point to its inflation mandate. When people complain to the government, the government can say that interest rates are a matter for the Bank of England. It’s an accountability sink again, but monetary policy is almost certainly better as a result of the fact that somehow nobody is to blame.
Yet all too often accountability sinks are created for the simple, ignoble reason that nobody wants to be held accountable if they can avoid it. Even worse, the sink may be fictional, a mere scapegoat. A generation of British politicians learnt that whenever anyone complained about a policy, it was always easy to blame Brussels, or unelected judges, or health and safety gone mad.
Accountability sinks range from the algorithm that raises your insurance premium to the institutional denial that has delayed many reckonings for the British state: infected blood, persecution of sub-postmasters, Hillsborough, the Windrush scandal and so many others. Faced with such an enormous range of misfiring accountability sinks, is there anything useful we can say about what we could change to make things better?
Perhaps there is. The common thread here is an imperviousness to feedback. No doubt the people involved in the squirrel incident wondered if there might be a better way to deal with paperwork problems, but there was no line of communication open to the relevant ministry, unaware of the gruesome consequences. Without feedback, nothing ever gets better.
The world is a complicated place and humans are imperfect. Mistakes will be made. The question is, what happens after the mistake? Does news ever reach anyone with the ability to change things? If so, do they? Some accountability sinks exist for a good reason, some do not. Either way, if there is no way to learn from mistakes, the mistakes will eventually bury us all.
Recent Comments