I am so glad to see our government back down on robodebt this week. They will no longer automatically raise a debt on
welfare recipients based on data matching, instead Centrelink staff should review the data and investigate before
accusing people of owing money. Perhaps hundreds of thousands of Australians were falsely accused of
taking money they weren't entitled to, causing immense stress and hardship for no good reason.
My question now is how do we prevent this happening again? Inevitably another scheme will come up as 'dole bludgers'
are a perennial scapegoat in Australia. How do we prevent a future scheme from making so many mistakes and causing so
much suffering?
I think the answer is that we need to foster a healthy mistrust of data. People seem to think that because a computer
says something it must be true. They don't give enough creedence to the possibility of mistakes and misinterpretations.
The reason robodebt got it so wrong was perhaps somewhat due to mistakes in data, but probably mainly due to a fundamental
misinterpretation of what Australian Tax Office income data means.
Having data is a good thing — better that we make judgements based on data than on guesses and personal prejudices.
We should be encouraging government and society to use more data to be more fair and more effective.
We just have to be careful to not accept that data uncritically. All data of a reasonable size has mistakes in it.
That's OK! Even with mistakes it may well be better than nothing. The problem is that people aren't conditioned to
think of data being wrong so they put too much faith in it. I think we should reverse that: we should always assume data
is wrong, not completely wrong but always partially wrong.
I reckon the designers of the robodebt scheme put too much faith in their data. If they had properly acknowledged the
risks of mistaken and misinterpreted data then I don't think it would ever have been launched.