I am so glad to see our government back down on robodebt this week. They will no longer automatically raise a debt on welfare recipients based on data matching, instead Centrelink staff should review the data and investigate before accusing people of owing money. Perhaps hundreds of thousands of Australians were falsely accused of taking money they weren't entitled to, causing immense stress and hardship for no good reason.

My question now is how do we prevent this happening again? Inevitably another scheme will come up as 'dole bludgers' are a perennial scapegoat in Australia. How do we prevent a future scheme from making so many mistakes and causing so much suffering?

I think the answer is that we need to foster a healthy mistrust of data. People seem to think that because a computer says something it must be true. They don't give enough creedence to the possibility of mistakes and misinterpretations. The reason robodebt got it so wrong was perhaps somewhat due to mistakes in data, but probably mainly due to a fundamental misinterpretation of what Australian Tax Office income data means.

Having data is a good thing — better that we make judgements based on data than on guesses and personal prejudices. We should be encouraging government and society to use more data to be more fair and more effective. We just have to be careful to not accept that data uncritically. All data of a reasonable size has mistakes in it. That's OK! Even with mistakes it may well be better than nothing. The problem is that people aren't conditioned to think of data being wrong so they put too much faith in it. I think we should reverse that: we should always assume data is wrong, not completely wrong but always partially wrong.

I reckon the designers of the robodebt scheme put too much faith in their data. If they had properly acknowledged the risks of mistaken and misinterpreted data then I don't think it would ever have been launched.