It’s a sad running gag that the best way for a Kiwi to get noticed at home is to be successful abroad.
The keynote lecture at this year’s New Zealand Economics Association conference was downright depressing. Success abroad may be necessary but seems far from sufficient.
And sometimes the stakes are incredibly high.
Auckland University of Technology’s Professor Rhema Vaithianathan told the assembled economists about work she has been doing in the United States.
Professor Vaithianathan and her team specialise in using administrative data – the data that the government and its agencies collect as part of doing business.
Allegheny County, around Pittsburgh, Pennsylvania, wanted to know whether better data could help front-line child protection service workers.
As Professor Vaithianathan explained, child protection workers have about ten minutes to decide whether to send in a case officer to assist a family, or whether to triage a report out of the system.
Those decisions are incredibly high-stakes.
Fail to send in a case officer when one was needed, and a child could face lifetime consequences of neglect, abuse, or injury. Or worse.
Intervene when intervention was not needed, and a family faces unpleasant interactions with an incredibly powerful state agency. At best, they have a stressful few weeks hanging under the terrible implicit threat of a child being removed from the family. At worst, a child is separated needlessly from their family.
Both errors are dreadful, leading to politicised cycles. Obvious problems caused by over- or under-intervention bring a swing in the other direction.
But is there a way of reducing the chances of both errors?
Case officers in the child protection system are presented with a lot of background information about a child’s family and circumstances. Better data analytics can tell the case officer what outcomes have been like for children in similar circumstances. It helps those officers make better decisions with the data that is already in front of them.
Professor Vaithianathan’s team built predictive models, using the agency’s own data. The models would tell child protection officers, for example, what proportion of similar children wound up being removed from their families.
Those officers would still make their own decisions. The data provided by the predictive models would help them to check their own intuitions. Remember that the alternative to using that kind of data is relying more heavily on the assessor’s intuitions about the risks.
The model worked. In a later randomised control trial, it was shown to reduce child hospitalisation by about a third. Child protection officers using the predictive model were less likely to send out case officers for low-risk black families, and more likely to send them out to high-risk white families that had been missed when relying on officer-intuition alone.
It’s an incredibly promising story. A fairly simple predictive modelling tool helped child protection officers to check their own intuitions, reduced over-intervention among black families, and meant that about one third fewer children overall were beaten badly enough to wind up in hospital.
It reduced the risk of both types of errors. Fewer officers intervened in cases where risks were low, and child protection services helped more families where risks really were high. Instead of a pendulum swaying between the two types of error, both types of error became less likely.
But I said that this keynote was downright depressing.
Nothing I’ve said so far is downright depressing.
A great keynote speaker will leave an obvious question dangling. So I took the bait and asked the session’s first question. New Zealand has had world-leading administrative data. And the prior National government’s investment approach was built to facilitate this kind of work.
So why was her team helping Allegheny County, and counties in Colorado, and Los Angeles, and other parts of America to make their systems better? Surely ours could also use that help.
We can all despair at the answer.
You see, Professor Vaithianathan’s team had started their work here. It’s where they started building these predictive models. It had been intended to help families here.
Then the bureaucracy ran a hamfisted attempt to prove up the model. Minister Anne Tolley damned the entire enterprise as experimenting on kids. And it was all shelved.
The change in government brought no appetite for better use of data in improving social services. So Kiwi experts instead help to improve American social services.
It is damning not only of both the prior National and current Labour governments, but also of New Zealand’s centralised approach to everything.
Here, you have to convince every possible veto player before anything can happen.
In the US, you need only to find the one county that wants to try something innovative. Prove it works, and others will pick up the innovation.
One third fewer kids being sent to hospital. Those are the stakes. Will we ever learn?