The New Zealand Herald this week revealed a government-commissioned research report – just as the opposition called for fewer external consultants.  

The University of Auckland report for the Ministry of Transport seeks to identify public support for new methods of financing transport infrastructure. Fuel excise and road user charges will apparently no longer be viable ways to fund transport. The reason is that electric cars will replace petrol and diesel.  

But while the Herald reported on the work as though it were an opinion poll, it really was something different.  

The researchers used Twitter, LinkedIn, and transport blogs to invite “transport stakeholders” to take part in a novel online conversation and consensus building tool.  

It was not a poll of any representative sample of Kiwis. It was a new way of running online focus groups. Participants could put their own views into the mix while approving or disapproving of other participants’ statements.   

The conversation included only 436 respondents, which is a small sample. And it was certainly not representative. The researchers admitted it was biased towards cycling advocates. 

If you ask cycling advocates about their preferred transport policies, they are hardly likely to favour motorways. 

And where advocates write their own statements, don’t expect unbiased wording.   

Consider a statement that 88% of participants agreed with: “Use innovative mechanisms such as congestion charges and pollution pricing to encourage people out of cars and into other forms of environmentally sustainable transport.” 

Hardly anyone would prefer using outdated mechanisms over “innovative” ones. Similarly, would anyone favour environmentally unsustainable transport? There were many such examples – and who would expect otherwise, given the method? 

In fairness, the researchers clarified that this was a pilot study to test their fancy new conversation tool. It allowed them to find ‘clusters’ of beliefs among participants. But those clusters might simply be the different groups of activists drawn to the survey at different times.  

One lesson might be that these kinds of focus groups need more structure. Activists proposed and supported alternative transport funding mechanisms that make little sense. It is hard to see the point in finding out that a high percentage of self-selected activist participants support odd policies.  

The cost of the report is unknown, but whatever was spent, was wasted – at least as far as transport research is concerned. It yielded no results that would have been of value to the Ministry of Transport. 

It is non-reports like this that give external consulting a bad reputation.