The audit office says evidence about the effectiveness of the cashless welfare card is unreliable.

The Australian National Audit Office (ANAO) has identified major flaws in the data collection used to evaluate the Healthy Welfare Card scheme.

Welfare recipients in several communities has been put on a trial of the card system, which puts 80 per cent of their Centrelink allowance on the card and only allows it to be used for essential items.

But auditors say it is “difficult to conclude” whether social harm (like alcoholism and violence) has been reduced due to a “lack of robustness in data collection”.

They found much data missing, including hospital admission figures for several towns.

When the audit office conducted some of its own analysis, it came up with very different figures to the scheme’s official review.

In one case, Social Services Minister Dan Tehan reported there were fewer ambulance call-outs in September 2016 compared to the previous year, but the ANAO’s analysis found there was actually a a 17 per cent increase in call-outs in that period.

Similarly with school attendance figures, Mr Tehan said there was an increase, but the ANAO found it had dropped.

Still, the Social Services Minister stands by the scheme.

“The cashless debit card is making a real difference in the communities where it operates,” he said in a statement.

“[It] is an important element of the Government's work to reduce welfare-funded social harm, and to help Australians escape welfare dependency.

“Independent evaluation by Orima Research found gambling, alcohol and drug consumption were reduced in Ceduna and East Kimberley.”

Taxpayers paid Orima $1.6 million for the evaluation, more than double the expected cost.

The company said costs rose due to the complexity of collecting data.

ANAO made a number of recommendations, including a new data monitoring strategy, risk management and procurement practices.

A second independent evaluation is underway, and Mr Tehan says he is confident it will “satisfy the ANAO recommendations”.