Keith Miller got two parking fines from a camera in Miramar, and the only way out was to prove his own innocence.
Miller, a former Wētā VFX supervisor, shopped at the New World in Miramar in December and went back in January. Smart Compliance Management runs the number plate cameras in that car park. They sent him a breach notice showing his car arriving and his car leaving, and billed him for one long overstay.
Two separate trips. The camera had missed one of the departures, so the system took the entry from one visit, the exit from another, and charged him for the hours in between. He told Stuff the system has "critical flaws that means they are catching too many innocent people". He won his appeal by proving his car had been parked at his own house.
Look at who did the work there. A camera made an accusation. Miller spent his evenings disproving it.
I spent six years on billing platforms at Vend, and I know what happens to a number once a system emits it. Nobody downstream treats it as an estimate. Support treats it as a fact, the customer gets a letter, and the burden of showing it's wrong lands on the one person in the chain with no access to the logs.
The seam
ANPR camera on a parking services truck. Photo by Mbrickn, CC BY 4.0, via Wikimedia Commons.
The Smart Compliance camera runs two kinds of computing, and Miller got caught in the join.
Reading a plate off a moving car is a machine learning problem. The model is right most of the time, wrong some of the time, and blind when another car leaving alongside blocks its view of the plate. Drivers in Belmont, Durham hit the identical failure back in 2022, at a car park run by the same company, where the BBC reported hundreds of £60 charges because the camera "thinks the car was there and it definitely wasn't". You cannot patch that away. It's the error rate of a perception system, and it never reaches zero.
The dwell-time calculation is not machine learning at all. It's arithmetic. Last exit minus first entry, a hard rule doing exactly what it was told.
That rule assumes the camera saw everything. It didn't.
graph TD
A[Camera reads plate<br/>ML model, error rate above zero] --> B[Event log<br/>one departure missing]
B --> C[Dwell rule<br/>last exit minus first entry]
C --> D[Breach notice<br/>$85]
D --> E[You prove where your car slept]
A hard rule trusting a soft signal, with nobody accountable for the join. In Belmont, older drivers paid up because they "don't want the hassle of arguing with the parking company". The appeal process works, technically. It just costs more than the fine.
The law already agrees with the machine
Section 137 of our Evidence Act 2006 says that if a party shows a machine ordinarily does what they claim, the court presumes it did so on the occasion in question, "in the absence of evidence to the contrary".
Read that again with Miller in mind. The camera is presumed right. He supplies the evidence to the contrary.
That presumption was written for instruments. A till, a speedometer, a breathalyser: devices that measure a thing that exists, where "did this machine work properly?" is a question with a findable answer. For an instrument the presumption is reasonable, and mostly harmless.
England had the same rule, uncodified, and it put nearly a thousand sub-postmasters through the courts. The Horizon accounting system generated shortfalls that were never there, and the people it accused could not audit it, so they pleaded, paid, and went to prison. Sir Wyn Williams found that the Post Office knew Horizon was capable of error and "maintained the fiction" it was accurate, and linked at least 13 suicides to the scandal.
Horizon was deterministic software with bugs in it. Provably, findably wrong. And it still took sixteen years and a television drama, because the presumption ran the other way and the sub-postmasters had nothing to push against.
Australia's Robodebt was arithmetic too, dividing an annual income figure by 26 fortnights and calling the result a debt. The Royal Commission called it "a crude and cruel mechanism, neither fair nor legal" after it chased hundreds of thousands of people.
Both of those systems could be proven wrong by anyone with the paperwork and the stamina. Hold that thought.
Now the machine can't be proven wrong
Detroit police ran grainy security footage through facial recognition, got Robert Williams, and arrested him outside his house in front of his daughters. The city settled for $300,000 and accepted a ban on arresting anyone on a face match alone.
Porcha Woodruff was eight months pregnant when Detroit police arrested her on the same kind of match. She had contractions in custody. Charges dropped. In August 2025 a federal judge dismissed her lawsuit, finding her lawyer hadn't shown the officer lacked probable cause. The judge said the arrest was "troubling for many reasons". Woodruff still lost.
Ask what "the machine malfunctioned" would even mean in her case. It didn't. A face recognition model returns a similarity score, and a threshold turns that score into an accusation. When it matches the wrong woman, the system is working to spec. There's no bug to find, no log to audit, no Horizon defect to hand a lawyer. The machine did what it ordinarily does, and that is precisely how it produced her.
That's the distinction nobody has drawn in law yet. Instruments measure. Estimators guess. We built a presumption of correctness for the first kind, and we have quietly pointed it at the second.
Someone chose 92.5%
Camera and live screen at a supermarket self-checkout. Photo by Folkestonesurvey, CC BY-SA 4.0, via Wikimedia Commons. Cropped.
Foodstuffs ran facial recognition across 25 North Island supermarkets and scanned 225,972,004 faces in six months. It threw 1,742 alerts, of which 1,208 were confirmed matches. Nine people were misidentified. Two got asked to leave.
One of them was Te Ani Solomon, a Māori woman, wrongly flagged as a trespassed offender at New World Westend in Rotorua on her 47th birthday, in front of her teenage son. She offered three forms of ID. Foodstuffs called it human error.
Buried in the Privacy Commissioner's report is the detail I keep coming back to: during the trial, Foodstuffs raised the alert threshold from 90% to 92.5% likelihood of a match.
Someone picked that number. Sitting at 90% catches more shoplifters and generates more Te Ani Solomons. Sitting at 95% does the reverse. No setting gives you zero of either, so that dial decides how much harm gets handed to innocent people in exchange for how much protection for staff. It's a policy question about who absorbs the cost, and a supermarket chain answered it.
To be fair to them, they answered it carefully, with real safeguards, and the Commissioner found the trial complied with the Privacy Act. Serious harm dropped by an estimated 16% during the trial. Retail workers get assaulted, and this technology helps. Detroit's post-settlement rule, where a face match is a lead you then have to corroborate with actual police work, is a decent piece of engineering design.
None of that is the problem. The problem starts when the output leaves the model, a probability hardens into a fact about a person, and that person has to argue their way back out.
The review that matters
Our government's AI Strategy went light-touch by design: no AI-specific law, lean on what we've got. Privacy Commissioner Michael Webster asked for his concerns to be minuted in the Cabinet paper, citing Robodebt and Horizon by name as cases where "thousands of people were harmed by technology rollouts that did not adequately consider how to manage risks". He was ignored politely.
The UK has worked out what it got wrong. The Ministry of Justice opened a call for evidence on its computer-evidence presumption, proposing to split evidence captured by a device from evidence generated by software. Instruments and estimators. Justice minister Sarah Sackman put it plainly: "A blanket 'no questions asked' acceptance of the accuracy of digital evidence can have a devastating impact on people's lives."
Better news at home: the Law Commission is now reviewing how government uses automated decision-making, with at least 19 pieces of legislation already authorising it. Commission President Dr Mark Hickford said most of the emphasis on digital has landed on efficiency and productivity, "but we also have to think about citizens and rule-of-law values".
Good. Put section 137 in scope. If you want to rely on evidence a model generated, prove it worked, rather than making Keith Miller prove it didn't.
He got his $85 back because he could show where his car slept that night. Most people can't be bothered, and the companies counting on that have done the maths.