How I failed to replicate an $86M project in 1 line of code

[new299]

The post starts with a false premise that:

"[a medium post has been] doing the rounds the last few days, describing how an automated license plate recognition (ALPR) system being developed for the Australian Victoria Police could just use the open-source ALPR system OpenALPR instead"

The original post was pretty much a tech demo, and noted a bunch of issues. I think the point was that an ALPR project shouldn't really cost $86M.

ALPR is a well studied problem, there are a number of commercial solutions. I've no doubt there are even a number of solutions already deployed in Australia. Such systems are routinely used, for example, to monitor the speed of traffic (see Traffic master).

And as the original post showed, there are even open source solutions to the problem that work pretty well.

So yea... $86M seems like a lot when there are probably off the shelf systems you can buy at a much lower cost...

[jacalata]

The Victorian police system was first running in cars on public roads in 2012. No doubt there are plenty of solutions available today that have been developed since then, but they probably weren't considered at the time. That $86m has covered development of the software and hardware and at least six years of running costs for the program. So yea, I'm sure $86m sounds like a lot when you compare it to some random piece of software that does a small portion of the project.

[itsdrewmiller]

That medium post is entitled "How I replicated an $86 million project in 57 lines of code". While it does contain some caveats, it's not like some copy editor misrepresented the authors intent when coming up with the headline.

[vosper]

In think the author of the original article really shot himself in the for by going with such a clickbaity title. I went into the post with low expectations, but I thought it was actually not bad, and as you say the point really was that the Australian government seems to be paying a lot of money for this system.

[perpetualcrayon]

What if they created the 57 line project, and spent $86 million to improve OpenALPR?

[jacalata]

Then it would be useless to the people who spent the money, because they wouldn't have the cars and cameras and ongoing support and research that they wanted to spend that money on.

[naikrovek]

The problem isn't ALPR, the problem is the imagery. You must calibrate your cameras and take a few steps to get quality images, or it won't matter how what software you use.

[zhanwei]

I agree with this article more. Yes, $86M is sure expensive with lots of efficiencies. But the first one is oversimplifying the technical aspect, which is missing the point on the sources of inefficiencies. The project can't be using openALPR at its current level and it is hopeless to use it as a starting point to get decent accuracy.

[alex_g]

The first article points out the inefficiency and does a good job of replicating some basic functionality. It's basically saying "with this starting pointing, the performance is not as good, but we could probably make it a lot better without spending $86 mil"

IMO, this one is saying "I'm gonna push the starting point even further back and complain about someone making a suggestion about inefficiencies."

But that's the internet I guess.

[sverige]

A little digging seems to indicate that the author makes a living as an engineer whose livelihood relies on customers paying for solutions provided by the company he works for, so naturally it is offensive that someone would criticize inflated costs associated with a third-party, closed source solution to a problem.

[kerkeslager]

Two problems:

1. You've shown the success of OpenALPR isn't perfect, but I'm sure the closed source alternative isn't perfect either. It might be better, but is it $86 million better?

2. If we spent $86 million on developing OpenALPR, it would be an incredible product that everyone could use, without paying so much to the overhead of an executive structure.

[jacalata]

To even begin to answer your question, you'd need to know what the project included besides the software that OpenALPR could theoretically replace. Like how much hardware has it bought? How much of that money went to paying police officers for time spent developing requirements, testing the prototypes, and learning the eventual system? How much of it went to the research needed to develop this system some five years before openALPR was available? How much went to paying the software developers a normal first world wage and benefits for five+ years?

[naikrovek]

YouTube-encoded video is going to give relatively awful results, no matter what vision software you put it through.

Good cameras, lenses, and camera placement will make or break a project like this.

[andrewchambers]

I'm pretty sure you could make it realtime with more than 10x improvement:

Scan a thinner section of the video because the sky/ground is useless, and process every 10th frame.

[Izmaki]

This requires a vehicle to be right in front of the (police) car. Not very practical as most of the license plates you get to scan will be in other locations of the video - you know, since the car in front of you may be there the next 10 minutes whereas cars drive past you and are parked to the side all the time.

[taitems]

Original author of the original article this person is responding to. Saw this response earlier today which made me a bit sad, but mostly excited that people are having a crack at it themselves.

It's currently Father's Day here in Australia so I don't have a chance to respond right now. On the road all day.

I'll have to do a follow up next weekend, I'm absolutely flat chat both in and out of work at the moment.

[coldtea]

Whether the actual project is worth exactly $1 or $86 million is not the issue. Not even whether a $1/67 lines of code can replicate a $86 million project.

The real substance -- and the first posts hints at it -- is that government (and private sector) projects are more often than not many times more expensive than they should be.

Sometimes because they are overengineered (in 2005 it would be 200 Java programmers building a EJB/SOAP/XML monster that requires 10 workstations to run), other times just because they can (charge more). The same way health related projects and vendors can charge a ton for trivial stuff (even plain plastic syringes).

In many countries, it's also because the bigger the project, the more greedy intermediaries can get a bite.

[jonathankoren]

I don't understand the hate that's being directed at this article. This author isn't the person basically said, "LOL. I'll string together some python in an afternoon and do it." This author tried to replicate the "How I replicated an $86 million project in 57 lines of code" article, and unsurprisingly came to the conclusion, "No, he didn't."

[naikrovek]

Well, YouTube-encoded video is a severe mistake and it was his assumption that this would be representative of video out of a commercial camera.

It is not.

The camera is also not calibrated. Text recognition does not work well on video whose distortion characteristics are not known.

I am not at all surprised that this is his result.

This morning I calibrated my dashcam and used OpenALPR to replicate this, and I got FAR better results than this blogger.

At the end of a 30-minute section of heavy traffic, I had many hundreds of license plates with 95% confidence or better. I do not have a way to verify the plates automatically, so I don't know how many of those are actually correct.

I filmed at 2560x1440 at 30fps.

[gcb0]

can't tell if incompetence or malicious.

everyone knows dashcam, even on 1080p, simple can't be used fir license plates! humans can't read most of them. specially after youtube compression! this is such a known fact that the more serious youtube reviews show a warning about all that before dashcam reviews.

[ww520]

Often time the devil is in the detail.

[knguyen0105]

OpenALPR has a premium service (http://www.openalpr.com/cloud-stream.html). Has anyone tried it?

[klondike_]

I think this is a good case for why all government software should be open source.

How many police departments around the world are spending absurd amounts of money for this technology? It would make much more sense both cost and productivity wise to have one big project that anyone can improve on instead of multiple inferior projects.

Not to mention that all of this is paid for with taxpayer money, so it is reasonable for those same taxpayers to have access to the code.

[lifeisstillgood]

Please join my campaign at http://www.oss4gov.org/manifesto - this is exactly why we want all government software to be open source

(Shameless plug)

[Namidairo]

I wonder how much better it would work if either article specified the Australian training data that is included, instead I believe it is defaulting to the US set.

As for speed, there appears to be an OpenCL option that should be investigated.