Barcode Detection API

[bob1029]

This stuff has been a nightmare for us for a long time. We have to scan pdf417 barcodes from physical identification as part of our offering.

Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens.

Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.

[disillusioned]

We ended up going with the purpose-built barcode sleds from Infinite Peripherals. The Linea Pro, I think. Extra battery, onboard illumination/laser guide, _really_ fast barcode engine in any orientation, and a few pretty simple SDK options. We ended up using "our custom webkit browser will fire a custom JS event on scan", which worked perfectly for our purposes.

Everything else is measurably slower: waiting for focus in a dim room, low framerate, weird other issues.

[jaxn]

We also scan pdf417 from ID cards. In our case, it was much easier to just use hardware 2d scanners. We are a b2b SaaS and they are scanning IDs all day, so a hardware solution makes sense.

The weird thing about pdf417 is that there is a lot of helpful info our there, up until a point. It's like everyone is collaborative and asking/answering questions up until the point they really figure it out, but those last few steps never much make it into open code.

A couple times a year I go searching to see if anyone has implemented solid pdf417 in WASM yet. It seems like a natural fit to me, and I am sure people have done it, but I have yet to see anything publicly available.

[unqueued]

I found the most reliable barcode scanner to be an old Symbol usb scanner.

I also found that I could scan barcodes off of a screen with a laser scanner if I put a translucent white plastic bag over the screen. Maybe someone with more knowledge about optics could give some insight into why that trick works. My understanding is that laser scanners should only be able to scan by reflection, and shouldn't work off of a screen at all.

[danachow]

Purpose built barcode scanners typically don’t fumble with autofocus have faster 0-latency autoexposure (adjusted during frame readout) and even the good 2D CMOS sensor ones will start decoding linear or stacked codes such as PDF417 while the frame is being transferred. That can’t be beaten for latency.

[fennecfoxy]

Idk but whatever library the NHS website is using to scan the QR codes on LFTs is flawless, as soon as the camera view opens it immediately scans, you'd miss it if you blinked.

[gruez]

>Secure context: This feature is available only in secure contexts (HTTPS), in some or all supporting browsers.

What's the point of this? I understand why you might want this for APIs that expose private user information, but this just performs some computation. You could do the same thing with a js library or webassembly. Is this just to punish http sites?

[imajoredinecon]

There might be more to the privacy characteristics of an API like this than meets the eye. For instance, if a user grants camera permissions scoped to http://insecure.website in order to scan a QR code and get it detected, it might be surprising to the user that on-path attackers would be able to see the user’s camera snaps. Making the API work in a degraded mode in insecure contexts (e.g. by disabling particularly privacy-sensitive data sources) might be a plausible alternative, but at this point it’s arguably cleaner and less risky to just make the interface HTTPS-only in order to avoid making developers and users reason through these nuances.

[dblohm7]

Basically, yes. Browser vendors have had an agreement for some time now to only expose new APIs in secure contexts in order to encourage HTTPS adoption.

[eternityforest]

I'm not a fan of the whole secure context separation thing. A lot of cool P2P stuff is impossible because a service worker can't access an HTTP server, which would be very useful for web sites to talk to IoT devices.

If there was something sensitive in front of my camera, I wouldn't give any random site permission anyway.

File system access should definitely be secure only, the rest might not need it.

Of course if Mozilla actually did FlyWeb we could almost totally deprecate HTTP except for some public legacy sites.

[chiph]

pdf417 barcodes are commonly used on US driver's licenses, and contain private information like the person's address, height, eye color, full name, etc. For the API to do its work in detecting a valid barcode, it needs to be able to read the entire barcode to compute a checksum/checkdigit. Which means it has to read the data.

[djbusby]

There is a full copy of the zxing library in JS. It's been working great for my purposes - scanning QR and C128 Barcodes.

E: https://github.com/zxing-js/library

[crossroadsguy]

ZXing is/was solid. I used it for Android, then moved to (suggested) Google’s ml-kit offering around a year ago.

ZXing is pretty much abandoned now. It was anyway “kind of” from Google.

[asyncscrum]

Used this in a b2b saas for a few years. Worked well and helped remove complexity in a previous way we did it that relied on a native app called pic2shop. We lost a bit of functionality specifically a clear target zone, green scanning line and better low light support. It made up for it in ease of support and reduction in perceived jank.

[bob1029]

This is what we use as a fallback if the user doesnt have a handheld 2D scanner. Very fast & accurate in most of my testing.

[jacob019]

I recently made a webapp for scanning barcodes using the phone camera for warehouse use. I used QuaggaJS. I disabled the barcode locator because it was too heavy, instead I just draw a box on the screen that the barcode is to be positioned in. The barcode detection and decoding happens entirely in the browser and it works very well.

[egfx]

I also built a web app for barcode scanning. For testing quaggaJS is fine but it’s highly inconsistent and requires an extreme degree of customization to get adequate results. I recommend scandit. It’s a good production ready library with a very nice web sdk.

[miohtama]

As the page does not discuss the motivation behind the API, does anyone know why this is ”native” API and not just a generic image processing script implemented with JavaScript / WebGL / etc.?

[michaelt]

> does anyone know why this is ”native” API and not just a generic image processing script

Consider the following facts:

1. This was designed by Google

2. Google Play Services provides an API for detecting barcodes, faces and text https://developers.google.com/android/reference/com/google/a...

3. Chrome gets a shape detection API for detecting barcodes, faces and text https://web.dev/shape-detection/

Of course, it's a matter of perspective whether you want to see this as lovers of the free web helping webapps to feature parity with native apps; or Google churning out yet another low-effort standard that's easy for them and difficult for all their competitors.

[PaulHoule]

At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software. I can tell iOS users to just "scan the code with the Camera" app but I have to tell Android users to find a QR scanner app on the app store.

My experience with an actual bottom-of-the-line prepaid phone was that the first QR scanner app I downloaded from Google Play worked right the first time but I had to try six different apps before I found one that worked with my Samsung Tablet.

A web-based scanner would be a possible answer, though really Google and the phone makers and the carriers should have prioritized a "just works" solution years ago.

[idealmedtech]

Native code allows for performance gains that a script (even a performant one) may not be able to match. After all, this is fundamentally an image processing problem, one of the more computationally expensive things a browser has to do.

[agucova]

It's a heavily standardized, very recurrent problem that can take a lot of advantage of native code and platform-specific optimization, so it's really an ideal target for a Web API.

[rahimnathwani]

To allow the implementation to take advantage of libraries provided by the operating system[0] (where available), without the web site developer needing to even know that it's happening.

[0] or things like Google Play Services.

[afavour]

Barcode detectors can be pretty big dependencies. Given that this will often just be forwarding directly so some kind of OS capability it makes sense to ship in browser.

[bena]

Because while barcode scanning is a subset of image processing, it's a use case that's pretty common.

[timando]

I wanted this to be a barcode scanning API where a web app can use the camera to scan a barcode without being able to see anything else in the image.

[duck-valentine]

There's a couple good datasets with trained models on Roboflow Universe that would pair nicely for a Computer Vision project/app:

Object Detection for QR Code and Bar-Codes - https://universe.roboflow.com/new-workspace-mrc2b/optiscan-l...

another one for just QR Codes - https://universe.roboflow.com/lihang-xu/qr-code-oerhe

[cyode]

Bit off-topic: can anyone recommend resources (videos/papers/tutorials) that could help me learn how QR barcode detection and scanning itself is implemented?

I've been thinking that building a basic scanner from scratch (i.e. only using vanilla cpp/python/whatever + image processing libraries, but nothing barcode-specific) could teach me a lot.

[lgvld]

I guess it helps to understand how to generate a QR code: https://www.nayuki.io/page/qr-code-generator-library (see also the references at the end)

Good luck. ;-)

[derac]

I forked another person's code and added bounding boxes, if you want to check it out. Seems like it works pretty well, though the bounding boxes are erratic.

https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93...

https://htmlpreview.github.io/?https://gist.githubuserconten...

[paxys]

The dream of the early 2000s is finally being realized.

[numlock86]

Cool. Another feature that Firefox won't support. Same with the WebSerial API and others. I sort of understand their reasoning that they want to make a web browser and not an application platform, but that's where the web is shifting to ... or rather has already for a long time. And people wonder why every new tech/browser is based on Chromium all the time. Well, this is one of the many reasons.

[schwartzworld]

Why would Firefox implement non-standard APIs?

[adamrezich]

all of these small features seem good in aggregate but each one makes it that much harder to ever build a competing browser, especially from scratch.

[schappim]

Here is an example of using the Barcode Detection API w/ WebRTC on Android / Chrome: https://files.littlebird.com.au/barcode20.html

[marcjuul]

Does anyone happen to know where in the Chromium code base the implementation is hiding?

The android version seems to use a library that's part of Google Play Services:

  /**
   * Implementation of mojo BarcodeDetection, using Google Play Services vision package.
   */

and

  // The vision library will be downloaded the first time the API is used
  // on the device; this happens "fast", but it might have not completed,
  // bail in this case. Also, the API was disabled between and v.9.0 and
  // v.9.2, see https://developers.google.com/android/guides/releases.

From: https://github.com/chromium/chromium/blob/c4d3c31083a2e14812...

The desktop version references a third party library called "barhopper" here:

https://github.com/chromium/chromium/blob/e1e495b29e1178a451...

and barhopper seems to come from a non-public Google repo:

    'src/third_party/barhopper': {
      'url': 'https://chrome-internal.googlesource.com/chrome/deps/barhopper.git' + '@' + 'ad3c4382875afdd0340f1549f8b9c93cbbc16e37',
      'condition': 'checkout_src_internal and checkout_chromeos',
  },

from https://source.chromium.org/chromium/chromium/src/+/master:D...

I didn't realize that Chromium had missing APIs compared to Chrome but maybe that's always been the case? This appears not to be a W3C standard:

https://wicg.github.io/shape-detection-api/

I was interested in this because there are no really good open source DataMatrix decoders (to my knowledge) with the one in ZXing being basically unusable (last I checked) and libdtmx being ok but not comparable to commercial offerings.

[sevenf0ur]

Yeah, I haven't been able to find any decent open source DataMatrix libraries as well. If you find one I'd love to know.

This feature uses native libraries and only works on Chrome OS, Android, and MacOS currently according to this page:

https://chromestatus.com/feature/4757990523535360

[veysiertekin]

A different context but related: a website plugin that I have built years ago for barcode scanning machines: JBarcode - https://github.com/veysiertekin/jbarcode

I didn't made any changes since then. But it can be easily ported into a more modern framework (React.js etc)

[Scoundreller]

I’m happy to see things like this. My ISP used to ask for a picture of my docsis modem sticker because people kept making mistakes entering in the MAC address.

Unfortunately, the csr was just keying them in and keyed mine in wrong so I was without internet for a few days.

Something to automatically pick it out would be a lot better.

[sleepy_keita]

I bought my first Android device to try this feature out. It's so much faster than zxing on iOS.

[noodlesUK]

I’d love to see this in safari on iOS as well. The native barcode detection capabilities in iOS are substantially better than libraries like zximg, and I like barcodes as an interface between the real world and tech. I also wish text recognition were in safari.

[pabs3]

Anyone know what the backend for this is? Is it zxing-cpp or similar?

Edit: guess it depends on the browser and OS.

[matthewmacleod]

Huh, TIL. Recognition performance is pretty modest (on Chrome/Mac at least) but it's fast and probably more than adequate for most close-up mobile device scans. It basically just wraps MacOS's Vision framework on the Mac, and ML Kit on Android.

[xg15]

It's a nice feature, but, honest question, why is this a web API?

I don't see anything this has to do with browsers. This could be implemented perfectly well in a library.

[amelius]

I'm still looking for an open source python library that can decode Data-Matrix barcodes as seen on Digikey bags. Should I give up and switch to JavaScript+browser?

[schappim]

For Digikey bags (we make eletronics) we use SwipeTrack app + JS.

[pabs3]

zxing-cpp has a Python wrapper that should work for that.

[michaelt]

Is pylibdmtx not up to it?

[Eyght]

I would love to use this for pairing a product with the correct image by decoding barcodes in the image.

[bricss]

In the latest Chrome: > Uncaught ReferenceError: BarcodeDetector is not defined

[PaulHoule]

... it decodes the barcode too?

[happytoexplain]

According to this documentation, yes, it returns the raw value encoded in the barcode.

What it does not appear to do is further decode the raw value into logical data. E.g. data matrices commonly contain GS1 element strings that are not trivial to decode. There are libraries for that, though.

[soylentgraham]

It does

[ape4]

Even thought its on mozilla.org is not supported by Firefox according to the matrix at the bottom.

[happytoexplain]

These docs are a general web developer resource that Mozilla maintains - they are not specific to Mozilla products.

[agucova]

MDN is meant to be a somewhat universal resource for web development. I've even seen official Google web dev resources pointing to MDN.

[dblohm7]

And if you look at the badges in the matrix, the API is not a standard yet, either.

[hunterb123]

You'll find many features in MDN docs that applies to.

That's mainly because of the expansiveness of MDN docs, but somewhat due to the funding distribution of Mozilla going to officers, not Firefox devs.