There is much discussion here regarding quantum efficiency (QE). Keep in mind that figures for sensors are generally _peak_ QE for a given colour filter array element. These can be quite high like 60-70%.
But - this is an 'area under the graph' issue. While it may peak at 60%, it can also fall off quickly and be much less efficient as the wavelength moves away from the peak for say red/green/blue.
From what I can tell from the tacky promo videos, the sensor is very sensitive for each colour over a wide range of wavelengths, probably from ultraviolet right up to 1200nm. That's a lot more photons being measured in any case, but especially at night.
Their use of the word 'broadband' sums it up. It's more sensitive over a much larger range of frequencies.
I also wouldn't be surprised if they are using a colour filter array with not only R/G/B but perhaps R/G/B/none or even R/IR/G/B/none. The no filter bit bringing in the high broadband sensitivity with the other pixels providing colour - don't need nearly as many of those.
Edit - one remarkable thing for me is based on the rough size of the sensor and the depth of field in the videos, this isn't using a lens much more than about f/2.4. You'd think it would be f/1.4 or thereabouts to get way more light but there is far too much DoF for that.
It would be interesting to see how this compares to theoretical limits. At a given brightness and collecting area, you get (with lossless optics) a certain number of photons per pixel per unit time. Unless your sensor does extraordinarily unlikely quantum stuff, at best it counts photons with some noise. The unavoidable limit is "shot noise": the number of photons in a given time is Poisson distributed, giving you noise according to the Poisson distribution.
At nonzero temperature, you have the further problem that your sensor has thermally excited electrons, which aren't necessarily a problem AFAIK. More importantly, the sensor glows. If the sensor registers many of its own emitted photons, you get lots of thermal noise.
Good low noise amplifiers for RF that are well matched to their antennas can avoid amplifying their own thermal emissions. I don't know how well CCDs can do at this.
Given that this is a military device, I'd assume the sensor is chilled.
Hang out with astronomers, they do this stuff all the time: poisson noise, dark current, readout noise, readout time, taking thousands of shots and stacking them according to the previous factors.
Theoretical limits are probably very high. For one, collecting area can be pretty large. Obviously you don't want a square meter lens, but a foot-wide one would probably be acceptable for a specialized camera.
Second, nocturnal animals like howls see very well in the dark. Granted, they don't see colors but they still show that in theory seeing in the night is physically impossible.
PS. regarding the collecting area, something I've been wondering for a bit : isn't the number of photons a lens can capture proportional to the square of the surface and not the surface itself? I know that sounds counter-intuitive but with interference, quantum mechanics and stuff, I checked the math once and I could not see where I was wrong.
I can see in a dark lab with my EMCCD camera. It doesn't look as good as in this video, but they got maybe a 3x improvement compared to what I can do at present, with minimal effort.
It's not enough to do crazy quantum stuff in the camera, the light source would have to be similarly manipulated in a quantum way as well.
My understanding is that the human eye has a surprisingly high quantum efficiency (about 12 to 30 % of photons are detected), and that the reason night looks dark is because there really aren't that many visible spectrum photons around.
My guess is that this camera is physically enormous.
Edit: Apparently, it's really small! I am dumbfounded. Then again, I guess it could have a hundred times the area of a human pupil and still be pretty small.
One would think with all the money the military throws into imaging technology that they would already have this.
For Special Operations use, it'd be nifty to have this technology digitally composited in real-time with MWIR imaging on the same wearable device. Base layer could be image intensification with this tech, then overlay any pixels from the MWIR layer above n temperature, and blend it at ~33% opacity. Enough to give an enemy a nice warm glow while still being able to see the expression on their face. Could even have specially made flashbangs that transmit an expected detonation timestamp to the goggles so they know to drop frames or otherwise aggressively filter the image.
Add some active hearing protection with sensitivity that far exceeds human hearing (obviously with tons of filtering/processing), and you're talking a soldier with truly superhuman senses.
That's not to mention active acoustic or EM mapping techniques so the user can see through walls. I mean, USSOCOM is already fast-tracking an "Iron Man" suit, so I don't see why they wouldn't want to replicate Batman's vision while they're at it.
> For Special Operations use, it'd be nifty to have this technology digitally composited in real-time with MWIR imaging on the same wearable device. Base layer could be image intensification with this tech, then overlay any pixels from the MWIR layer above n temperature, and blend it at ~33% opacity. Enough to give an enemy a nice warm glow while still being able to see the expression on their face
See the two images of an SUV (the top one has caption "Color Low Light Night Vision Midnight Image Fused with thermal infrared RED ALERT FLIR Image") on the product page [1].
> For Special Operations use, it'd be nifty to have this technology digitally composited in real-time with MWIR imaging on the same wearable device. Base layer could be image intensification with this tech, then overlay any pixels from the MWIR layer above n temperature, and blend it at ~33% opacity. Enough to give an enemy a nice warm glow while still being able to see the expression on their face.
I believe this is what CAT did with their S60 phone camera - fusing together the image of the thermal sensor and the camera to have a high-resolution thermal image even without an expensive-as-hell sensor.
Can someone wake me up in the future? When we have digital eyes, and we can walk around at night as if it were day except the stars would be glittering. Sometimes, I'm so sad to know I'll not live to know these things and I'm incredibly envious of future generations.
They list a lot of potentially useful applications on the product's own web site. I wonder how long it will take for this sort of technology to be commercially viable for things like night vision driving aids. High-end executive cars have started to include night vision cameras now, but they're typically monochrome, small-screen affairs. I would think that projecting an image of this sort of clarity onto some sort of large windscreen HUD would be a huge benefit to road safety at night. Of course, if actually useful self-driving cars have taken over long before it's cost-effective to include a camera like this in regular vehicles, it's less interesting from that particular point of view.
That really is incredible. I wonder how they keep the noise level down and if the imaging hardware has to be chilled and if so how far down. Pity there is no image of the camera (and its support system), I'm really curious how large the whole package is. It could be anything from hand-held to 'umbilical to a truck' sized.
Watch when the camera tilts upwards and you see all the stars.
They say it's hybrid IR-visible. I wonder if the trick is to use IR as the luma and then chroma-subsample by having giant pixels to catch lots of photons.
This is an amazing device. I've taken night photos that look like frames of this movie on my digital camera, but they require a 60-second exposure and a tripod, and they're -- still frames.
My brother in law experimented with this camera a few years back on family portraits. The camera picks up a lot of "dark" details. Skin displays pale and veins are very defined. My nieces called it "the vampire camera".
I've asked my dermatologist about using wide spectrum cameras to better diagnose conditions. She said its being discussed, researched.
TMI: I have an autoimmune disease affecting my skin that my care givers have attempted to track with photos over the years. It's not very effective. But with just human eye and just the right lighting, its much easier to see what's what. Sun light vs florescent, angle vs straight on.
There are far infrared cameras that capture thermal radiation from 9-15 NM band. They nicely allow to see in complete darkness. They do not use CCD but rather microbalometers.
But they are expensive. 640x480 can cost over 10000 USD and cameras with smaller resolution like those used in high-end cars still cost over thousand USD.
FLIR recently started selling their low-resolution uncooled sensors in small quantities:
* 80x60 resolution for $200
* 160x120 resolution for $240 (doesn't seem to be in stock on Digikey though).
So maybe Peltier on the sensor, heat sink attached to body, body hermetically sealed. Sensors probably tested for best noise quality (probably a really low yield on that).
Shutter time variation or mechanical iris closing depending on the amount of incident light to change the exposure?
Otherwise with any direct source of light in an image it would immediately overexpose (in some low light cameras that could even damage the sensor). It's super impressive.
Why can we see terrain much better than with a naked eye, but stars definitely worse? There isn't even a hint of Mikly Way, which should be easily visible with naked eye in a desert with zero light pollution.
Or this was shot during full moon, carefully avoiding it getting in sight? Then it is not much better than (fully adapted) naked eye.
[+] [-] qume|9 years ago|reply
But - this is an 'area under the graph' issue. While it may peak at 60%, it can also fall off quickly and be much less efficient as the wavelength moves away from the peak for say red/green/blue.
From what I can tell from the tacky promo videos, the sensor is very sensitive for each colour over a wide range of wavelengths, probably from ultraviolet right up to 1200nm. That's a lot more photons being measured in any case, but especially at night.
Their use of the word 'broadband' sums it up. It's more sensitive over a much larger range of frequencies.
I also wouldn't be surprised if they are using a colour filter array with not only R/G/B but perhaps R/G/B/none or even R/IR/G/B/none. The no filter bit bringing in the high broadband sensitivity with the other pixels providing colour - don't need nearly as many of those.
Edit - one remarkable thing for me is based on the rough size of the sensor and the depth of field in the videos, this isn't using a lens much more than about f/2.4. You'd think it would be f/1.4 or thereabouts to get way more light but there is far too much DoF for that.
[+] [-] amluto|9 years ago|reply
At nonzero temperature, you have the further problem that your sensor has thermally excited electrons, which aren't necessarily a problem AFAIK. More importantly, the sensor glows. If the sensor registers many of its own emitted photons, you get lots of thermal noise.
Good low noise amplifiers for RF that are well matched to their antennas can avoid amplifying their own thermal emissions. I don't know how well CCDs can do at this.
Given that this is a military device, I'd assume the sensor is chilled.
[+] [-] greglindahl|9 years ago|reply
[+] [-] grondilu|9 years ago|reply
Second, nocturnal animals like howls see very well in the dark. Granted, they don't see colors but they still show that in theory seeing in the night is physically impossible.
PS. regarding the collecting area, something I've been wondering for a bit : isn't the number of photons a lens can capture proportional to the square of the surface and not the surface itself? I know that sounds counter-intuitive but with interference, quantum mechanics and stuff, I checked the math once and I could not see where I was wrong.
[+] [-] frozenport|9 years ago|reply
[+] [-] mschuster91|9 years ago|reply
There's a promo video of the manufacturer. At 0:16 you can see something looking like cooling fins on the side. https://youtu.be/c_0s06ORTkY?t=16s
[+] [-] doubleunplussed|9 years ago|reply
My understanding is that the human eye has a surprisingly high quantum efficiency (about 12 to 30 % of photons are detected), and that the reason night looks dark is because there really aren't that many visible spectrum photons around.
My guess is that this camera is physically enormous.
Edit: Apparently, it's really small! I am dumbfounded. Then again, I guess it could have a hundred times the area of a human pupil and still be pretty small.
[+] [-] joshvm|9 years ago|reply
https://www.youtube.com/watch?time_continue=328&v=c_0s06ORTk...
Anecdotal evidence on the internet suggests it's around £6k, but that seems far too low.
[+] [-] edgartaor|9 years ago|reply
[+] [-] rl3|9 years ago|reply
For Special Operations use, it'd be nifty to have this technology digitally composited in real-time with MWIR imaging on the same wearable device. Base layer could be image intensification with this tech, then overlay any pixels from the MWIR layer above n temperature, and blend it at ~33% opacity. Enough to give an enemy a nice warm glow while still being able to see the expression on their face. Could even have specially made flashbangs that transmit an expected detonation timestamp to the goggles so they know to drop frames or otherwise aggressively filter the image.
Add some active hearing protection with sensitivity that far exceeds human hearing (obviously with tons of filtering/processing), and you're talking a soldier with truly superhuman senses.
That's not to mention active acoustic or EM mapping techniques so the user can see through walls. I mean, USSOCOM is already fast-tracking an "Iron Man" suit, so I don't see why they wouldn't want to replicate Batman's vision while they're at it.
[+] [-] jamessb|9 years ago|reply
See the two images of an SUV (the top one has caption "Color Low Light Night Vision Midnight Image Fused with thermal infrared RED ALERT FLIR Image") on the product page [1].
[1]: https://www.x20.org/color-night-vision/
[+] [-] mschuster91|9 years ago|reply
I believe this is what CAT did with their S60 phone camera - fusing together the image of the thermal sensor and the camera to have a high-resolution thermal image even without an expensive-as-hell sensor.
[+] [-] kobeya|9 years ago|reply
They do... the OP points to a product sold to military.
The US government develops its military technology buy giving companies like this R&D grants and then buying their product.
[+] [-] telesilla|9 years ago|reply
[+] [-] avdempsey|9 years ago|reply
[+] [-] babyrainbow|9 years ago|reply
All things considered, you might not need to...
[+] [-] khazhou|9 years ago|reply
You're already in the future, man!
[+] [-] akurilin|9 years ago|reply
[+] [-] Silhouette|9 years ago|reply
[+] [-] colordrops|9 years ago|reply
1. It would be nice to see a split screen against a normal view of the scene as it would be seen by the typical naked eye.
2. Our light pollution must SUCK for nocturnal animals that see well at night.
[+] [-] jacquesm|9 years ago|reply
Watch when the camera tilts upwards and you see all the stars.
[+] [-] anamexis|9 years ago|reply
[+] [-] 19eightyfour|9 years ago|reply
If they can increase the dynamic range to bring detail to the highlights it is basically perfect.
I've never seen a valley look like that with a blue sky above with stars in it. Truly incredible.
The 5M ISO rating is pretty funny. 1/40 f1.2 ISO 5M.
[+] [-] Animats|9 years ago|reply
[+] [-] return0|9 years ago|reply
[+] [-] cameldrv|9 years ago|reply
[+] [-] floatboth|9 years ago|reply
Holy shit, my Canon 600D is pretty bad at 2500, goes to crap at 3200, and 6400 is an absolute noise mess…
[+] [-] Nrsolis|9 years ago|reply
http://www.kenrockwell.com/sony/a7s-ii.htm
EDIT: fixed the actual ISO.
[+] [-] dreamcompiler|9 years ago|reply
[+] [-] caublestone|9 years ago|reply
[+] [-] specialist|9 years ago|reply
TMI: I have an autoimmune disease affecting my skin that my care givers have attempted to track with photos over the years. It's not very effective. But with just human eye and just the right lighting, its much easier to see what's what. Sun light vs florescent, angle vs straight on.
[+] [-] sydd|9 years ago|reply
[+] [-] _0w8t|9 years ago|reply
But they are expensive. 640x480 can cost over 10000 USD and cameras with smaller resolution like those used in high-end cars still cost over thousand USD.
[+] [-] fest|9 years ago|reply
I've used the first one- resolution is quite low, but a few years ago you couldn't even buy such sensors as a hobbyist (example image: http://faili.wot.lv/tmp/IMG_20160320_163431.jpg)
These are the same sensors which are used in FLIR smartphone-attached thermal cameras.
[+] [-] teh_klev|9 years ago|reply
https://www.x20.org/color-night-vision/
[+] [-] batbomb|9 years ago|reply
[+] [-] drenvuk|9 years ago|reply
[+] [-] jacquesm|9 years ago|reply
Otherwise with any direct source of light in an image it would immediately overexpose (in some low light cameras that could even damage the sensor). It's super impressive.
[+] [-] peteretep|9 years ago|reply
[+] [-] anovikov|9 years ago|reply
Or this was shot during full moon, carefully avoiding it getting in sight? Then it is not much better than (fully adapted) naked eye.