For the people kinda worried: this is a highly specialized piece of glass that is extremely complicated to manufacture at present and must, due to the laws of thermodynamics, not be 100% transparent. It's not going to allow surveillance through existing glass installations in any form, just possibly new ones if there's room for the support equipment and through the use of 4-5 digit piles of cash.
Any camera glass like this will have at least a mild tint and will be used in specialty applications. It'll also have pretty horrible SNR, resolution, and low light performance.
Currently the structural component of this tech is mainly used in extremely high end aerospace applications (various heads up display type systems) so it's unlikely you'll ever run across one of these within the next decade.
Nasty remote sensing tech people can be worried about right now: RF surveillance from various combinations of mmWave, wall penetrating radar, and wifi interferometry. Add in the fact that your IPhone has mac randomization but every other device you own including your car's TPMS doesn't. Also Geiger mode lidar is fun, one company I worked for mapped the inside of a random person's house with it as a demo.
>For the people kinda worried: this is a highly specialized piece of glass that is extremely complicated to manufacture at present and must, due to the laws of thermodynamics, not be 100% transparent. It's not going to allow surveillance through existing glass installations in any form
If you're worried that your Airbnb host is going to use it to spy on you (which was mentioned in the article), unless you already scrutinize every nail hole, photo frame, electrical appliance, electrical outlets, smoke detector, etc, this doesn't open up a new vulnerability.
Pinhole cameras with a lens as small as 2mm are already readily available and cheap, no ones going to use an expensive "window camera" to spy on you when they already have so many other options.
Perhaps those that fear government surveillance or other well funded adversaries may have cause for concern, but few of us are in that category.
> due to the laws of thermodynamics, not be 100% transparent.
How useful is this statement though? Regular glass isn't 100% transparent either, even in just the visible spectrum. Shouldn't we be more concerned with the delta in the visible spectrum is if we're concerned about easy identification? (before mentioning that plenty of glass is purposefully tinted and dynamic tinting is an application here) And reasonably, couldn't we, theoretically, pick up a decent signal from simply capturing the reflections around the glass edge? I mean we can now do 3D reconstructions from pointing a camera at a mirrored ball. I'm sure it'd be very noisy, but there is signal. I mean to have the capability of projecting you'd have the ability to do the reverse too given that I doubt the internal structure of the glass would be (that) directionally dependent. Right? I can be missing something, it's been awhile since I've done optics.
Thanks. The original description made this seem like far-future technological magic. A system that can somehow analyze a random pane of glass and derive all the transformations needed to use it as a high-precision waveguide? I actually had a manager ask me to develop such a thing, and I asked him how many dozen optics PhDs I could hire to accomplish this feat.
I agree that this shouldn't be anywhere near the top of people's privacy concern list. A $1 traditional digital camera can already be hidden very easily and this probably costs thousands of dollars at least if you could even get it.
There are some places where a not-completely 90% transparent piece of glass might not raise too much suspicion, but it could be more difficult to retrofit with a more traditional "pinhole" style spycam.
Like the wall of a shower stall, which you would expect to be less than 100% transparent.
I do think this threat model is less likely, however.
What I think is more likely is a Minority Report style system where there are pervasive cameras everywhere, constantly scanning your face, etc.... Maybe not constantly scanning your retinas and identifying you that way, but facial recognition alone would be more than enough. Either way, once you're recognized, you could be easily tracked silently, or you could be called out publicly by a nearby display.
> It's not going to allow surveillance through existing glass installations in any form, just possibly new ones if there's room for the support equipment and through the use of 4-5 digit piles of cash.
The article is poorly written, as it only discusses the camera component. Strangely, they chose stock images of holographic and optical displays, but didn’t mention that even once.
1) ZEISS unveils holographic Smart Glass at CES 2024, both for displays/projection/filtering, but also another component which is a holographic camera
2) The holocam works by utilizing coupling, decoupling, and light guiding elements to redirect incident light to a concealed sensor, eliminating the need for visible cutouts or installation spaces in visible areas.
3) ZEISS doesn’t plan to be manufacturer, so other companies can use the tech
Edit: theres an actually good demo video showing the real state of the tech rather than mockups: https://www.youtube.com/watch?v=NORPeCcIXRQ buried below in the comments so just surfacing higher. everything else is artists lying to you.
---
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
> coupling, decoupling and light guiding elements to divert incident light to a concealed sensor
So, there's a camera in the dash looking up at the windshield and focusing where it expects to see a face, thereby using the windshield as a reflector? And maybe there's some additional etching and deposited films in the windshield to support the angles required?
And perhaps you can put cameras elsewhere, and similarly subtly modify the windshield or other glass to look at other things as well?
>ESA and NASA space missions have carried this trailblazing ZEISS technology on board for many years. It is also well established in the semiconductor and medical technology sectors.
Huh. Seems like it should be fairly easy to find info on then... though some googling around makes me think they might just be referring to their more general diffraction gratings and whatnot.
Second, #3 is always a red flag for me. It is sometimes code for "we can do this in the lab but we have no idea how one would manufacture it." A similar analogy is "we've got this great idea for a program you can license but no one here knows how to actually code it up."
Third, the impact of this going mainstream would be hard to underestimate. All those people working on transparent displays like they do in sci-fi movies? Yup they could do that. A video conference system with solid eye contact (mentioned in a couple of places) sure you could do that too. A mirror that could show you wearing different clothes? Yup I could see how that would be coded.
That #3 though. That is what tempers my enthusiasm. Did I miss any announcement that they had a display at CES? (or was it just an announcement) If the former I would seriously consider flying over to Vegas to check this out.
It's a reverse light guide. We've been beam forming for a long time, it's unsurprising that the reverse is possible (imaging through a light pipe).
The principal issue will be gathering enough energy. A well lit source like a bathroom mirror (mirror behind the light guide) could work pretty well I'd wager. If the light guide is too efficient then it will appear opaque, so there is a trade-off.
I find "turns any window" pretty misleading. Unless I'm missing something this needs very special glass or at the very least a special coating/laminate.
For folks worried about privacy, it will almost always be more convenient and cost-effective to install a tiny spy camera somewhere.
Zeiss isn't going to aspire to sell cheap glass on razor thin margins.
Years ago I was able to visit Zeiss in Oberkochen. They had a fantastic headquarters with a few older lithographs. I think a couple of the instruments were 80 millions dollars or so.
There’s a facility across from the Autobahn that was so sensitive trucks going by would throw off their machines. They had to put padding on the autobahn to prevent it. This was after they put the foundation on some kind of suspension. My coworker said they hire the most PHDs in Europe.
Sounds like LIGO, but they're in the US. They had to put the AC unit for their entire facility on suspension because their instruments were so sensitive. And they ask people to not accelerate or decelerate so quickly when driving around the campus.
> Given the current fear around hidden cameras in Airbnbs, the idea of every single window (or even shower door) in a rental property being able to spy on you is a little disconcerting.
While there are some really interesting potential applications for this tech, it is also more than a little disconcerting.
The ubiquity of camera phones and the emergence of tech like those Meta glasses is already pushing us to disconcerting (albeit interesting and in some cases very useful) places, but some of these cutting edge concepts worry me. WiFi seeing through walls also comes to mind…
I'm surprised this isn't the top comment. I'm all for the benefits of this tech, and hadn't even thought about the airbnb style implication.
People didn't like that Google Glass could always be filming, now we don't even have a physical camera.
Rayban/Meta (I believe) have a sensor to detect that the wearer has not attempted to cover the light which shows that the camera is in use, but how will that work when every piece of glass is a camera.
After some searching I found a patent I think may be related to this https://patents.google.com/patent/WO2020225109A1/en because it uses the phrase "Holocam", is German and was filed by Audi (the press release mentions automotive applications as the primary initial use case). It's a translation from German which makes it a bit tougher to parse than the usual patent.
The total lack of any deeper information beyond the bold yet vague claims in the press release is frustrating. The PR makes it sound like a miraculous breakthrough destined to change everything. The source release on the Zeiss site only adds two bits of info.
> "The transparency of the holographic layer has only a minimal effect on the brilliance of the image reproduction. It is also possible to detect spectral components as additional information to complement the visible image. The resulting data provide insights into environmental contamination such as air pollution and UV exposure."
However, experience shows that in reality bold+vague claims like this inevitably come with significant trade-offs and constraints which limit its applications (little things like cost, power, fidelity, size, speed, etc). This is especially true in early implementations of new tech. That said, it may still be both interesting and useful. Unfortunately, we have no way to even think about how it might be useful because Zeiss marketing has chosen to play 'hide the ball' instead of just releasing a technical explainer outlining relevant trade-offs, limitations, etc.
If I was talking to someone from Zeiss my first questions would be about how much the additional components impact the optical characteristics of the glass, what the resolution of the resulting image data is, how large are the components needed at the edges and how far away can they be from the capture zone? Then, of course, how the output of the resulting imaging system maps into traditional camera/lens metrics like f-stops, aperture, imager size/density, gain, focal length, etc. Zeiss is an optics company after all.
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
Agree. Obviously a few orders of magnitude in cost reduction would be required... but this seem like it could have interesting potential for energy generation in high-rise buildings, which currently have near-zero solar footprint. With that said, the lack of any mention regarding efficiency makes this part of the press release seem like a bit of smoke and, erm, mirrors.
Do a search on "non line of sight imaging" (aka NLOS). Here's just one public paper.[1] If this is what is being published, what do you think the intelligence agencies have?
> This means that everything from the window in your car to the screen on your laptop to the glass on your front door can now possess an invisible image sensor.
Retailers, marketers, and data brokers are salivating.
I can't wait until the windows in our homes plaster ads over everything every time we look outside.
It'll sure be distracting when it's the windshields of our cars, but I do look forward to the legal drama when companies get sued for painting their "holographic" ads on top of the adspace other people already paid to pollute with their own advertisements.
The telescreen received and transmitted simultaneously. Any sound that Winston made, above the level of a very low whisper, would be picked up by it; moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment.
"Webcams that enable you to look anywhere on your screen."
Minor aside, but does anyone actually care about this? Forever ago, I was told to try and look into the camera in order to project eye contact during video calls, but now that just seems like a cultural hangup that arose from people not being familiar with video calling. Now that it is more ubiquitous, I feel like we all have collectively agreed that the eye contact thing is unnecessary?
It's not just cultural. We've evolved to recognize eye-contact. Newborn babies immediately know when you look at them. Eye contact with a dog communicates dominance.
You've just acclimated to losing that signal during video chats. Bring it back, and you'll have a richer experience.
More generally, we have evolved special neural hardware to recognize gaze, with a special case for when gaze is directed at our eyes. Gaze is important because it's a sign of where people are attending. And as Herb Simon said: "in the information age, attention is the scarce resource."
This is beyond even science fiction. I could have never imagined something like this was even possible - I still can’t. Is there a demo of this tech in action?
Yeah it sounds like a great option to place a camera invisibly in the middle top half of your screen so you can actually look a person in the eye when videoconferencing. No more weirdly looking down or to the side for everyone.
I'm sure it can be used for creepiness as well but I see the benefits too.
I've read through all the comments here but I still don't have the slightest idea of how this works.
I see some references to "light guides" and words like "coupling" but I don't know what those mean at all, and all of my googling is not helping to explain how light guides embedded in a thin, mostly transparent layer could possibly be used to project a hi-res holographic 3D light field out of a piece of glass. How big is each guide? What is it exactly -- what is it actually made of? How is it shaped? How are they arranged? How are they illuminated? How do you manufacture something like this?
[+] [-] AnarchismIsCool|2 years ago|reply
Any camera glass like this will have at least a mild tint and will be used in specialty applications. It'll also have pretty horrible SNR, resolution, and low light performance.
Currently the structural component of this tech is mainly used in extremely high end aerospace applications (various heads up display type systems) so it's unlikely you'll ever run across one of these within the next decade.
Nasty remote sensing tech people can be worried about right now: RF surveillance from various combinations of mmWave, wall penetrating radar, and wifi interferometry. Add in the fact that your IPhone has mac randomization but every other device you own including your car's TPMS doesn't. Also Geiger mode lidar is fun, one company I worked for mapped the inside of a random person's house with it as a demo.
[+] [-] Johnny555|2 years ago|reply
If you're worried that your Airbnb host is going to use it to spy on you (which was mentioned in the article), unless you already scrutinize every nail hole, photo frame, electrical appliance, electrical outlets, smoke detector, etc, this doesn't open up a new vulnerability.
Pinhole cameras with a lens as small as 2mm are already readily available and cheap, no ones going to use an expensive "window camera" to spy on you when they already have so many other options.
Perhaps those that fear government surveillance or other well funded adversaries may have cause for concern, but few of us are in that category.
[+] [-] thfuran|2 years ago|reply
It's perhaps worth noting, however, that it wouldn't be unusual for a window to have only ~60% transmittance in visible spectrum.
[+] [-] godelski|2 years ago|reply
How useful is this statement though? Regular glass isn't 100% transparent either, even in just the visible spectrum. Shouldn't we be more concerned with the delta in the visible spectrum is if we're concerned about easy identification? (before mentioning that plenty of glass is purposefully tinted and dynamic tinting is an application here) And reasonably, couldn't we, theoretically, pick up a decent signal from simply capturing the reflections around the glass edge? I mean we can now do 3D reconstructions from pointing a camera at a mirrored ball. I'm sure it'd be very noisy, but there is signal. I mean to have the capability of projecting you'd have the ability to do the reverse too given that I doubt the internal structure of the glass would be (that) directionally dependent. Right? I can be missing something, it's been awhile since I've done optics.
[+] [-] nonrepeating|2 years ago|reply
[+] [-] sroussey|2 years ago|reply
[+] [-] neallindsay|2 years ago|reply
It's still creepy though.
[+] [-] ryukoposting|2 years ago|reply
The truly pathetic thing is that virtually all of these devices could use RPA but don't, because nobody remembered to flip that flag from "n" to "y"
[+] [-] bradknowles|2 years ago|reply
Like the wall of a shower stall, which you would expect to be less than 100% transparent.
I do think this threat model is less likely, however.
What I think is more likely is a Minority Report style system where there are pervasive cameras everywhere, constantly scanning your face, etc.... Maybe not constantly scanning your retinas and identifying you that way, but facial recognition alone would be more than enough. Either way, once you're recognized, you could be easily tracked silently, or you could be called out publicly by a nearby display.
[+] [-] MrYellowP|2 years ago|reply
[+] [-] ijhuygft776|2 years ago|reply
Total surveillance is just a matter of time...
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] JCharante|2 years ago|reply
5 figures doesn't sound like much for an organized crime network
[+] [-] Log_out_|2 years ago|reply
[+] [-] Krasnol|2 years ago|reply
I mean, this will be worse if the tech advances no?
[+] [-] KMnO4|2 years ago|reply
The Zeiss site is a much better read:
https://www.zeiss.com/corporate/en/about-zeiss/present/newsr...
In summary:
1) ZEISS unveils holographic Smart Glass at CES 2024, both for displays/projection/filtering, but also another component which is a holographic camera
2) The holocam works by utilizing coupling, decoupling, and light guiding elements to redirect incident light to a concealed sensor, eliminating the need for visible cutouts or installation spaces in visible areas.
3) ZEISS doesn’t plan to be manufacturer, so other companies can use the tech
[+] [-] lawlessone|2 years ago|reply
So you can't just stick on any existing window or pane of glass? That's good.
[+] [-] swyx|2 years ago|reply
---
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
what? holy shit?
[+] [-] 0xNotMyAccount|2 years ago|reply
> coupling, decoupling and light guiding elements to divert incident light to a concealed sensor
So, there's a camera in the dash looking up at the windshield and focusing where it expects to see a face, thereby using the windshield as a reflector? And maybe there's some additional etching and deposited films in the windshield to support the angles required?
And perhaps you can put cameras elsewhere, and similarly subtly modify the windshield or other glass to look at other things as well?
[+] [-] Groxx|2 years ago|reply
Huh. Seems like it should be fairly easy to find info on then... though some googling around makes me think they might just be referring to their more general diffraction gratings and whatnot.
[+] [-] ChuckMcM|2 years ago|reply
Second, #3 is always a red flag for me. It is sometimes code for "we can do this in the lab but we have no idea how one would manufacture it." A similar analogy is "we've got this great idea for a program you can license but no one here knows how to actually code it up."
Third, the impact of this going mainstream would be hard to underestimate. All those people working on transparent displays like they do in sci-fi movies? Yup they could do that. A video conference system with solid eye contact (mentioned in a couple of places) sure you could do that too. A mirror that could show you wearing different clothes? Yup I could see how that would be coded.
That #3 though. That is what tempers my enthusiasm. Did I miss any announcement that they had a display at CES? (or was it just an announcement) If the former I would seriously consider flying over to Vegas to check this out.
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] xyzzy_plugh|2 years ago|reply
The principal issue will be gathering enough energy. A well lit source like a bathroom mirror (mirror behind the light guide) could work pretty well I'd wager. If the light guide is too efficient then it will appear opaque, so there is a trade-off.
I find "turns any window" pretty misleading. Unless I'm missing something this needs very special glass or at the very least a special coating/laminate.
For folks worried about privacy, it will almost always be more convenient and cost-effective to install a tiny spy camera somewhere.
Zeiss isn't going to aspire to sell cheap glass on razor thin margins.
[+] [-] wil421|2 years ago|reply
There’s a facility across from the Autobahn that was so sensitive trucks going by would throw off their machines. They had to put padding on the autobahn to prevent it. This was after they put the foundation on some kind of suspension. My coworker said they hire the most PHDs in Europe.
[+] [-] fnordpiglet|2 years ago|reply
[+] [-] stronglikedan|2 years ago|reply
[+] [-] haswell|2 years ago|reply
While there are some really interesting potential applications for this tech, it is also more than a little disconcerting.
The ubiquity of camera phones and the emergence of tech like those Meta glasses is already pushing us to disconcerting (albeit interesting and in some cases very useful) places, but some of these cutting edge concepts worry me. WiFi seeing through walls also comes to mind…
[+] [-] internetter|2 years ago|reply
[+] [-] pedalpete|2 years ago|reply
People didn't like that Google Glass could always be filming, now we don't even have a physical camera.
Rayban/Meta (I believe) have a sensor to detect that the wearer has not attempted to cover the light which shows that the camera is in use, but how will that work when every piece of glass is a camera.
[+] [-] mrandish|2 years ago|reply
The total lack of any deeper information beyond the bold yet vague claims in the press release is frustrating. The PR makes it sound like a miraculous breakthrough destined to change everything. The source release on the Zeiss site only adds two bits of info.
> "The transparency of the holographic layer has only a minimal effect on the brilliance of the image reproduction. It is also possible to detect spectral components as additional information to complement the visible image. The resulting data provide insights into environmental contamination such as air pollution and UV exposure."
However, experience shows that in reality bold+vague claims like this inevitably come with significant trade-offs and constraints which limit its applications (little things like cost, power, fidelity, size, speed, etc). This is especially true in early implementations of new tech. That said, it may still be both interesting and useful. Unfortunately, we have no way to even think about how it might be useful because Zeiss marketing has chosen to play 'hide the ball' instead of just releasing a technical explainer outlining relevant trade-offs, limitations, etc.
If I was talking to someone from Zeiss my first questions would be about how much the additional components impact the optical characteristics of the glass, what the resolution of the resulting image data is, how large are the components needed at the edges and how far away can they be from the capture zone? Then, of course, how the output of the resulting imaging system maps into traditional camera/lens metrics like f-stops, aperture, imager size/density, gain, focal length, etc. Zeiss is an optics company after all.
[+] [-] mannykannot|2 years ago|reply
The paper mentioned in the Description section can be read here: https://www.academia.edu/52566311/_title_Volume_phase_hologr...
My guess is that each of the items in patent diagram labeled 20 and 22 are gratings of this sort, perhaps with the fringes angled at 45 degrees.
Some additional info may be gleaned from the one patent citing this one, invented by one of the co-inventors of the latter: https://patents.google.com/patent/DE102019206354A1/en
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] pushcx|2 years ago|reply
[+] [-] JumpCrisscross|2 years ago|reply
This is pretty cool.
[+] [-] mrexroad|2 years ago|reply
[+] [-] aftbit|2 years ago|reply
1: https://www.pnas.org/doi/full/10.1073/pnas.2024468118
[+] [-] nickpinkston|2 years ago|reply
https://www.youtube.com/watch?v=NORPeCcIXRQ
[+] [-] raphman|2 years ago|reply
https://www.microsoft.com/applied-sciences/uploads/publicati... https://www.microsoft.com/applied-sciences/projects/the-wedg...
[+] [-] teeray|2 years ago|reply
Retailers, marketers, and data brokers are salivating.
[+] [-] autoexec|2 years ago|reply
It'll sure be distracting when it's the windshields of our cars, but I do look forward to the legal drama when companies get sued for painting their "holographic" ads on top of the adspace other people already paid to pollute with their own advertisements.
[+] [-] thr0waway987|2 years ago|reply
[+] [-] ericfrederich|2 years ago|reply
[+] [-] tqi|2 years ago|reply
Minor aside, but does anyone actually care about this? Forever ago, I was told to try and look into the camera in order to project eye contact during video calls, but now that just seems like a cultural hangup that arose from people not being familiar with video calling. Now that it is more ubiquitous, I feel like we all have collectively agreed that the eye contact thing is unnecessary?
[+] [-] toomim|2 years ago|reply
You've just acclimated to losing that signal during video chats. Bring it back, and you'll have a richer experience.
More generally, we have evolved special neural hardware to recognize gaze, with a special case for when gaze is directed at our eyes. Gaze is important because it's a sign of where people are attending. And as Herb Simon said: "in the information age, attention is the scarce resource."
[+] [-] sheepscreek|2 years ago|reply
[+] [-] beginning_end|2 years ago|reply
[+] [-] wkat4242|2 years ago|reply
I'm sure it can be used for creepiness as well but I see the benefits too.
[+] [-] ChrisMarshallNY|2 years ago|reply
But even relatively poor image quality might be cool, if you had a camera built into your monitor, so you could do direct-eye-contact video calls.
[+] [-] crazygringo|2 years ago|reply
I see some references to "light guides" and words like "coupling" but I don't know what those mean at all, and all of my googling is not helping to explain how light guides embedded in a thin, mostly transparent layer could possibly be used to project a hi-res holographic 3D light field out of a piece of glass. How big is each guide? What is it exactly -- what is it actually made of? How is it shaped? How are they arranged? How are they illuminated? How do you manufacture something like this?
Can anyone ELI15 how this works?