So VR is hard enough - to avoid jitter that makes users feel sick you have to respond to a user's head movement, render a new frame with the new information the user should see, respond to any button presses, then draw the frame, in under 14-20ms.
Magic Leap was always a much more difficult problem... they have to respond to a user's head movement, parse the scene the user is looking at (which could be anything), figure out what to draw and where to draw it on the user's environment, then render, all in the same 14-20ms window.
Compounding that they have to do it with a much weaker CPU/GPU/battery than Oculus and friends, which use a phone, or are tethered to a PC with a $1,000 GPU. You wear Magic Leap on your head, no cables.
On its face I was always sort of surprised to hear about the good demos; it always seemed like such a difficult problem, and I know the VR folks are having a tough enough time with it.
That's not the big unsolved problem with augmented reality. Those are all problems VR systems can already solve with enough money and transistors behind them.
The AR big problem is displaying dark. You can put bright things on the display, but not dark ones. Microsoft demos their VR systems only in environments with carefully controlled dim lighting. The same is true of Meta. Laster just puts a dimming filter in front of the real world to dim it out so the overlays show up well.
Is there any AR headgear which displays the real world optically and can selectively darken the real world? Magic Leap pretended to do that, but now it appears they can't. You could, of course, do it by focusing the real scene on a focal plane, like a camera, using a monochrome LCD panel as a shutter, and refocusing the scene to infinity. But the optics for that require some length, which means bulky headgear like night vision glasses. Maybe with nonlinear optics or something similarly exotic it might be possible. But if there was a easy way to do this, DoD would be using it for night vision gear.
Two of the major unsolved problems he talks about are latency and the ability to draw black - I would be surprised if magic leap had solved both of these alone and in secret.
Their vapid press releases didn't inspire confidence either.
So VR is hard enough - to avoid jitter that makes users feel sick you have to respond to a user's head movement, render a new frame with the new information the user should see, respond to any button presses, then draw the frame, in under 14-20ms.
A bit of a tangent, but: for some people, it's impossible for VR to avoid making them feel sick. It's fundamental to them wearing a VR headset rather than a technical challenge to be overcome. It's related to the fact that VR headsets can't project photons from arbitrary positions with arbitrary angles towards your eyes (i.e. a screen is planar, but the real world is a 3D volume). Turns out, evolution has ensured our bodies are very good at determining the difference between a screen's projection and the real world, resulting in a sick feeling when there's a mismatch.
I think that when people accept it's inevitable some subset of users will get sick, the VR ecosystem will grow at a faster rate.
I worked on Hololens stuff at Microsoft. It does everything you describe, and IMO does it really well. It's fairly light and is wireless. The tracking is excellent. A lot of the scene parsing is done in special hardware, with low latency. It updates a persistent model of the environment, so it isn't rebuilding everything every frame.
You don't need to parse the environment from scratch 60 times every second. As long as you get head tracking right you can just focus on what's moving and assume it will keep moving the same way. Further the demos all seem to be around a fairly static environmenment. Remeber you don't need to render the background so a simplified internal model works fine with AR.
If it works near a closeline blowing in the wind that's really hard and impressive.
The problem is even worse. In VR you're competing on latency with human muscle feedback and what your vestibular system is telling you.
In AR, you're competing with on latency with the human visual system that you're trying to augment, which is a race you can't win.
The only thing you can do is try to be very fast (<= 10 ms) so the latency becomes unnoticeable. Unfortunately right now this isn't possible unless you custom-engineer everything in your vision path optimized for latency. Fun R&D project but enormously time- and capital intensive with no guarantees for success.
A trick i've heard at least one of the VR devices is doing isn't re-rendering the entire scene but since most of the movement is just head tracking, it renders a larger buffer and scrolls around in that buffer between frames to get a quicker response.
The latency / motion sickness issues probably aren't as bad, since you still see the surrounding environment to get a stable bearing. The display tech sounds very hard, though.
I could never really figure out how the variable focus lightfield worked.
I sort of assumed it was like a stack of images in the Z plane. So not only are they doing everything you mentioned, they are also rendering the 3D scene in slices?
Isn't the head-tracking separate from the graphics rendering? I mean, you could just render onto an area that's somewhat larger than your fov, and select some area every couple of ms based on the head movement.
Not surprising, given that the hype train was billowing at full steam. A friend of a friend interviewed there and eventually walked away saying "Wow. That is a lot of cool-aid"
Meanwhile, teams like Oculus are reportedly trying to sidestep AR by investing in "mixed reality", which will bring live camera feeds in to VR, with screens still covering both eyes.
Wow. So if Magic Leap is vaporware, does that leave Microsoft as the only ones publicly making progress in AR? Plenty of companies are shipping VR, but AR ends up being the future, HoloLens is building up quite a lead.
Seems like SnapChat isn't doing too bad. Sure, maybe their tech isn't as advanced... but they've begun solving the equally hard problem of getting it into people's hands naturally and getting them to use it and even pay for it.
There's also Meta which is taking preorders for its second developers kit at $949. It has significantly larger field of view (90 degrees) but must be tethered to a computer
"Magic Leap is vaporware"
Where are you getting that? The article seems to imply that feature to feature they are matching the Hololens and that is the worst case scenario.
Edit: On second thought you maybe meeting the exact definition of Vaporware in that they are not manufacturing it yet[1]. No indication that they won't be able to though.
Hololens has the lead but whoever get's to the consumer market first will still have a huge advantage. The only thing stopping MS from getting to that point is having a consumer friendly price.
ODG has had AR glasses on the market for longer than Hololens, with an actual glasses form factor, running on Android, and should be announcing their new model at CES2017.
I went to https://membership.theinformation.com/subscribe because it looks like something I'd pay for. I hate the trend of not putting pricing upfront. I'm just going to assume I can't afford it.
Yep. The Information did a ton of deep investigative work on this. If you find this article interesting you should consider supporting them by subscribing.
Sadly the way journalism works today is one outlet does a ton of time-consuming, expensive digging. Then within minutes of posting, every other outlet copies the story
Karl Guttag, one of the MagicLeap critics and analysts has an interesting update on what he thinks ML will actually release [0]. The product they might release is actually different from what all the hype has been about: LCOS Microdisplay, two sequential focus planes, Variable Focus, photonics Chip.
> But at least one of these videos — showing an alien invader game that let the wearer of the supposed headset or glasses make use of real-world objects — was created by visual effects studio Weta Workshop. Prior to today, it was believed Weta had simply created the visual assets for the game. However, The Information reveals the entire video was created by the studio.
Was this ever really in doubt? You don't have to be a physicist or AR expert to note that throughout the video they occlude bright background colours with dark AR elements - somehow projecting 'black light'. Whilst I wouldn't expect the man on the street to pick this up it would be nice if journalists about to pen a breathless puff piece would at least give the subject matter 30 seconds of consideration.
I always find it instructive to read glassdoor reviews about a company I'm interested in. There's obvious astroturfing and upset people, but lots of interesting information also leaks out there.
Further behind than expected. I have always suspected that this is a giant money pit with zero chance of success. Another decade or two and maybe technology will catch up to the desire to understand the world around you in real time, but today it's not possible. It's not magic, it's dreaming.
Hmm interesting. So the article makes it sound like one of their faked videos was used to bring in software developers but if they come in and it's simply not working anywhere near the level shown, wouldn't that seem suspicious?
I don't think this necessarily means the company is a scam or anything like that but it seems a little...suspect at least to me if I was interviewing with them.
Curious if the community has an issue with it or not.
I can't help but have similar feelings about this, and I think your question about whether "the community has an issue with it" is especially pertinent.
Many journalists and internet pundits suggest that Magic Leap's technology was obviously over-hyped. See for example the title of The Verge's article. Adding further evidence that it was too good to be true, they used faked videos to attract software developers, as you have pointed out.
So how is it that Google, Alibaba, and Andreessen Horowitz were convinced to hand over hundreds of millions of dollars each to this company? Was it fraud, over-promising on behalf of Magic Leap's founders, or could these huge VC firms not see what everyone else could see? Is it really that easy to secure a billion dollars in funding? The question as to "what's going on here" relates as much to the startup / VC community as it does to Magic Leap.
Nah, an engineer working in VR/AR/Computer vision would know the video was not a truthful representation of the actual thing, but more something they were "aiming at", or a presentation of a vision.
I don't think it really fooled anyone into working for them. Not so sure about investors.
There is a ton of AR/VR hype driven by Adobe After Effects and not real product results. Now whenever I see AR/VR tech that follows this pattern, I stop paying attention. It's unfortunate for the entire industry.
It's not surprising because they are not basing it on any traditional widely manufactured components. It would be as if you promise to ship flat panel displays before LCD production lines have even been created and LCD only existed in the labs.
So, either it's a scam, or, they legitimately plan to create a new industry which requires multiple years capital investment to get production off the ground.
>>The company has raked in $1.4 billion in funding
Let that sink in. Is it a record for startup funding with no product or service released? I don't recall how far along Uber was when they passed that mark.
And what exit number do they need to be considered a success for the VCs?
I thought they had given up a couple of years ago on the gimbal-mounted fiber with the piezo collar. They raised the last round to buy a MEMS fab which I assumed was to do a DLP-like thing down a fibre laminate.
And in fact the Guttag piece linked in this comment page also suggests this (says the DLP demo was the most impressive). I doubt they could use actual DLP (because the TI DLP group is a pain to work with) but it would make sense that their MEMS chip would be also a micro mirror array.
> $1.4 billion in funding at a valuation of $4.5 billion
Man this number doesn't look good in more than one way. Valuations aside, I'm guessing the cumulative dilution for everyone in the company is super high
What is troubling about AR/MR which I feel a lot of people overlook is the limited FOV. From the demo videos shown by Magic Leap (and Hololens) on YouTube it feels as if the entire FOV is available, which is barely the case - the Hololens has an FOV of 120x120 and Magic Leap is said to have a lower angle than that.
Verge covered Hololens and they mention [1] the FOV as a limitation.
Unfortunately Hololens FOV is more like 30×20°. Other AR players are mostly in the same ballpark at the moment. The good thing is that this gives very high pixel density. The other thing is that you don't see the edges unless the 3D render is cut off, so it may not feel as tunnelled as in VR.
Yes, that's the ugly truth. And their PR departments do their best to hide this almost deal-breaking fact. The current experience is underwhelming (limited FOV) in comparision to the faked PR demonstration videos.
It's interesting just how many Magic Leap stories get posted to HN. In the last month or so I must have seen at least 3 stories about Magic Leap, but no other VR/AR technologies.
I don't even remember the last time I saw a story on HN about Oculus, or Vive, or whatever the Sony Playstation VR product is called, nor about the Hololens. But the Magic Leap stories are popping up like mushrooms.
These things often come in clusters, though we usually penalize posts that are from a string of copycat or follow-up posts that don't add new information. I have the impression that this one does, though, albeit cribbed from a better source.
[+] [-] kevinburke|9 years ago|reply
Magic Leap was always a much more difficult problem... they have to respond to a user's head movement, parse the scene the user is looking at (which could be anything), figure out what to draw and where to draw it on the user's environment, then render, all in the same 14-20ms window.
Compounding that they have to do it with a much weaker CPU/GPU/battery than Oculus and friends, which use a phone, or are tethered to a PC with a $1,000 GPU. You wear Magic Leap on your head, no cables.
On its face I was always sort of surprised to hear about the good demos; it always seemed like such a difficult problem, and I know the VR folks are having a tough enough time with it.
[+] [-] Animats|9 years ago|reply
Is there any AR headgear which displays the real world optically and can selectively darken the real world? Magic Leap pretended to do that, but now it appears they can't. You could, of course, do it by focusing the real scene on a focal plane, like a camera, using a monochrome LCD panel as a shutter, and refocusing the scene to infinity. But the optics for that require some length, which means bulky headgear like night vision glasses. Maybe with nonlinear optics or something similarly exotic it might be possible. But if there was a easy way to do this, DoD would be using it for night vision gear.
[+] [-] gonehome|9 years ago|reply
http://blogs.valvesoftware.com/abrash/why-you-wont-see-hard-...
Two of the major unsolved problems he talks about are latency and the ability to draw black - I would be surprised if magic leap had solved both of these alone and in secret.
Their vapid press releases didn't inspire confidence either.
[+] [-] sillysaurus3|9 years ago|reply
A bit of a tangent, but: for some people, it's impossible for VR to avoid making them feel sick. It's fundamental to them wearing a VR headset rather than a technical challenge to be overcome. It's related to the fact that VR headsets can't project photons from arbitrary positions with arbitrary angles towards your eyes (i.e. a screen is planar, but the real world is a 3D volume). Turns out, evolution has ensured our bodies are very good at determining the difference between a screen's projection and the real world, resulting in a sick feeling when there's a mismatch.
I think that when people accept it's inevitable some subset of users will get sick, the VR ecosystem will grow at a faster rate.
[+] [-] psyc|9 years ago|reply
[+] [-] Retric|9 years ago|reply
If it works near a closeline blowing in the wind that's really hard and impressive.
[+] [-] rsp1984|9 years ago|reply
In AR, you're competing with on latency with the human visual system that you're trying to augment, which is a race you can't win.
The only thing you can do is try to be very fast (<= 10 ms) so the latency becomes unnoticeable. Unfortunately right now this isn't possible unless you custom-engineer everything in your vision path optimized for latency. Fun R&D project but enormously time- and capital intensive with no guarantees for success.
[+] [-] simcop2387|9 years ago|reply
[+] [-] highd|9 years ago|reply
[+] [-] markingram|9 years ago|reply
[+] [-] monk_e_boy|9 years ago|reply
I sort of assumed it was like a stack of images in the Z plane. So not only are they doing everything you mentioned, they are also rendering the 3D scene in slices?
[+] [-] ant6n|9 years ago|reply
[+] [-] temuze|9 years ago|reply
Color, Clinkle, Theranos, Magic Leap... they all have that in common. I'd be interested if there are counter examples.
[+] [-] evv|9 years ago|reply
Meanwhile, teams like Oculus are reportedly trying to sidestep AR by investing in "mixed reality", which will bring live camera feeds in to VR, with screens still covering both eyes.
[+] [-] Analemma_|9 years ago|reply
[+] [-] gkoberger|9 years ago|reply
[+] [-] nartam11|9 years ago|reply
[+] [-] pj_mukh|9 years ago|reply
Edit: On second thought you maybe meeting the exact definition of Vaporware in that they are not manufacturing it yet[1]. No indication that they won't be able to though.
[1]: https://en.wikipedia.org/wiki/Vaporware
[+] [-] flukus|9 years ago|reply
[+] [-] tmikaeld|9 years ago|reply
[+] [-] jobigoud|9 years ago|reply
[+] [-] badsock|9 years ago|reply
[+] [-] gumby|9 years ago|reply
[+] [-] minimaxir|9 years ago|reply
[+] [-] voltagex_|9 years ago|reply
Edit: possibly $400USD/year from https://hunterwalk.com/2013/12/17/400-for-the-information-is...
[+] [-] bobbygoodlatte|9 years ago|reply
Sadly the way journalism works today is one outlet does a ton of time-consuming, expensive digging. Then within minutes of posting, every other outlet copies the story
[+] [-] melling|9 years ago|reply
https://news.ycombinator.com/item?id=13135371
[+] [-] Jerry2|9 years ago|reply
[0]: http://www.kguttag.com/2016/12/06/magic-leap-when-reality-hi...
[+] [-] guycook|9 years ago|reply
Was this ever really in doubt? You don't have to be a physicist or AR expert to note that throughout the video they occlude bright background colours with dark AR elements - somehow projecting 'black light'. Whilst I wouldn't expect the man on the street to pick this up it would be nice if journalists about to pen a breathless puff piece would at least give the subject matter 30 seconds of consideration.
[+] [-] beachy|9 years ago|reply
If by "game" they meant gaming potential employees and investors, then the assertion may well have been true.
[+] [-] bane|9 years ago|reply
https://www.glassdoor.com/Reviews/Magic-Leap-Reviews-E799754...
[+] [-] coldcode|9 years ago|reply
[+] [-] BinaryIdiot|9 years ago|reply
I don't think this necessarily means the company is a scam or anything like that but it seems a little...suspect at least to me if I was interviewing with them.
Curious if the community has an issue with it or not.
[+] [-] NuclearFishin|9 years ago|reply
Many journalists and internet pundits suggest that Magic Leap's technology was obviously over-hyped. See for example the title of The Verge's article. Adding further evidence that it was too good to be true, they used faked videos to attract software developers, as you have pointed out.
So how is it that Google, Alibaba, and Andreessen Horowitz were convinced to hand over hundreds of millions of dollars each to this company? Was it fraud, over-promising on behalf of Magic Leap's founders, or could these huge VC firms not see what everyone else could see? Is it really that easy to secure a billion dollars in funding? The question as to "what's going on here" relates as much to the startup / VC community as it does to Magic Leap.
[+] [-] jobigoud|9 years ago|reply
I don't think it really fooled anyone into working for them. Not so sure about investors.
[+] [-] lowglow|9 years ago|reply
[+] [-] cromwellian|9 years ago|reply
So, either it's a scam, or, they legitimately plan to create a new industry which requires multiple years capital investment to get production off the ground.
[+] [-] russdill|9 years ago|reply
[+] [-] WhitneyLand|9 years ago|reply
Let that sink in. Is it a record for startup funding with no product or service released? I don't recall how far along Uber was when they passed that mark.
And what exit number do they need to be considered a success for the VCs?
[+] [-] CardenB|9 years ago|reply
[+] [-] joezydeco|9 years ago|reply
http://www.stereoscopynews.com/hotnews/3d-technology/vr-ar/4...
http://www.businessinsider.com/magic-leap-production-begins-...
[+] [-] gumby|9 years ago|reply
[+] [-] gumby|9 years ago|reply
[+] [-] cocktailpeanuts|9 years ago|reply
Man this number doesn't look good in more than one way. Valuations aside, I'm guessing the cumulative dilution for everyone in the company is super high
[+] [-] relfor|9 years ago|reply
1: https://youtu.be/4p0BDw4VHNo?t=3m
[+] [-] jobigoud|9 years ago|reply
Unfortunately Hololens FOV is more like 30×20°. Other AR players are mostly in the same ballpark at the moment. The good thing is that this gives very high pixel density. The other thing is that you don't see the edges unless the 3D render is cut off, so it may not feel as tunnelled as in VR.
[+] [-] frik|9 years ago|reply
[+] [-] pmoriarty|9 years ago|reply
I don't even remember the last time I saw a story on HN about Oculus, or Vive, or whatever the Sony Playstation VR product is called, nor about the Hololens. But the Magic Leap stories are popping up like mushrooms.
[+] [-] dang|9 years ago|reply
These things often come in clusters, though we usually penalize posts that are from a string of copycat or follow-up posts that don't add new information. I have the impression that this one does, though, albeit cribbed from a better source.
[+] [-] jobigoud|9 years ago|reply
It is, in fact, called the Playstation VR.