They're paving the way for their revolutionary consumer AR eyewear (which may be delayed now apparently). Although maybe we'll see tech from some of their recent AR acquisitions (PrimeSense/Metaio) integrated into the upcoming iPhone, possibly in the form of an advanced depth/surface mapping sensor of sorts.
What's fascinating is Apple has a large team focusing on HCI & AR - what "pinch-to-zoom" and multitouch did for smartphones is what Apple will need to successfully pull off the "big bang" for consumer AR.
Studied the market for AR development solutions recently. A few thoughts:
- Probably using a lot of Metaio tech, a leading AR dev solutions provider Apple acquired/shuttered in May 2015
- Unfortunate naming. Sounds extremely similar to ARToolkit [1], the leading open source/free AR dev solution.
- Markerless is the future of AR, so good on Apple for getting this markerless tech out to compete with Google Tango and Microsoft Hololens. The status quo is currently marker-based recognition using "tags" or QR codes. It's also much harder than marker-based, or "on body" (Snapchat, Facebook AR Studio) recognition. I'd say Google Tango has some of the most impressive SDKs out there for tracking, area learning, and depth perception, but it does require specialized smartphone hardware from Lenovo or Asus, which significantly limits its utility and mkt penetration at this point. Google was just too early with Glass and got burned, but it needs a stronger hardware platform, which is something that Apple can deliver.
- The Unreal-engine AR video demo was cool and the graphics seemed decent for real-time rendering on an iPad, but the real future of AR applications (read: $$$) will come from industrial applications on wearable devices or head mounted displays such as the Hololens, Vuzix, Daqri, or Epson Moverio devices. Examples of industrial/enterprise AR applications: Remote help, complex assembly, pick & pack, line monitoring, materials handling, systems training, etc. This is the market Microsoft and PTC (Vuforia, ThingWorx) are targeting, not the consumer gaming/advertising markets, which use more "basic" forms of AR. This is not to say Pokemon Go/Snapchat do not generate a lot of revenue, but it's very debatable if they can be considered true AR applications.
- Other vendors in the AR development solutions space include: Aurasma (HPE), Blippar, Catchoom, EON Reality, Kudan, Pikkart, Wikitude, among others.
- In conclusion, Apple is currently stuck between a rock and a hard place. It's somewhat late to market with ARKit, will have to compete with Facebook, Google, and Snapchat for consumer AR-oriented developer mindshare, and will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family.
- The best way for Apple to alleviate this jam is to release its own AR glasses/headset, which it is widely rumored to be developing.
> In conclusion, Apple is currently stuck between a rock and a hard place. It's somewhat late to market with ARKit, will have to compete with Facebook, Google, and Snapchat for consumer AR-oriented developer mindshare, and will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family.
See also portable music, phones, tablets …
It's Apple's SOP, and if they've shown anything in the last 10-15 years, entering markets late is not a barrier to them competing
> In conclusion, Apple is . . . somewhat late to market . . . The best way for Apple to alleviate this jam is to release its own AR glasses/headset, which it is widely rumored to be developing.
I'm not sure this line of reasoning holds up. You mention 3 markets:
1. consumer
2. enterprise
3. the "market" for developer mindshare
Apple doesn't especially care about being early or late to any of these markets. Let's take them one by one.
1. There is currently no real consumer market for AR, so Apple is not late. And I don't see any reason why Apple would not be content to wait and see if one develops. It's not clear how releasing an improved version of Google Glass saves Apple from any kind of "jam". People are not clamoring for this type of product, because its value is highly unclear.
2. There is currently no real enterprise market for AR, so Apple is not late. Apple has no history of making industrial equipment. Are you saying Apple is about to take the unusual step of starting a whole new sales channel, targeting a whole new class of buyers, to try to get first-mover advantage in a new and unproven market helping warehouse workers do line monitoring and "remote help"? It seems unlikely.
3. Apple has a huge, strong developer base it is in no danger of losing. Again, it's not clear how offering an AR SDK gets them out of a jam.
If I had to guess as to why they are playing in this area, it may be to help seed the ground with more iOS devs familiar with this tech, so that in the event that the market does develop, they are in a good position to harvest ideas, talent, and apps.
Almost certainly - Metaio presented realtime lighting estimation at InsideAR (their dev conference) in late 2014. It's also been shown by Hans Kaufmann's research at TU Wien in Austria [1] although this has tighter links with other vendors than Metaio.
| Other vendors in the AR development solutions space include
I've had the pleasure of dealing with most of these folks within the last few years. They all have their own specialisms around certain tracking methodologies, pricing models and other features. Evaluate them with your own purposes in mind, and choose the right tool for the job.
| will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family
In this market, Vuforia really does have it fairly cornered - it seems to be a chunk of what PTC are chasing, although they're far from turning a blind eye to B2C.
For AR, to win, you do not have to be the first one to market to win, its who can make the best platform for developers will win. As an AR developer, previous platforms like Vuforia, ARToolkit, and even Tango are either limiting or very hard to work with.
Currently locking the AR to the real world requires a QR code-like mark but they are demoing AR with no mark. Not having a mark requires 3D data of something in the field of view:
As you say, the Wingnut demo had nice content, but the whole experience looked unbelievably bad. Like, "original Wii ads with people jumping behind their couches" bad. If they showed it as a tech demo for integrating VFX in real time preview when shooting a movie I would have bought it 100%.
There's also Sturfee which focuses on outdoor spaces which is a very hard problem for simple AR toolkits like this. Depth is hard and so most of these players cannot do outdoor AR.
sturfee.com
DISCLAIMER: I work at Sturfee and it's really great.
I thought we might be seeing an AR app built into the Camera or Map app.
Nope just a kit for developers to build games only or does the ARkit allow access to the camera app? I'm dying to see innovation from Apple like in the camera app when you open it there's an option to view how things look in the past(100 years ago what did this spot in New York City look like.. where you can take a pic of yourself in the past).
I'm sure there's oodles that can be done with an AR Camera.
Apple and the others to me seem to be slow to innovate.
Even ignoring that you'll need multiple photos to do the 3D reconstruction you need for augmented reality, that 100 year old photos may be lacking in resolution, and giving a few decades of leeway either way for that "100 years ago" I doubt even Google has enough data to have a reasonable chance of having a photo for random locations you might point your camera at.
As a developer, this was the most exciting thing for me from today's WWDC opening speech. Can't wait to try this out myself. The demo they did with the AR game was really neat.
I find it funny that Apple is just now releasing an AR SDK as when the iPhone first came out there were a lot of AR apps available. Most of them were pretty boring, just games like shooting spaceships or POI explorers, but they didn't look that much more primitive than the apps demoed today. I wonder why they didn't jump on AR back then.
How does it compare to Google Tango? Tango needs special hardware, but ARKit is not. Even if it required, there would be a definitely device (iPhone or iPad). Traction wise Apple could win, but what about performance and accuracy wise?
For now it's more like how Snapchat's latest world filters work (try them out). Basic environment tracking, good enough for a smooth experience, considering you keep your camera steady.
Tango takes the tracking to the next level. Maybe future iPhones will have that kind of IMU and other sensors, allowing Apple to expand their AR API.
HealthKit is still going. They announced some big wins with HealthKit on the Apple Watch but they weren't included in the keynote. More than likely, there wasn't enough time considering the keynote without any HealthKit announcements was already longer than normal.
Does anyone have resources for "State of the art" algorithms and research that is used to do the sensor fusion (camera frames + motion sensor data)? I found this but not sure its the most up to date. I'm also looking for more detail in terms of code:
(the last one being released just a few days ago...)
Of course there are a lot more resources out there and a lot of the cutting edge research is involved with deep learning techniques (starting with e.g. https://github.com/alexgkendall/caffe-posenet).
You may also want to watch out for this conference http://cvpr2017.thecvf.com/ scheduled for the end of this month and is usually a good source of cutting edge research...
Google has at least three AR products: Glass is still in business for enterprise, Tango tablets and phones are finally out, and future versions of Daydream are supposed to be capable of inside-out tracking via an AR tracking system.
Microsoft has the HoloLens, which technology they have also cannibalized for a series of headsets they are dubbing "Mixed Reality" for the depth camera pass-through they are performing.
HTC is supposedly developing their own AR/inside-out system.
Leap Motion, Intel, Qualcomm, Meta, and Magic Leap are still considered in the running, with varying levels of believability.
And then there are a slew of different companies that are doing AR solutions with either the standard webcam (e.g. Facebook, Snap, Vuforia, Wikitude) or their own, standalone camera (e.g. Zed, Occipital).
[+] [-] synaesthesisx|8 years ago|reply
What's fascinating is Apple has a large team focusing on HCI & AR - what "pinch-to-zoom" and multitouch did for smartphones is what Apple will need to successfully pull off the "big bang" for consumer AR.
[+] [-] scarlac|8 years ago|reply
I'm curious, what have you read/seen that makes you think this unknown product is delayed before being announced?
[+] [-] roymurdock|8 years ago|reply
- Probably using a lot of Metaio tech, a leading AR dev solutions provider Apple acquired/shuttered in May 2015
- Unfortunate naming. Sounds extremely similar to ARToolkit [1], the leading open source/free AR dev solution.
- Markerless is the future of AR, so good on Apple for getting this markerless tech out to compete with Google Tango and Microsoft Hololens. The status quo is currently marker-based recognition using "tags" or QR codes. It's also much harder than marker-based, or "on body" (Snapchat, Facebook AR Studio) recognition. I'd say Google Tango has some of the most impressive SDKs out there for tracking, area learning, and depth perception, but it does require specialized smartphone hardware from Lenovo or Asus, which significantly limits its utility and mkt penetration at this point. Google was just too early with Glass and got burned, but it needs a stronger hardware platform, which is something that Apple can deliver.
- The Unreal-engine AR video demo was cool and the graphics seemed decent for real-time rendering on an iPad, but the real future of AR applications (read: $$$) will come from industrial applications on wearable devices or head mounted displays such as the Hololens, Vuzix, Daqri, or Epson Moverio devices. Examples of industrial/enterprise AR applications: Remote help, complex assembly, pick & pack, line monitoring, materials handling, systems training, etc. This is the market Microsoft and PTC (Vuforia, ThingWorx) are targeting, not the consumer gaming/advertising markets, which use more "basic" forms of AR. This is not to say Pokemon Go/Snapchat do not generate a lot of revenue, but it's very debatable if they can be considered true AR applications.
- Other vendors in the AR development solutions space include: Aurasma (HPE), Blippar, Catchoom, EON Reality, Kudan, Pikkart, Wikitude, among others.
- In conclusion, Apple is currently stuck between a rock and a hard place. It's somewhat late to market with ARKit, will have to compete with Facebook, Google, and Snapchat for consumer AR-oriented developer mindshare, and will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family.
- The best way for Apple to alleviate this jam is to release its own AR glasses/headset, which it is widely rumored to be developing.
[1] https://artoolkit.org/
[+] [-] oliyoung|8 years ago|reply
See also portable music, phones, tablets …
It's Apple's SOP, and if they've shown anything in the last 10-15 years, entering markets late is not a barrier to them competing
[+] [-] sandover|8 years ago|reply
I'm not sure this line of reasoning holds up. You mention 3 markets:
1. consumer
2. enterprise
3. the "market" for developer mindshare
Apple doesn't especially care about being early or late to any of these markets. Let's take them one by one.
1. There is currently no real consumer market for AR, so Apple is not late. And I don't see any reason why Apple would not be content to wait and see if one develops. It's not clear how releasing an improved version of Google Glass saves Apple from any kind of "jam". People are not clamoring for this type of product, because its value is highly unclear.
2. There is currently no real enterprise market for AR, so Apple is not late. Apple has no history of making industrial equipment. Are you saying Apple is about to take the unusual step of starting a whole new sales channel, targeting a whole new class of buyers, to try to get first-mover advantage in a new and unproven market helping warehouse workers do line monitoring and "remote help"? It seems unlikely.
3. Apple has a huge, strong developer base it is in no danger of losing. Again, it's not clear how offering an AR SDK gets them out of a jam.
If I had to guess as to why they are playing in this area, it may be to help seed the ground with more iOS devs familiar with this tech, so that in the event that the market does develop, they are in a good position to harvest ideas, talent, and apps.
[+] [-] chrisdun|8 years ago|reply
Almost certainly - Metaio presented realtime lighting estimation at InsideAR (their dev conference) in late 2014. It's also been shown by Hans Kaufmann's research at TU Wien in Austria [1] although this has tighter links with other vendors than Metaio.
| Other vendors in the AR development solutions space include
I've had the pleasure of dealing with most of these folks within the last few years. They all have their own specialisms around certain tracking methodologies, pricing models and other features. Evaluate them with your own purposes in mind, and choose the right tool for the job.
| will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family
In this market, Vuforia really does have it fairly cornered - it seems to be a chunk of what PTC are chasing, although they're far from turning a blind eye to B2C.
Edit: Grammar fail.
[1] https://www.ims.tuwien.ac.at/research/virtual-and-augmented-...
[+] [-] mendeza|8 years ago|reply
[+] [-] asmithmd1|8 years ago|reply
https://studio.thingworx.com/
Currently locking the AR to the real world requires a QR code-like mark but they are demoing AR with no mark. Not having a mark requires 3D data of something in the field of view:
https://www.youtube.com/watch?v=-S-4oJ8C_fo
skip to minute 19 to see the markless demo
[+] [-] Jare|8 years ago|reply
[+] [-] artfiend|8 years ago|reply
[+] [-] pducks32|8 years ago|reply
[+] [-] runesoerensen|8 years ago|reply
[+] [-] wsinks|8 years ago|reply
[+] [-] stesch|8 years ago|reply
[+] [-] paul9290|8 years ago|reply
Nope just a kit for developers to build games only or does the ARkit allow access to the camera app? I'm dying to see innovation from Apple like in the camera app when you open it there's an option to view how things look in the past(100 years ago what did this spot in New York City look like.. where you can take a pic of yourself in the past).
I'm sure there's oodles that can be done with an AR Camera.
Apple and the others to me seem to be slow to innovate.
[+] [-] basisword|8 years ago|reply
[+] [-] Someone|8 years ago|reply
[+] [-] htormey|8 years ago|reply
[+] [-] huangc10|8 years ago|reply
[+] [-] satysin|8 years ago|reply
[+] [-] quantumwannabe|8 years ago|reply
[+] [-] tuyguntn|8 years ago|reply
[+] [-] asadlionpk|8 years ago|reply
Tango takes the tracking to the next level. Maybe future iPhones will have that kind of IMU and other sensors, allowing Apple to expand their AR API.
[+] [-] agumonkey|8 years ago|reply
[+] [-] dpkonofa|8 years ago|reply
[+] [-] enos_feedler|8 years ago|reply
http://user.it.uu.se/~thosc112/pubpdf/holsgs2006.pdf
[+] [-] steinomri|8 years ago|reply
(the last one being released just a few days ago...)
Of course there are a lot more resources out there and a lot of the cutting edge research is involved with deep learning techniques (starting with e.g. https://github.com/alexgkendall/caffe-posenet).
You may also want to watch out for this conference http://cvpr2017.thecvf.com/ scheduled for the end of this month and is usually a good source of cutting edge research...
[+] [-] Stanleyc23|8 years ago|reply
[+] [-] josecastillo|8 years ago|reply
https://developer.apple.com/documentation/avfoundation/avdep...
[+] [-] huangc10|8 years ago|reply
[+] [-] aSig|8 years ago|reply
[+] [-] rfinman|8 years ago|reply
[+] [-] tarentel|8 years ago|reply
[+] [-] izacus|8 years ago|reply
[+] [-] aylmao|8 years ago|reply
Facebook made a big deal in F8 out of adding cameras to all its applications. They're in the AR game big time.
https://www.nytimes.com/2017/04/18/technology/mark-zuckerber...
[+] [-] moron4hire|8 years ago|reply
Microsoft has the HoloLens, which technology they have also cannibalized for a series of headsets they are dubbing "Mixed Reality" for the depth camera pass-through they are performing.
HTC is supposedly developing their own AR/inside-out system.
Leap Motion, Intel, Qualcomm, Meta, and Magic Leap are still considered in the running, with varying levels of believability.
And then there are a slew of different companies that are doing AR solutions with either the standard webcam (e.g. Facebook, Snap, Vuforia, Wikitude) or their own, standalone camera (e.g. Zed, Occipital).
[+] [-] wtfishackernews|8 years ago|reply
[+] [-] mattbettinson|8 years ago|reply