I love all the experimentation in this area but I'm afraid systems like this are going to fall into a sort of uncanny valley of VR. Your brain is incredibly good at picking out the discrepancies between the real world and a high-resolution 120fps image with surround sound - just as it is incredibly good at recognizing objects and motions masquerading as human.
It's like going from 2D/24FPS to 3D/48FPS. You'd think it would be a huge increase in perceived realism but it's quite the opposite. What you want is magic window, which is what you get when a film or even a game really pulls you in and your perception sort of synchronizes with something completely out of step with your physiology and totally unbelievable for many other reasons. Instead, you get a feeling of unreality or surreality, and the final gap between the media and your perception is unbridgeable.
I think that's part of why VR/3D/HFR stuff has always been so unconvincing, and why it's no coincidence that it's finally only catching on when we pair it with totally unrealistic games and demos. You need the fantasy. Because it's so clearly not reality, and anything that attempts to replicate reality too closely will be rejected.
Anyway, cool tech though, and I'm sure I'll enjoy using it some day. But for now I think the foreseeable future of VR is in virtual environments, not in the duplication or capture of real ones.
You're talking about Presense, which is the point at which your brain flips and is convinced what its seeing is real.
Michael Abrash has a wonderful deck he wrote while at Valve about what it will take for VR to create Presense in the viewer. It's worth a read if you care about how we'll navigate past the uncanny valley of VR you describe: http://media.steampowered.com/apps/abrashblog/Abrash%20Dev%2...
I;m not entirely sure if this is adding to or disagreeing with the points you make…
A few weeks ago I commented on some thread about robot maids and such. My point was and is that when we picture a far off idea that we suspect is coming (AI, VR, human-less hotel-resorts) we need mental placeholders where people go now. Technology tends to replace people in the way reddit replaces librarians but we can’t picture reddit in advance until the world wide web or something similar exists. So, flying cars, robot mades and sexy female AIs. They’re placeholders for future automation or changes that make the need go away rather then meeting the need.
Back to Jump. Our starting point is a made up idea of what VR entertainment looks like. Game of Thrones in VR. Jerry Springer in VR. Xuxa in VR.
You make a valid point that this won’t work. I totally agree. But, I’m fairly optimistic about the ingenuity of artists. Lots of things run into these kinds of problems. A conversation between two actors pretending to be doctors is not uncanny, but there are arts to making it palatable. They will figure out POV issues. Should you be a character or an invisible eye like in flat films? Maybe they could use filters to make it look like ‘A Scanner Darkly’ to get over the issue you describe.
A few months ago I was walking around the Unreal 4 Paris Apartment demo [0] with a Occulus DK2. The resolution isn't quite there, but it definitely felt real. I think that VR can definitely provoke feelings of reality, at least if the people working on it keep going forward in resolution like they have been.
My experience of The Hobbit was that the 24fps version felt like being in Middle Earth, and the 48fps felt like being on the film set - which strictly speaking was the more realistic experience, but far less satisfying.
You might be interested in this talk that Michael Abrash gave about 2 months ago at Facebook F8 [1]. It starts at around 00:42:20.
It is a fascinating talk where he provides a number of interesting demonstrations regarding how the brain interprets information, including depth perception, color interpretation, movement, auditory cues [2] etc. Highly recommended.
[2] Absolutely fascinating. What you hear depends on what you are looking at, not just on what you are hearing. This demonstration starts at approximately 01:02:00 - https://www.youtube.com/watch?v=UDu-cnXI8E8&t=3720
Something that's maybe not clear from that page, but was mentioned in the Google IO keynote: this is not just stitching of multiple video streams.
Google does some heavy-duty machine learning / computer vision on their servers to extract 3D information from these video streams (they mentioned having depth data). Then they presumably re-generate seamless stereoscopic 360-degree video from this 3D model. That's how they can get stereo from mono data.
It's something like those hyperlapse videos from Microsoft, they also first do number crunching on lot of video data to generate 3D model and then use this to fill in missing data.
There are a couple of consumer ready 360 camera options available to purchase today ($300-$400), albeit not 3d steroscopic. I have personally played with the ricoh theta and the kodak pixpro and think they are good products. Here are some links if you want to check them out, I don't work for any of these companies but am a VR enthusiast...
Looking at the top comments - I guess I'm completely crazy. I 100% thought this was a late April's fools. They legitimately think people are going to buy a 3D printed (~$12 hat), that holds about $5k worth of gopros? So... this is like another Nexus Q/really smart guy's horrible idea that's just going to disappear with no explanation?
And why not? It's not about the hardware - it's about the videos they can create with a simple tool (and a few thousand to buy the cameras), which puts this technology within reach of the semi-professionals and higher-end hobbyists, instead of specialized equipment in Google Streetview cars. Like how Cardboard gives simple VR to anyone with a smartphone. It's not about whether it's a killer app, it's about planting the idea into people's minds - and eventually, about creating content. Because that's Google's trade - content. Cool 3D videos = more views than a regular gopro video = more advertising income.
Looks great, though as other commenters have pointed out, 16 GoPros makes for an expensive rig.
I've been watching this space for a while and pretty much all of the consumer varieties (usually Kickstarter-backed) have pretty terrible video quality, so it looks like I'll be sitting this one out for a while yet.
As others have said, the cost of this is peanuts compared to professional cameras.
I'd bet one of the first killer-apps for VR will be sports: Imagine several of these rigs set up around (and over) a football field, so that spectators can choose to "be" at the right point on the field to observe the game, and have complete freedom to look around the field.
The interesting aspect here is the iPhone availability for Cardboard and the use of Youtube as the platform for 3D VR.
Youtube already had stereoscopic support, but there were not many clients available, content was limited and difficult to create with 3rd party solutions that all cost money. It never took off and couldn't ride the wave of the studio and CE company driven 3D momentum, which quieted down after a while anyway.
This time Google ticks all the right boxes. Client devices on iOs and Android. The videoplayer everyone uses. A $5 cardboard enabler. A capture platform with GoPro and free assembly software.
I'm looking forward to trying out Cardboard on iOS. Anyone have a recommendation for a good viewer manufacturer? (I'm a bit surprised at how much some of the kits are, given they're made of, er, cardboard).
16 GoPros? Sounds expensive. Certainly not something I would buy, since it would gather dust on the shelves along with my other gadgets after a few initial uses.
That being said, this may be the right time to reinvest in 360 degree stereoscopic video since the viewing technology is finally catching up. Point Grey has been making similar spherical cameras since the 90's [1]
Pretty cool idea overall. Other questions I had: Does it have a single data interface? How do you charge this behemoth.
The 16-GoPro rig isn't for a home user, it's for YouTube channel owners and other video publishers who are making money from their videos and want some more immersive content.
Gopro's are what $400?
That'd be $6400, which yeah may be out of the price range of your average consumer but my brother who does film work will routinely spend more than $2000 on a lens. If there is a market for these kind of videos, then it's a drop in the bucket.
Has anyone see the opposite of this done - where you have a 360 degree array of cameras pointed at a target? This would allow the viewer to pan around the point of interest during an action sequence.
I'd like to see an array of cameras using quadcopters flying around a target - say a whale jumping out of the water or a football player. This would allow the viewer to see all side of the action. Great for sports playbacks.
I don't think a quadcopter can hold cameras with enought distance apart from each other in order for a shot to be any useful, I mean think about it, whale jumping or a football players bodies are much bigger then a quadcopter radius, so the 360 views will most probably be useless
I was just thinking about how the idea has been around a long time. Even back in the early 2000s I was working on a project that used a camera on a special tripod that would let you take pictures in 360 degrees and stitch them together with some software in post (quicktime I think?).
But I think the neat thing is using readily-available GoPro cameras and stitching the video together quickly. That takes a lot of processing power to do on the fly.
I'm curious how they are planning to record audio. Binaural sound can give a sense of presence even without video. The common approach to recording it is by placing microphones in your own or a fake heads ears and that won't work in VR. It'd be interesting to see an array of microphone elements perhaps in a cleverly shaped mold to go along with this.
I wonder if this is what gets VR to take off. Seems like Oculus will be way too expensive for practical use. Once people start getting content out there all you'll need is a $20 cardboard kit to take advantage of that.
You could have several of these say at a music festival hanging around and watch them from your house at your leisure without having to deal with heat and crowds. And worse you go see an artist on one stage and realize they're a dud and you have to now travel to another stage. With this, boom, you can switch in an instant.
I think LightField camera technology is ultimatly going to win the VR camera wars, it provides much better depth information. If you havn't seen the otoy tech yet it is worth checking out...
This is really exciting. I came up with the same design last year and have been really wanting to make a rig like this. My gym had treadmills with screens, and you could select a virtual hike through places like Oregon and New Zealand. So I really want to capture some of these hikes with this rig, and replay them while walking or running on an Omni [1]. I'm really excited about the potential for VR to turn exercise into an enjoyable experience, instead of a chore.
And one day I would love to travel around the world with a team and film a documentary like Samsara. [2]
The most realistic virtual reality I have ever participated in was the Duke immersive Virtual Environment (DiVE). There is a program that simulates a kitchen. There are cereal boxes, silverware, etc that you can pick up and even throw across the room. When I opened the refrigerator door, I instinctively moved my body out of the way. The crazy thing is that it's not even the best tech out there. The resolution was crappy and the physics were a little bit off. But if you stopped actively paying attention to the details, even for just a moment, it was enough to make some part of your brain think it's real. VR doesn't have to much better (if at all) in order to be really immersive.
If you're at Duke or somewhere nearby I highly recommend checking it out. They have visiting hours fairly frequently. http://virtualreality.duke.edu/
This is a really fun hack that really did make me smile and laugh... but I do wonder what the sensor/processing/display latency is like on hardware/software that wasn't originally designed to do this.
I once played with an early Oculus, it basically amounted to something that wasn't really succeeding at tricking my senses other than giving me a headache and making me want to vomit at the same time.
I'm sure similar things were said about early cinema though.
Looking at the picture immediately makes me think of it as an oversized crown or tiara you're supposed to wear on your head. Sort of a "Glass 3.0".
The "Camera Crown" is an entirely impractical idea of course (that is, not the actual idea), but that does lead me to wonder how it's meant to be moved through spaces, if at all.
[+] [-] devindotcom|10 years ago|reply
It's like going from 2D/24FPS to 3D/48FPS. You'd think it would be a huge increase in perceived realism but it's quite the opposite. What you want is magic window, which is what you get when a film or even a game really pulls you in and your perception sort of synchronizes with something completely out of step with your physiology and totally unbelievable for many other reasons. Instead, you get a feeling of unreality or surreality, and the final gap between the media and your perception is unbridgeable.
I think that's part of why VR/3D/HFR stuff has always been so unconvincing, and why it's no coincidence that it's finally only catching on when we pair it with totally unrealistic games and demos. You need the fantasy. Because it's so clearly not reality, and anything that attempts to replicate reality too closely will be rejected.
Anyway, cool tech though, and I'm sure I'll enjoy using it some day. But for now I think the foreseeable future of VR is in virtual environments, not in the duplication or capture of real ones.
[+] [-] andrewparker|10 years ago|reply
Michael Abrash has a wonderful deck he wrote while at Valve about what it will take for VR to create Presense in the viewer. It's worth a read if you care about how we'll navigate past the uncanny valley of VR you describe: http://media.steampowered.com/apps/abrashblog/Abrash%20Dev%2...
[+] [-] Nadya|10 years ago|reply
https://i.imgur.com/avVTybvh.jpg
The brain is actually fairly easily tricked.
http://blogs.discovermagazine.com/notrocketscience/2011/05/2...
[+] [-] netcan|10 years ago|reply
I;m not entirely sure if this is adding to or disagreeing with the points you make…
A few weeks ago I commented on some thread about robot maids and such. My point was and is that when we picture a far off idea that we suspect is coming (AI, VR, human-less hotel-resorts) we need mental placeholders where people go now. Technology tends to replace people in the way reddit replaces librarians but we can’t picture reddit in advance until the world wide web or something similar exists. So, flying cars, robot mades and sexy female AIs. They’re placeholders for future automation or changes that make the need go away rather then meeting the need.
Back to Jump. Our starting point is a made up idea of what VR entertainment looks like. Game of Thrones in VR. Jerry Springer in VR. Xuxa in VR.
You make a valid point that this won’t work. I totally agree. But, I’m fairly optimistic about the ingenuity of artists. Lots of things run into these kinds of problems. A conversation between two actors pretending to be doctors is not uncanny, but there are arts to making it palatable. They will figure out POV issues. Should you be a character or an invisible eye like in flat films? Maybe they could use filters to make it look like ‘A Scanner Darkly’ to get over the issue you describe.
This is what art does. They’ll figure it out.
[+] [-] nickysielicki|10 years ago|reply
A few months ago I was walking around the Unreal 4 Paris Apartment demo [0] with a Occulus DK2. The resolution isn't quite there, but it definitely felt real. I think that VR can definitely provoke feelings of reality, at least if the people working on it keep going forward in resolution like they have been.
[0]: https://youtu.be/Y6PQ19BEE24
[+] [-] jl6|10 years ago|reply
[+] [-] Arjuna|10 years ago|reply
It is a fascinating talk where he provides a number of interesting demonstrations regarding how the brain interprets information, including depth perception, color interpretation, movement, auditory cues [2] etc. Highly recommended.
[1] https://www.youtube.com/watch?v=UDu-cnXI8E8&t=2540
[2] Absolutely fascinating. What you hear depends on what you are looking at, not just on what you are hearing. This demonstration starts at approximately 01:02:00 - https://www.youtube.com/watch?v=UDu-cnXI8E8&t=3720
[+] [-] bd|10 years ago|reply
Google does some heavy-duty machine learning / computer vision on their servers to extract 3D information from these video streams (they mentioned having depth data). Then they presumably re-generate seamless stereoscopic 360-degree video from this 3D model. That's how they can get stereo from mono data.
It's something like those hyperlapse videos from Microsoft, they also first do number crunching on lot of video data to generate 3D model and then use this to fill in missing data.
[+] [-] prbuckley|10 years ago|reply
Good for stills and video https://theta360.com/en/about/theta/
Action camera similar to GoPro http://kodakpixpro.com/Americas/cameras/activeCam/sp360.php
http://www.vsnmobil.com/products/v360
[+] [-] joshu|10 years ago|reply
[+] [-] tw04|10 years ago|reply
[+] [-] Cthulhu_|10 years ago|reply
[+] [-] ddingus|10 years ago|reply
I wouldn't underestimate what some people will do for a little spiff.
Maybe they won't do it again, but they might do it once, or on occasion. If one could manage churn reasonably? Who knows?
That said, not on your life. Not doing it.
[+] [-] untog|10 years ago|reply
I've been watching this space for a while and pretty much all of the consumer varieties (usually Kickstarter-backed) have pretty terrible video quality, so it looks like I'll be sitting this one out for a while yet.
[+] [-] wanderingstan|10 years ago|reply
As others have said, the cost of this is peanuts compared to professional cameras.
I'd bet one of the first killer-apps for VR will be sports: Imagine several of these rigs set up around (and over) a football field, so that spectators can choose to "be" at the right point on the field to observe the game, and have complete freedom to look around the field.
[+] [-] gy3b|10 years ago|reply
http://www.engadget.com/2015/04/29/xiaomi-yi-camera-versus-g...
[+] [-] mod|10 years ago|reply
I don't think price will be a big inhibitor here, honestly.
[+] [-] intrasight|10 years ago|reply
[+] [-] julianpye|10 years ago|reply
Youtube already had stereoscopic support, but there were not many clients available, content was limited and difficult to create with 3rd party solutions that all cost money. It never took off and couldn't ride the wave of the studio and CE company driven 3D momentum, which quieted down after a while anyway.
This time Google ticks all the right boxes. Client devices on iOs and Android. The videoplayer everyone uses. A $5 cardboard enabler. A capture platform with GoPro and free assembly software.
[+] [-] pimlottc|10 years ago|reply
[+] [-] jsprogrammer|10 years ago|reply
[+] [-] adityasankar|10 years ago|reply
That being said, this may be the right time to reinvest in 360 degree stereoscopic video since the viewing technology is finally catching up. Point Grey has been making similar spherical cameras since the 90's [1]
Pretty cool idea overall. Other questions I had: Does it have a single data interface? How do you charge this behemoth.
[1] http://www.ptgrey.com/360-degree-spherical-camera-systems
[+] [-] qq66|10 years ago|reply
[+] [-] soperj|10 years ago|reply
[+] [-] cbhl|10 years ago|reply
https://theta360.com/en/
http://www.amazon.com/Ricoh-Theta-Degree-Spherical-Panorama/...
[+] [-] schooldistrict|10 years ago|reply
[+] [-] proee|10 years ago|reply
I'd like to see an array of cameras using quadcopters flying around a target - say a whale jumping out of the water or a football player. This would allow the viewer to see all side of the action. Great for sports playbacks.
[+] [-] usaphp|10 years ago|reply
[+] [-] codeonfire|10 years ago|reply
[+] [-] Buge|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] seanherron|10 years ago|reply
[+] [-] TheGRS|10 years ago|reply
But I think the neat thing is using readily-available GoPro cameras and stitching the video together quickly. That takes a lot of processing power to do on the fly.
[+] [-] etcet|10 years ago|reply
[+] [-] aaroninsf|10 years ago|reply
In theory that could be processed just-in-time to produce an appropriate stereo image based on head tracking.
http://en.wikipedia.org/wiki/Ambisonics
In practice the computational overhead would likely be on par with graphics processing and you might want an APU :)
[+] [-] nathan_f77|10 years ago|reply
[+] [-] gdilla|10 years ago|reply
[+] [-] Yhippa|10 years ago|reply
You could have several of these say at a music festival hanging around and watch them from your house at your leisure without having to deal with heat and crowds. And worse you go see an artist on one stage and realize they're a dud and you have to now travel to another stage. With this, boom, you can switch in an instant.
[+] [-] prbuckley|10 years ago|reply
http://home.otoy.com/otoy-demonstrates-first-ever-light-fiel...
[+] [-] nathan_f77|10 years ago|reply
And one day I would love to travel around the world with a team and film a documentary like Samsara. [2]
[1] https://www.kickstarter.com/projects/1944625487/omni-move-na... [2] http://www.imdb.com/title/tt0770802/
[+] [-] polymathist|10 years ago|reply
If you're at Duke or somewhere nearby I highly recommend checking it out. They have visiting hours fairly frequently. http://virtualreality.duke.edu/
[+] [-] relet|10 years ago|reply
[+] [-] barbs|10 years ago|reply
http://digital--underground.tripod.com/id9.htm
[+] [-] a-dub|10 years ago|reply
I once played with an early Oculus, it basically amounted to something that wasn't really succeeding at tricking my senses other than giving me a headache and making me want to vomit at the same time.
I'm sure similar things were said about early cinema though.
[+] [-] jdeisenberg|10 years ago|reply
[+] [-] bigtones|10 years ago|reply
[+] [-] ipsin|10 years ago|reply
The "Camera Crown" is an entirely impractical idea of course (that is, not the actual idea), but that does lead me to wonder how it's meant to be moved through spaces, if at all.