The interesting part is that a lot of what he is talking about is already possible.
- You can use augmented reality or VR with something like Oculus to display things in 3D space.
- You can use Motion Capture technologies, plus maybe some convenient remote-control/keyboard-like devices to put a human in that virtual space and manipulate things.
Only 2 technologies(Oculus + Mocap) would be enough to accomplish something like this:
http://youtu.be/VzFpg271sm8
Give a bunch of people oculus rifts and put them in the same virtual reality - and we have the dynamic stage he is talking about.
They don't even have to be in the same room of course, we have 3D realtime MMORPGs and second-life that already solved that problem!! You could attend a virtual talk by sitting in your apartment with oculus rift on your head.
And of course "downloading a room" instead of a book is already very doable. We already have computer games and engines. That room is like a video game level. Downloading it is not a problem, to explore it all you need is oculus and a simple game controller or a keyboard/mouse.
And in terms of creating such content, the closest thing I can think of is 3D editors, like Maya or Houdini, especially houdini(http://www.sidefx.com), which is the brilliant combination of computer graphics and programming, that allows you to model and program dynamic things visually.
I've also been thinking about the concrete representation of symbols and the negative "inhumane" impact of screens, but my conclusion was much different than Victor's: screens take us away from the fullest experience of world, and so their use should be minimized. This is basically Victor's conclusion minus the possibility of what I might call "somatic programs" at scale.
That screens are problematic asserts a moral imperative for programmers to avoid writing applications that are designed to be experiential cul-de-sacs. This, in turn, goes very much against the grain of most technologists and the people who fund them, who are all looking to make the most potent screens they can to pull people out of real life as much as possible. This makes great economic sense, but it is unethical.
Most of us who are so fundamentally tied to a screen for so many years at a time would do well to systematically disengage and interact with the real-world as much as possible. What I wouldn't give at this point in my life to put on a hard-hat and help build a bridge!
I'd push back on the dog in a cage analogy.
I think it could be argued that we've narrowed our communication mediums for purpose of effectiveness and efficiency. Also affordable ubiquity.
I would argue it's more like limiting a fish to water rather than a dog to a cage.
Not to say it can't be improved in light of today's tools though.
Great talk overall, a lot of interesting insights!
The analogy was an example of something that is inhumane for analogous reasons that restricted media is inhumane. It doesn't allow for full dog-life or full intellectual-life, some critical and important experiences and behaviors are excluded.
Changing the analogy to a fish in a tank suggests an assertion that these other intellectual modes are not important for human experience and wouldn't make communication more effective and efficient.
To tweak the analogy further: Otters are very good at swimming, but their habitats should include land.
I'm actually quite interested in this spatial learning thing. I had an aboriginal acquaintance who explained to me how his culture would attach meaning to places and even physical orientation (North South East West etc). However, it has been so far very very difficult for me to wrap my head around that. Visual notation imo, has been the most efficient way of communicating between humans. It also helps a lot in understanding.
A lot of what he says harks back to a 1980s/1990s/early 2000s style of teaching with an increasing use of "aids". Using props to teach math is a bit silly (well, asides from geometry, which is inherently spatial). I really cannot see how we can represent the learning of linear algebra in a dynamic spatial manner.
How for example, would one represent higher dimensional spaces spatially, when you really are limited to only 3 dimensions in real life? I am aware that Iron Man style projection tech may actually help in visualizing it, but again, that's a visual aspect of learning, not spatial. The most efficient way of doing so unfortunately is still in algebraic notation.
Perhaps I'm doubtful because the use of props, spaces, songs, moving your body etc has never worked for me in school. Ever. Can anyone share any studies done on the use of these cheesy 1980s-1990s educational aids (the closest analogue to all the things Brett mentioned) in aiding learning?
EDIT: If you're interested, there's been a good book that I read, in attempt to wrap my head around the concept (however to no avail) - Wisdom Sits in Places.
I had a Spanish teacher and later a Latin teacher (teaching adults) who favored TPR and would tell people to jump a specified number of times, or to touch their heads, and so on. I don't have a clear opinion about the effectiveness of TPR but I thought I'd mention the name so you could use it to search for research if you like.
There is also a century-old approach to using colors to visualize 4D geometry (I think the color represents the offset in the 4th spatial dimension), proposed by Charles H. Hinton; they're called "Hinton cubes". Martin Gardner describes them and reports that some people have found them too effective, I guess becoming mildly obsessed with them or having the learned associations interfere somehow with day-to-day visual perception.
When I learned/taught myself algebra in an experimental school in 6th grade, we had these alegrba blocks and 2x2 grid, the left/right sides representing the sides of the equation and the top/bottom sides representing postitive/negative.
There are some pretty big limitations on how far in algebra you can go with that. I do think it created a solid foundation for my mental model as I transitioned to only using symbolic equations.
i think the word learning is a misnomer here, the term learning in itself is dubious enough
look to the best line, and i'd argue one of the most important ideas to ingestively portray to a mainstream population, in 2008 cinema, also an argument could be made to increase that reach.. perhaps in the last decade, century, however hyperbolically you care to extend the reach.. from the film burn after reading(i):
CIA Superior: What did we learn, Palmer?
CIA Officer: I don't know, sir.
CIA Superior: I don't fuckin' know either. I guess we learned not to do it again.
CIA Officer: Yes, sir.
CIA Superior: I'm fucked if I know what we did.
i think what victor is suggesting here would aide moreso in efforts to improve understanding than facilitate a traditional understanding of learning
when quantum mechanics was young there was a sudden rush of interest in quantum understanding and one of the modes of expressing improvements in that understanding was to create models(ii) sometimes completely replacing old models, if i remember victor even says bohrs name in relation to this concept
this i think is the sorts of things victor is alluding to, stead seeing it as learning think of it as new perspectives to improve understanding.. interestingly though one would have to learn the concepts, both current and avant, in order to know when a new expressive medium improves on the old
so knowledge accumulation is a part of this sort of environment, but arguable only subordinate to the true goals
the shot of the stick figure pushing on a bond between elements in a molecular model i feel is the best representation of this idea
i'd love to have a room like that:
a central tool with supporting data surrounding it
the central tool is imbued with evidencial data w
when i manipulate the tool new data is expressed in different ways around me
i apply my changes physically
the the data maps all changing in sync
lets say i want to try to bond a new element in place of an old one, i can do it physically and watch as the object changes before me
in the picture the stick figure is applying force on that bond, perhaps the idea the figure is testing is what if i bend this bond or strain it, what then? or if i move this bond over here does this better represent our understanding of molecules?
.
my issue with the talk was two fold, i thought victor tended to talk down efforts others have already made or are making to do something with similar goals but in a wholly different means, and victor also failed to mention the number of people already doing the very things he is suggesting,
it came off as unnecessarily dramatic and i found those were the times when he really lost me to wandering thoughts
and also i found it interesting that victor seemed to suggest this new dynamic reality would manifest without the use of current forms of creation
things like, ~programming of the future will be different than programming of today~ which i both agree and disagree with, i think both the current and future ways to program will be in tandem and it will be more of a matter of who touches what aspects of it,
i think like hardware design and manufacture programming will become a specialised field
look at compilers, that is people really interested in low level bit moving granting people with high level interests.. in the programming sense stead some cogitive nobile sense.. access to their expertise from within the comforts of the higher level
the same i believe will be true but that gap will likely grow as it has with hardware manufacture
there will always be a need and opportunity for tinkers to challenge the manufacture to keep it fresh, like the hardware scene(iii), but realistically this shift in software will most likely have the same dramatic effect on quality of life as the hardware model has
1. Walking around a gallery and waving your arms is not enough exercise
2. Incremental progress is giving us VR through Tactical Haptics, Oculus Rift, etc. He says his thing is not VR but R or dynamic R. The distinction is unclear, sounds like marketing.
I also had the idea of talking about systems, particularly political systems through models and simulations. Having spent too much time talking about politics, economics, and computer holy wars in text, it feels like a few interactive models could have saved all that time for millions of people.
People start thinking in spreadsheets. A lot of the weird merger acquisitions since the 1980's were basically caused by them. Economists already use fancier models. And as with spreadsheets, garbage in, garbage out. Still I feel they'd be useful in casual conversation.
That last part catches my interest. Could someone do to the agent-based model what the spreadsheet did to arithmetic, i. e. make it low-effort and scalable?
Great talk and ideas. But the idea of representing mathematical equations in the physical form is like supporting the same medium of symbols and just representing it in a different way. But fundamentally math was invented using pen and paper, so the centuries of mathematical thoughts were already slaves to the 2D medium. Maybe it would be interesting to think about, what would happen if we take away 2D surfaces and ask people to reinvent math in a medium that Bret is talking about. How would someone go about discovering, solving, proving mathematical systems without pen and paper? How would you represent 4, 5 or 20 multi-dimentional spaces in a 3D environment ? Maybe pen&paper was kind of abstracting out limitations of physical world so that mind can explore even further.
The progress in n-category theory is largely about that. Instead of just studying objects, operations and equations, like "a group is a set g, together with multiplication + certain axioms" one studies objects, relations between objects, relations between relations etc. To study this up to level two, you visualize objects as regions in 2d space, relations between objects as lines between them and relations between relations as points where lines meet. Equations are then equalities between such pictures. If you look at those equalities as a dynamic process, which starts at time t=0 with one picture and time t=1 with the other, you get a picture in three dimensional space, which you can visualize as a movie (to be mathematically precise after the choice of a morse function you get a movie). One of the easiest examples would be the zig-zag identity. See for example
https://golem.ph.utexas.edu/category/2006/10/classical_vs_qu...
and scroll down.
One person who has completely gone overboard with that is Jacob Lurie, he has practically single handedly written down a foundational work of higher algebra and algebraic topology, which essentially precisely corresponds to what you are asking. How does mathematics look like if we weren't constrained to 2d paper to write down equations. Of course to actually write down those things you have to somehow project the equations to several essentially 1d sections or invoke very abstract methods to reason about them. Of course algebraic topology already has tools for that.
The early impetus came from Grothendieck I believe, who began to create a theory of "higher groupoids" in several thousand pages, ordinary groupoids being categories in which every morphism is an isomorphism.
On TV in the Netherlands we have a program that aims to educate the public (state sponsored tv), every couple of months there is a new episode, they are listed here (but they are in Dutch): http://dewerelddraaitdoor.vara.nl/DWDD-University.2772.0.htm...
So far they were about the big bang, the quantum world, Einstein (All from Robbert Dijkgraaf, now director of the Institute for Advanced Study in Princeton), The cow and silicon valley. They were always set in a large theater (with live public) with a stage full of props (Inflating balloons to represent the inflating universe, buckets of sand to represent the amount of stars in a galaxy, cameras on rails to represent relativity, living cows as well as all their nice parts of meat on display.) These "presentations" are very nice to watch and my wife who is not a scientist always enjoys them as well.
There are also museums like Nemo in Amsterdam (https://www.e-nemo.nl/nl/ontdek/brein/) and Naturalis in Leiden (http://www.naturalis.nl/en/), these places are filled with rooms with stuff to touch and experiments to perform. I'm sure the US is laced with those as well so it is not that ground-breaking of an idea.
A holo-deck would be the killer feature for this way of communicating.
On the topic of how representations relate to programming... if you have ever tried to express some tricky math in code you might have had a sense that it would be more clear if it were written out on paper with the proper notation. That's because mathematicians have streamlined their reasoning by deciding one day, well, I use the sum function a lot, let's call it Σ. Or I'd like to be able to recognize integrals from a mile away so let's call it ∫. Not to mention those symbols come along with a spatial organization: the lower limit of integration goes below, the upper limit goes on top.
Programmers can't easily streamline like this. Maybe there are multiple forces locking us in: the keyboard, text editor, and programming language. Sure, maybe the optimal way to write code is with a string of ascii characters arranged left to right top to bottom but I doubt it. So even in the visual-symbolic space there is unexplored potential.
I'm glad someone is out there tackling these assumptions and much more, I should keep an eye on his work.
Looks interesting, going to watch rest later. I wonder if the graph of trade with England is really the 'basis' for all scientific graphs of data. Didn't Descartes develop the idea of graphing Y in relation to X?
great talk! one touchstone i wish he would have embraced is woodworking. having listened to the talk, i can see how woodworking is experienced through many channels: tactile (hand planes, saws), visual, spatial, symbolic (plans, designs), auditory (sonic feedback tells you a lot of about the wood, and how well a tool is working), environmental (the workshop)... i feel like adopting his channels perspective helps me understand why that activity is so engaging. certainly gives credence to his claims, too.
I think something like ipython can get to where he wants to go. I have already used the RISE profile and MDTraj so I can show people my notebook in slide layout, and pull protein models directly from PDB (Brad talks about the provenance of the data). So I can run code live, manipulate meaningful models live. I think some solid navigation wouldn't be too far off. Now we just need holograms. Cheap blue lasers, people. We need cheap blue lasers.
VR still lacks good tactile, kinesthetic representations. Not to say it's not a step in the right direction, but it is arguably taking a deeper step into visual-only representation.
[+] [-] rayalez|11 years ago|reply
- You can use augmented reality or VR with something like Oculus to display things in 3D space.
- You can use Motion Capture technologies, plus maybe some convenient remote-control/keyboard-like devices to put a human in that virtual space and manipulate things.
Only 2 technologies(Oculus + Mocap) would be enough to accomplish something like this: http://youtu.be/VzFpg271sm8
Give a bunch of people oculus rifts and put them in the same virtual reality - and we have the dynamic stage he is talking about.
They don't even have to be in the same room of course, we have 3D realtime MMORPGs and second-life that already solved that problem!! You could attend a virtual talk by sitting in your apartment with oculus rift on your head.
And of course "downloading a room" instead of a book is already very doable. We already have computer games and engines. That room is like a video game level. Downloading it is not a problem, to explore it all you need is oculus and a simple game controller or a keyboard/mouse.
And in terms of creating such content, the closest thing I can think of is 3D editors, like Maya or Houdini, especially houdini(http://www.sidefx.com), which is the brilliant combination of computer graphics and programming, that allows you to model and program dynamic things visually.
[+] [-] javajosh|11 years ago|reply
That screens are problematic asserts a moral imperative for programmers to avoid writing applications that are designed to be experiential cul-de-sacs. This, in turn, goes very much against the grain of most technologists and the people who fund them, who are all looking to make the most potent screens they can to pull people out of real life as much as possible. This makes great economic sense, but it is unethical.
Most of us who are so fundamentally tied to a screen for so many years at a time would do well to systematically disengage and interact with the real-world as much as possible. What I wouldn't give at this point in my life to put on a hard-hat and help build a bridge!
[+] [-] cusack|11 years ago|reply
Great talk overall, a lot of interesting insights!
[+] [-] indrax|11 years ago|reply
Changing the analogy to a fish in a tank suggests an assertion that these other intellectual modes are not important for human experience and wouldn't make communication more effective and efficient.
To tweak the analogy further: Otters are very good at swimming, but their habitats should include land.
[+] [-] chewxy|11 years ago|reply
I'm actually quite interested in this spatial learning thing. I had an aboriginal acquaintance who explained to me how his culture would attach meaning to places and even physical orientation (North South East West etc). However, it has been so far very very difficult for me to wrap my head around that. Visual notation imo, has been the most efficient way of communicating between humans. It also helps a lot in understanding.
A lot of what he says harks back to a 1980s/1990s/early 2000s style of teaching with an increasing use of "aids". Using props to teach math is a bit silly (well, asides from geometry, which is inherently spatial). I really cannot see how we can represent the learning of linear algebra in a dynamic spatial manner.
How for example, would one represent higher dimensional spaces spatially, when you really are limited to only 3 dimensions in real life? I am aware that Iron Man style projection tech may actually help in visualizing it, but again, that's a visual aspect of learning, not spatial. The most efficient way of doing so unfortunately is still in algebraic notation.
Perhaps I'm doubtful because the use of props, spaces, songs, moving your body etc has never worked for me in school. Ever. Can anyone share any studies done on the use of these cheesy 1980s-1990s educational aids (the closest analogue to all the things Brett mentioned) in aiding learning?
EDIT: If you're interested, there's been a good book that I read, in attempt to wrap my head around the concept (however to no avail) - Wisdom Sits in Places.
[+] [-] schoen|11 years ago|reply
https://en.wikipedia.org/wiki/Total_physical_response
I had a Spanish teacher and later a Latin teacher (teaching adults) who favored TPR and would tell people to jump a specified number of times, or to touch their heads, and so on. I don't have a clear opinion about the effectiveness of TPR but I thought I'd mention the name so you could use it to search for research if you like.
There is also a century-old approach to using colors to visualize 4D geometry (I think the color represents the offset in the 4th spatial dimension), proposed by Charles H. Hinton; they're called "Hinton cubes". Martin Gardner describes them and reports that some people have found them too effective, I guess becoming mildly obsessed with them or having the learned associations interfere somehow with day-to-day visual perception.
[+] [-] shkkmo|11 years ago|reply
There are some pretty big limitations on how far in algebra you can go with that. I do think it created a solid foundation for my mental model as I transitioned to only using symbolic equations.
[+] [-] justifier|11 years ago|reply
look to the best line, and i'd argue one of the most important ideas to ingestively portray to a mainstream population, in 2008 cinema, also an argument could be made to increase that reach.. perhaps in the last decade, century, however hyperbolically you care to extend the reach.. from the film burn after reading(i):
i think what victor is suggesting here would aide moreso in efforts to improve understanding than facilitate a traditional understanding of learningwhen quantum mechanics was young there was a sudden rush of interest in quantum understanding and one of the modes of expressing improvements in that understanding was to create models(ii) sometimes completely replacing old models, if i remember victor even says bohrs name in relation to this concept
this i think is the sorts of things victor is alluding to, stead seeing it as learning think of it as new perspectives to improve understanding.. interestingly though one would have to learn the concepts, both current and avant, in order to know when a new expressive medium improves on the old
so knowledge accumulation is a part of this sort of environment, but arguable only subordinate to the true goals
the shot of the stick figure pushing on a bond between elements in a molecular model i feel is the best representation of this idea
i'd love to have a room like that:
lets say i want to try to bond a new element in place of an old one, i can do it physically and watch as the object changes before mein the picture the stick figure is applying force on that bond, perhaps the idea the figure is testing is what if i bend this bond or strain it, what then? or if i move this bond over here does this better represent our understanding of molecules? .
my issue with the talk was two fold, i thought victor tended to talk down efforts others have already made or are making to do something with similar goals but in a wholly different means, and victor also failed to mention the number of people already doing the very things he is suggesting, it came off as unnecessarily dramatic and i found those were the times when he really lost me to wandering thoughts
and also i found it interesting that victor seemed to suggest this new dynamic reality would manifest without the use of current forms of creation
things like, ~programming of the future will be different than programming of today~ which i both agree and disagree with, i think both the current and future ways to program will be in tandem and it will be more of a matter of who touches what aspects of it,
i think like hardware design and manufacture programming will become a specialised field
look at compilers, that is people really interested in low level bit moving granting people with high level interests.. in the programming sense stead some cogitive nobile sense.. access to their expertise from within the comforts of the higher level
the same i believe will be true but that gap will likely grow as it has with hardware manufacture
there will always be a need and opportunity for tinkers to challenge the manufacture to keep it fresh, like the hardware scene(iii), but realistically this shift in software will most likely have the same dramatic effect on quality of life as the hardware model has
.
(i) http://www.imdb.com/title/tt0887883/?ref_=ttqt_qt_tt
(ii) https://www.timetoast.com/timelines/atomic-model-timeline--3
(iii) https://www.youtube.com/watch?v=PdcKwOo7dmM
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] Detrus|11 years ago|reply
1. Walking around a gallery and waving your arms is not enough exercise 2. Incremental progress is giving us VR through Tactical Haptics, Oculus Rift, etc. He says his thing is not VR but R or dynamic R. The distinction is unclear, sounds like marketing.
I also had the idea of talking about systems, particularly political systems through models and simulations. Having spent too much time talking about politics, economics, and computer holy wars in text, it feels like a few interactive models could have saved all that time for millions of people.
That said, simulations are widely used to communicate about financial topics, mainly spreadsheets. And the effects of using spreadsheets are pretty interesting https://medium.com/backchannel/a-spreadsheet-way-of-knowledg...
People start thinking in spreadsheets. A lot of the weird merger acquisitions since the 1980's were basically caused by them. Economists already use fancier models. And as with spreadsheets, garbage in, garbage out. Still I feel they'd be useful in casual conversation.
[+] [-] jamesrom|11 years ago|reply
2. He's talking beyond VR. He never says VR isn't a stepping stone. But he does make a good point about it being dehumanising.
[+] [-] dropit_sphere|11 years ago|reply
[+] [-] 31reasons|11 years ago|reply
[+] [-] orbifold|11 years ago|reply
One person who has completely gone overboard with that is Jacob Lurie, he has practically single handedly written down a foundational work of higher algebra and algebraic topology, which essentially precisely corresponds to what you are asking. How does mathematics look like if we weren't constrained to 2d paper to write down equations. Of course to actually write down those things you have to somehow project the equations to several essentially 1d sections or invoke very abstract methods to reason about them. Of course algebraic topology already has tools for that.
The early impetus came from Grothendieck I believe, who began to create a theory of "higher groupoids" in several thousand pages, ordinary groupoids being categories in which every morphism is an isomorphism.
[+] [-] teekert|11 years ago|reply
On TV in the Netherlands we have a program that aims to educate the public (state sponsored tv), every couple of months there is a new episode, they are listed here (but they are in Dutch): http://dewerelddraaitdoor.vara.nl/DWDD-University.2772.0.htm...
So far they were about the big bang, the quantum world, Einstein (All from Robbert Dijkgraaf, now director of the Institute for Advanced Study in Princeton), The cow and silicon valley. They were always set in a large theater (with live public) with a stage full of props (Inflating balloons to represent the inflating universe, buckets of sand to represent the amount of stars in a galaxy, cameras on rails to represent relativity, living cows as well as all their nice parts of meat on display.) These "presentations" are very nice to watch and my wife who is not a scientist always enjoys them as well.
There are also museums like Nemo in Amsterdam (https://www.e-nemo.nl/nl/ontdek/brein/) and Naturalis in Leiden (http://www.naturalis.nl/en/), these places are filled with rooms with stuff to touch and experiments to perform. I'm sure the US is laced with those as well so it is not that ground-breaking of an idea.
A holo-deck would be the killer feature for this way of communicating.
[+] [-] JeroenRansijn|11 years ago|reply
[+] [-] tmerr|11 years ago|reply
Programmers can't easily streamline like this. Maybe there are multiple forces locking us in: the keyboard, text editor, and programming language. Sure, maybe the optimal way to write code is with a string of ascii characters arranged left to right top to bottom but I doubt it. So even in the visual-symbolic space there is unexplored potential.
I'm glad someone is out there tackling these assumptions and much more, I should keep an eye on his work.
On an unrelated note, his website is entertaining: http://worrydream.com/
[+] [-] joshkpeterson|11 years ago|reply
[+] [-] arikrak|11 years ago|reply
http://en.wikipedia.org/wiki/Cartesian_coordinate_system
[+] [-] the_cat_kittles|11 years ago|reply
[+] [-] niels_olson|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] GraffitiTim|11 years ago|reply
Edit: he later dismisses this, but I don't think he's right to do so.
[+] [-] bchjam|11 years ago|reply
(edit: relevant link, Victor's rant on hands in interaction design. http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...)
[+] [-] mottey|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] pokpokpok|11 years ago|reply