I think most people believe that the problem with robots is that we don't have the right software, and if we just knew how to program them then today's robots could be incredibly useful in everyday life. From that perspective, this move from OpenAI seems dumb.
That belief is wrong. Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
In the past the software and hardware were equally bad, but today machine learning is advancing like crazy, while the hardware is improving at a snail's pace in comparison. Solving robotics is now a hardware problem, not a software problem. When the hardware is ready, the software will be comparatively easy to develop. Without the right hardware, you can't develop the appropriate software.
OpenAI is right to ignore robotics for now. It's a job for companies with a hardware focus, for at least the next decade.
You're demonstrably wrong. I can waldo any number of commercially available arms to do work humans do today. Surgeons waldo precise robots to conduct surgeries as a matter of course. Every piece of construction machinery operated by a human today is an incredibly useful robot lacking sufficiently capable software.
"When the hardware is ready, the software will be comparatively easy to develop." I take it you've never written any software for a robot? The long tail of the real world takes years and years to handle. Probably the most advanced robotics company, at the cutting edge of the ML+Robotics, is Covariant and their entire business model rests on an understanding that the long tail can and should be handled by humans.
I agree that OpenAI is right to cut out the hardware, but all your reasoning about why is wrong.
The reason, which they state, is that data collection on physical devices is slow and modification to those devices is slow and maintenance on those devices is expensive. You want to simulate everything, not because it reproduces the real world in high fidelity, that doesn't matter, but because it gives you approximations with sufficient variety and complexity that you can continually challenge your AI, and you can do all that at 1M fps.
I don't think it's really true to say that the issue is hardware or software. There is lots of robotics in everyday life, from autonomous vacuums in our homes to autonomous factories producing the goods we consume. The reason we don't have millions of little robots buzzing around us is... there's very little need for it.
The average human spends most of their time barely engaged, our brains and bodies are operating far below what we're capable of, the romanticised sci-fi vision of a world filled with intelligent robots performing every menial task for humans builds on the idea that humans have better things to do, but do we? We already have enough knowledge and resources to end world hunger, to bring a high standard of living to every human, but we choose not to: our problem is social, not software or hardware.
As an aside, I'd dispute the claim that hardware is lagging behind software: Tesla has lots of money and lots of smart people and they haven't been able to deliver self driving cars after more than a decade of promises (because of software).
> Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
You're absolutely wrong. Anyone with basic electronics knowledge and a few hundred bucks can build a passable robot body out of hobby grade servos and 3D printed parts. If you're willing to spend $10k+ you can make something quite capable.
Programming it to then actually do anything, let alone anything useful in the real world, is still out of reach for all but a tiny fraction of companies.
Hardware still has a long way to go before it's as capable as biological systems but it's usable. Real world AI is far from that in most areas.
Robotics software is incredibly complex. Even with machinery that was a perfect replica of a human body to the most minute details, throwing some ML algorithms at it wouldn’t get us anywhere.
If it worked that way, my job would be much easier.
I'm not an expert, but I'd suggest hardware is trailing software, but there is still great progress happening in materials design, soft-robots, miniaturization, etc. Just as in the computer era, we go through phases where the software is ahead of hardware, then hardware gets ahead of software. It seems that is a pendulum that swings, and the argument many people have to the integrated pipeline Apple operates.
I think self driving car is an incredibly useful robots where massive adoption is under way (very early stage still); and in many less challenging areas, self driving capable vechles have been taking over.
robots do a lot today, it's just not AI and most of it isn't ML. I think the ML folks find what robots do today "tiresome and manually trained" but that doesn't stop robots from producing billions of dollars a year in goods.
Also, regular ML researchers sit at tables with laptops. Robotics people need electronics labs and electronics technicians, machine shops and machinists, test tracks and test track staff...
If you have to build stuff, and you're not in a place that builds stuff on a regular basis, it takes way too long to get stuff built.
My prediction is that dropping the real-world interactions will severely slow down their progress in other areas. But then again, I'm super biased because my current work is to make AI training easier by building specialized hardware.
Reinforcement learning can work quite well if you produce the hardware, so that your simulation model perfectly matches the real-world deployment system. On the other hand, training purely on virtual data has never really worked for us because the real world is always messier/dirtier than even your most realistic CGI simulations. And nobody wants an AI that cannot deal with everyday stuff like fog, water, shiny floors, rain, and dust.
In my opinion, most recent AI breakthroughs have come from restating the problem in a way that you can brute-force it with ever-increasing compute power and ever-larger data sets. "end to end trainable" is the magic keyword here. That means the keys to the future are in better data set creation. And the cheapest way to collect lots of data about how the world works is to send a robot and let it play, just like how kids learn.
I dont think this is cynical and I don't think it's a bad thing. OpenAI is not a huge org. The truth in 2021 is that not only is robotics 'just not there yet' in terms of being a useful vehicle for general intelligence research (obviously robotics research itself is still valuable), there is also nothing really pointing at this going to be the case in the next 5-10 years.
Given that, unless they want to commercialise fruit picking or warehouse robots, it seems sensible.
Makes sense I guess, integrating robot hardware requires an entirely different set of skills to ML research and has a much slower dev cycle.
I think OpenAI has progressively narrowed down its core competency - for a company like 3M it would be something like "applying coatings to substrates", and for OpenAI it's more like "applying transformers to different domains".
It seems like most of their high-impact stuff is basically a big transformer: GPT-x, copilot, image gpt, DALL-E, CLIP, jukebox, musenet
their RL and gan/diffusion stuff bucks the trend, but I'm sure we'll see transformers show up in those domains as well.
Is the prevailing opinion that progress in reinforcement learning is dependent on algorithmic advances, as opposed to simply scaling existing algorithms? If that is the case, I could see this decision as an acknowledgement that they are not well positioned to push the frontier of reinforcement learning - at least not compared to any other academic or industry lab. Where they have seen success, and the direction it seems they are consolidating their focus, is in scaling up existing algorithms with larger networks and larger datasets. Generative modeling and self supervised learning seem more amenable to this engineering-first approach, so it seems prudent for them to concentrate their efforts in these areas.
Yes, it feels like we have squeezed most of the performance out of current algorithms and architectures. OpenAI and deepmind have thrown tremendous compute against the problem with little overall progress (overall, alpha go is special). There was a big improvement in performance by bringing in function approximators in the form of deep networks. Which as you said can scale upwards nicely with more data and compute. In my opinion as an academic in the deep RL, it feels like we are missing some fundamental pieces to get another leap forward. I am uncertain what exactly the solution is but any improvement in areas like sample efficiency, stability, or task transfer could be quite significant. Personally I’m quite excited about the vein of learning to learn.
I think the premise of your question actually points to the real problem. In RL, b/c your current policy and actions determine what data you see next, you can't really just "scale existing algorithms" in the sense of shoving more of the same data through them on more powerful processors. There's a sequential process of acting/observing/learning which is bottlenecked on your ability to act in your environment (ie through your robot). Off-policy learning exists, but scaling up the amount of data you process from a bad initial policy doesn't really lead anywhere good.
This is lunacy. The first country/company to replace human labour with general bipedal robots, will reap wealth beyond imagination. The short sitedness is astonishing, if you ask me.
I genuinely believe how we as a society act once human labour is replaced is first aspect of the great filter.
We are decades away from being able to build a general bipedal robot that can snake out a plugged toilet or dig a trench or nail shingles to a roof. It's just not a rational goal yet. Aim lower.
There are no mechanisms in place for the generated wealth to benefit the replaced people, the wealth will go mainly to vanishingly few persons self selected to be okay with gross economic inequality.
We have been at this since at least the dawn of the industrial revolution and do not have it right yet.
Backing off and taking it slow now to let some cultural adjustments happen is a responsible step.
My cultural norms are repulsed by the thought of me not working as much as possible, it is how I expect my value to society to be gauged (and rewarded).
This line of reasoning will be (is) obsolete and we need another in its place globally.
I hope some may have better ideas of what these new cultural norms should look like than I with my too much traditional indoctrination.
I only know what I will not have it look like;
humanity as vassals of non corporeal entities or elites.
I totally agree. I worked at a robotics company about a decade ago, and I was familiar with the people at Willow garage.
Robotics research is going to be extremely binary. It's expensive and frustrating, and there's little use for it until it works as well as human labor, which is a high bar.
But, once that Rubicon is crossed, I believe there will be a sort of singularity in that space. It's related to but somewhat orthogonal to the singularity that's prognosticated for g a i.
> replace human labour with general bipedal robots
No need for bipeds, car factories employ dumb robot arms, no humans needed. Not very general purpose robots though.
The first country/company to create robots that can be instructed similar to a humans to do any job will indeed have great benefits, but how long until that happens? Not within any amount of time that an investor wants to see. I'm unsure if I will ever see that in my life (counting on ~60 years to go still maybe?)
Imagine that there only needs to be ten people to “run the world”. What is the population size going to be then? Ten? As large as possible? Somehow it seems that the way we’re headed, it’ll be ten plus some administrative overhead.
You might be right long term - but think about it short term. Labor in low cost countries is very cheap. A few thousand a year. Unless you can build these machines for low 10's of thousands and maintain them for 100's per year, the economics won't work. Construction robots might be a good counter example because you can't offshore them.
If robots are doing all the work how will people make money to buy the stuff the robots make? Is Jeff Bezos going to own the whole world or are we going to have another French revolution?
I think the comments are confounding shutting down the robotics research team with eliminating all RL research. Most robotics teams don't use data-hungry RL algorithms because the cost of interacting with the environment is so expensive. And even if the team has a simulator that can approximate the real world to produce infinite data, there is still the issue of the "simulator-gap" with the real world.
I don't work for openAI but I would guess they are going to keep working on RL (e.g hide and seek, gym, DoTA style Research) to push the algorithmic SoTA. But translating that into a physical robot interacting with the physical world is extremely difficult and a ways away.
With the mentioning that they can shift their focus to domains with extensive data that they can build models of action with etc... Why not try the following (If easily possible)
---
Take all the objects on the various 3D warehouses (thingiverse, and all the other 3d modeling repos out there) -- and have a system whereby an OpenAI 'Robotics' platform can virtually learn to manipulate and control a 3D model (solidworks/blender/whatever) and learn how to operate it.
It would be amazing to have an AI robotics platform where you feed it various 3D files of real/planned/designed machines, and have it understand the actual constituancy of the components involved, then learn its degrees of motion limits, or servo inputs etc... and then learn to drive the device.
Then, give it various other machines which share component types, built into any multitude of devices - and have it eval the model for familiar gears, worm-screws, servos, motors, etc... and have it figure out how to output the controller code to run an actual physically built out device.
Let it go through thousands of 3D models of things and build a library of common code that can be used to run those components when found in any design....
Then you couple that code with Copilot and allow for people to have a codebase for controlling such based on what OpenAI has already learned....
As Copilot is already built using a partnership with OpenAI...
I suspect it's because at a certain point detailed physics matters and simulating things well enough is really hard. A robotic arm might flex just a bit, a gear may not mesh quite correctly, signals may take just a bit longer to get somewhere, a grip might slip, a plastic object might break from too much force, etc, etc.
I'm sure the overhead and upkeep of a robotics lab far outweighs that of a computer lab for software research.
Are there any Open* organizations for robotics that could perhaps fill the void here? I think robotics is really important and I think the software is a big deal also, but it's important that actual physical trials of these AIs are pursued. I would think that seeing something in real space like that offers an unparalleled insight for expert observers.
I remember the first time I ever orchestrated a DB failover routine, my boss took me into the server room when it was scheduled on the testing cluster. Hearing all the machines spin up and the hard drives start humming, that was a powerful and visceral moment for me and really crystallized what seemed like importance about my job.
Designing robots to pick fruit and make coffee / pizzas cannot have a positive ROI until labor laws make the bsuiness-case for them. Majority of use cases where we can use robots for activities humans cannot perform (fast spot welding on production line, moving nuclear fuel rods, etc) have been solved. It is smart to focus on language and information processing, given that we are producing so much more of it, everyday.
modeless|4 years ago
That belief is wrong. Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
In the past the software and hardware were equally bad, but today machine learning is advancing like crazy, while the hardware is improving at a snail's pace in comparison. Solving robotics is now a hardware problem, not a software problem. When the hardware is ready, the software will be comparatively easy to develop. Without the right hardware, you can't develop the appropriate software.
OpenAI is right to ignore robotics for now. It's a job for companies with a hardware focus, for at least the next decade.
iandanforth|4 years ago
"When the hardware is ready, the software will be comparatively easy to develop." I take it you've never written any software for a robot? The long tail of the real world takes years and years to handle. Probably the most advanced robotics company, at the cutting edge of the ML+Robotics, is Covariant and their entire business model rests on an understanding that the long tail can and should be handled by humans.
I agree that OpenAI is right to cut out the hardware, but all your reasoning about why is wrong.
The reason, which they state, is that data collection on physical devices is slow and modification to those devices is slow and maintenance on those devices is expensive. You want to simulate everything, not because it reproduces the real world in high fidelity, that doesn't matter, but because it gives you approximations with sufficient variety and complexity that you can continually challenge your AI, and you can do all that at 1M fps.
roombad|4 years ago
The average human spends most of their time barely engaged, our brains and bodies are operating far below what we're capable of, the romanticised sci-fi vision of a world filled with intelligent robots performing every menial task for humans builds on the idea that humans have better things to do, but do we? We already have enough knowledge and resources to end world hunger, to bring a high standard of living to every human, but we choose not to: our problem is social, not software or hardware.
As an aside, I'd dispute the claim that hardware is lagging behind software: Tesla has lots of money and lots of smart people and they haven't been able to deliver self driving cars after more than a decade of promises (because of software).
taneq|4 years ago
You're absolutely wrong. Anyone with basic electronics knowledge and a few hundred bucks can build a passable robot body out of hobby grade servos and 3D printed parts. If you're willing to spend $10k+ you can make something quite capable.
Programming it to then actually do anything, let alone anything useful in the real world, is still out of reach for all but a tiny fraction of companies.
Hardware still has a long way to go before it's as capable as biological systems but it's usable. Real world AI is far from that in most areas.
trainsplanes|4 years ago
If it worked that way, my job would be much easier.
pedalpete|4 years ago
justicezyx|4 years ago
TheRealPomax|4 years ago
DantesKite|4 years ago
dekhn|4 years ago
minimaxir|4 years ago
Animats|4 years ago
Also, regular ML researchers sit at tables with laptops. Robotics people need electronics labs and electronics technicians, machine shops and machinists, test tracks and test track staff...
If you have to build stuff, and you're not in a place that builds stuff on a regular basis, it takes way too long to get stuff built.
fxtentacle|4 years ago
Reinforcement learning can work quite well if you produce the hardware, so that your simulation model perfectly matches the real-world deployment system. On the other hand, training purely on virtual data has never really worked for us because the real world is always messier/dirtier than even your most realistic CGI simulations. And nobody wants an AI that cannot deal with everyday stuff like fog, water, shiny floors, rain, and dust.
In my opinion, most recent AI breakthroughs have come from restating the problem in a way that you can brute-force it with ever-increasing compute power and ever-larger data sets. "end to end trainable" is the magic keyword here. That means the keys to the future are in better data set creation. And the cheapest way to collect lots of data about how the world works is to send a robot and let it play, just like how kids learn.
high_derivative|4 years ago
Given that, unless they want to commercialise fruit picking or warehouse robots, it seems sensible.
taneq|4 years ago
mountainriver|4 years ago
madisonmay|4 years ago
amelius|4 years ago
zitterbewegung|4 years ago
Jack000|4 years ago
I think OpenAI has progressively narrowed down its core competency - for a company like 3M it would be something like "applying coatings to substrates", and for OpenAI it's more like "applying transformers to different domains".
It seems like most of their high-impact stuff is basically a big transformer: GPT-x, copilot, image gpt, DALL-E, CLIP, jukebox, musenet
their RL and gan/diffusion stuff bucks the trend, but I'm sure we'll see transformers show up in those domains as well.
varelse|4 years ago
https://arxiv.org/abs/2102.02202
Not to mention a bunch of relatively inexpensive reinforcement learning research relying on consumer knockoffs of Spot from Boston Dynamics...
Really does seem like they are following the money and while there's nothing wrong with that it's also nothing like their original mission.
ansk|4 years ago
nrmn|4 years ago
abeppu|4 years ago
andyxor|4 years ago
fartcannon|4 years ago
I genuinely believe how we as a society act once human labour is replaced is first aspect of the great filter.
nradov|4 years ago
tejtm|4 years ago
We have been at this since at least the dawn of the industrial revolution and do not have it right yet. Backing off and taking it slow now to let some cultural adjustments happen is a responsible step.
My cultural norms are repulsed by the thought of me not working as much as possible, it is how I expect my value to society to be gauged (and rewarded).
This line of reasoning will be (is) obsolete and we need another in its place globally.
I hope some may have better ideas of what these new cultural norms should look like than I with my too much traditional indoctrination.
I only know what I will not have it look like; humanity as vassals of non corporeal entities or elites.
Zababa|4 years ago
Humans ARE genral bipedal robots. The price of these robots is determined by the minimum wage.
danbmil99|4 years ago
Robotics research is going to be extremely binary. It's expensive and frustrating, and there's little use for it until it works as well as human labor, which is a high bar.
But, once that Rubicon is crossed, I believe there will be a sort of singularity in that space. It's related to but somewhat orthogonal to the singularity that's prognosticated for g a i.
ragebol|4 years ago
No need for bipeds, car factories employ dumb robot arms, no humans needed. Not very general purpose robots though.
The first country/company to create robots that can be instructed similar to a humans to do any job will indeed have great benefits, but how long until that happens? Not within any amount of time that an investor wants to see. I'm unsure if I will ever see that in my life (counting on ~60 years to go still maybe?)
toxik|4 years ago
danielmarkbruce|4 years ago
throwaway_45|4 years ago
xnx|4 years ago
cweill|4 years ago
I don't work for openAI but I would guess they are going to keep working on RL (e.g hide and seek, gym, DoTA style Research) to push the algorithmic SoTA. But translating that into a physical robot interacting with the physical world is extremely difficult and a ways away.
samstave|4 years ago
With the mentioning that they can shift their focus to domains with extensive data that they can build models of action with etc... Why not try the following (If easily possible)
---
Take all the objects on the various 3D warehouses (thingiverse, and all the other 3d modeling repos out there) -- and have a system whereby an OpenAI 'Robotics' platform can virtually learn to manipulate and control a 3D model (solidworks/blender/whatever) and learn how to operate it.
It would be amazing to have an AI robotics platform where you feed it various 3D files of real/planned/designed machines, and have it understand the actual constituancy of the components involved, then learn its degrees of motion limits, or servo inputs etc... and then learn to drive the device.
Then, give it various other machines which share component types, built into any multitude of devices - and have it eval the model for familiar gears, worm-screws, servos, motors, etc... and have it figure out how to output the controller code to run an actual physically built out device.
Let it go through thousands of 3D models of things and build a library of common code that can be used to run those components when found in any design....
Then you couple that code with Copilot and allow for people to have a codebase for controlling such based on what OpenAI has already learned....
As Copilot is already built using a partnership with OpenAI...
marcinzm|4 years ago
verall|4 years ago
adenozine|4 years ago
Are there any Open* organizations for robotics that could perhaps fill the void here? I think robotics is really important and I think the software is a big deal also, but it's important that actual physical trials of these AIs are pursued. I would think that seeing something in real space like that offers an unparalleled insight for expert observers.
I remember the first time I ever orchestrated a DB failover routine, my boss took me into the server room when it was scheduled on the testing cluster. Hearing all the machines spin up and the hard drives start humming, that was a powerful and visceral moment for me and really crystallized what seemed like importance about my job.
spiritplumber|4 years ago
xyproto|4 years ago
machiaweliczny|4 years ago
dougSF70|4 years ago
coolspot|4 years ago
unknown|4 years ago
[deleted]
wly_cdgr|4 years ago
amerine|4 years ago