Today we’re excited to be making our first contribution back to the tech community by open sourcing Orchestra, a system for coordinating project teams of experts and machines. It can be used to reimagine and implement all sorts of workflows, from professional services (like legal, design, and data analysis) to new product experiences (like M from Facebook). Orchestra is based upon years of research into machine-mediated expert teams. We hope you’ll come up with new and creative ways of using it!
At Unlimited Labs, we’re excited about building a brighter future of work and are currently building our first product on top of Orchestra. We’d love to answer any questions you have on here, or feel free to email the team at [email protected]!
This looks absolutely wonderful. A few random thoughts:
1. I'm going to play with this as a way of documenting processes. Being able to answer "how do you develop features" or "what's best practice for X" with a usable process is great.
1 a) I'm particularly interested in using this for documenting how to perform experiments, and tracking experiments as they go.
2. I think this is amazing for progressively automating things. Split the task up into manageable, clear chunks to be performed by people. Each step can then (if it seems worth it) be replaced by code.
3. I don't think this is present at the moment (and it probably shouldn't be for a first release) but I'd like to see process hierarchies and importing others. I'd like to say 'Use github's "GITFLOW V2" process for managing feature additions and use "MY_COMPANY_QA V3" for reviewing features before full release'.
Anyway, this is great, and I look forward to playing around with it.
edit -
Even without these, I can already see myself using it just for me.
edit 2 - It'd be good to work on the getting started, so that there's just a "run this script" or "clone this repo". There's a lot of "download this file and copy it here" that feels like it should be scripted.
Your comment about embedding external QA workflows is super interesting. Right now Orchestra does support what we call 'Review Policies', which allow you to specify that a given workflow step be reviewed k times, or with probability X, or by specific expert Z. But it would be interesting to think of whole workflows for QA.
Re: getting started, yeah, we're working on a script--turns out cross-platform support is hard!
Great stuff! For now it seems like this leaves out the job of finding crowd workers. Could it be possible to do something like designing microtasks in MTurk and "stitching" the workflow together using Orchestra?
(Orchestra dev here too!) That's a great question. We're hoping that emerging online expert communities like Upwork or Dribbble will make it easier and easier to find experts suited to the work, but it's definitely not a solved problem (or tech recruiting would be so much easier!).
Re: MTurk, there's nothing stopping you from stitching microtasks together with Orchestra, but I think that undersells Orchestra's strengths (e.g., collaboration among workers) a bit.
This might seem a dumb question, but can you talk about how this improves on existing productivity workflows? Does it integrate with tasking systems like Jira or is this meant to be a complete a replacement?
It seems high-level (and great) I'm just curious what implementing it "in the field" would look like.
Products like Jira help organize tasks within workflows, but don't automate any that organization. Orchestra takes care of things like "which reporter should I assign to a new story?" without requiring humans to make those decisions.
Research from the Flash teams project that I mentioned in another comment (http://stanfordhci.github.io/flash-teams/index.html) has shown that this type of automation can decrease end-to-end workflow time by as much as 2x!
Does this tie into products like PivotalTracker or JIRA to make it useful for agile software development?
It would be nice to be able to create tasks from stories in Pivotal, assign a reviewer to them, and, when a reviewer thumbs-up, have it automatically built into a staging server. Anything like that in the works?
We haven't built that integration yet, but it's totally possible! Really all it should take is a Machine Step that calls out from Python to, say, Pivotal's APIs (https://pypi.python.org/pypi/pivotaltracker).
I don't know of any formal case studies, but Adam Marcus and myself (two devs on the project) worked on a large-scale workflow at GoDaddy where we automatically organized data extraction experts into hierarchies of review. That work actually got published at a Databases conference: see Adam's nice summary here: http://blog.marcua.net/post/128120315677
On the research side of things, the flash teams work by Daniela Retelny at Stanford (http://stanfordhci.github.io/flash-teams/) has done great proof-of-concept things like build an app in a day with distributed teams.
[+] [-] jrbotros|10 years ago|reply
Today we’re excited to be making our first contribution back to the tech community by open sourcing Orchestra, a system for coordinating project teams of experts and machines. It can be used to reimagine and implement all sorts of workflows, from professional services (like legal, design, and data analysis) to new product experiences (like M from Facebook). Orchestra is based upon years of research into machine-mediated expert teams. We hope you’ll come up with new and creative ways of using it!
At Unlimited Labs, we’re excited about building a brighter future of work and are currently building our first product on top of Orchestra. We’d love to answer any questions you have on here, or feel free to email the team at [email protected]!
[+] [-] marcua|10 years ago|reply
If you're curious how it can be used, Daniel Haas put together a wonderful example of how Orchestra could be used in a newsroom:
http://orchestra.readthedocs.org/en/latest/example_use.html
Happy to take questions!
[+] [-] IanCal|10 years ago|reply
1. I'm going to play with this as a way of documenting processes. Being able to answer "how do you develop features" or "what's best practice for X" with a usable process is great.
1 a) I'm particularly interested in using this for documenting how to perform experiments, and tracking experiments as they go.
2. I think this is amazing for progressively automating things. Split the task up into manageable, clear chunks to be performed by people. Each step can then (if it seems worth it) be replaced by code.
3. I don't think this is present at the moment (and it probably shouldn't be for a first release) but I'd like to see process hierarchies and importing others. I'd like to say 'Use github's "GITFLOW V2" process for managing feature additions and use "MY_COMPANY_QA V3" for reviewing features before full release'.
Anyway, this is great, and I look forward to playing around with it.
edit -
Even without these, I can already see myself using it just for me.
edit 2 - It'd be good to work on the getting started, so that there's just a "run this script" or "clone this repo". There's a lot of "download this file and copy it here" that feels like it should be scripted.
[+] [-] dhaas|10 years ago|reply
Your comment about embedding external QA workflows is super interesting. Right now Orchestra does support what we call 'Review Policies', which allow you to specify that a given workflow step be reviewed k times, or with probability X, or by specific expert Z. But it would be interesting to think of whole workflows for QA.
Re: getting started, yeah, we're working on a script--turns out cross-platform support is hard!
[+] [-] usman-m|10 years ago|reply
[+] [-] dhaas|10 years ago|reply
Re: MTurk, there's nothing stopping you from stitching microtasks together with Orchestra, but I think that undersells Orchestra's strengths (e.g., collaboration among workers) a bit.
[+] [-] CoreSet|10 years ago|reply
It seems high-level (and great) I'm just curious what implementing it "in the field" would look like.
[+] [-] dhaas|10 years ago|reply
Research from the Flash teams project that I mentioned in another comment (http://stanfordhci.github.io/flash-teams/index.html) has shown that this type of automation can decrease end-to-end workflow time by as much as 2x!
[+] [-] SamBam|10 years ago|reply
It would be nice to be able to create tasks from stories in Pivotal, assign a reviewer to them, and, when a reviewer thumbs-up, have it automatically built into a staging server. Anything like that in the works?
[+] [-] dhaas|10 years ago|reply
[+] [-] zallarak|10 years ago|reply
[+] [-] dhaas|10 years ago|reply
On the research side of things, the flash teams work by Daniela Retelny at Stanford (http://stanfordhci.github.io/flash-teams/) has done great proof-of-concept things like build an app in a day with distributed teams.
[+] [-] keredhaas|10 years ago|reply
[+] [-] dharma1|10 years ago|reply
[+] [-] ggiacaglia|10 years ago|reply
[+] [-] johndoe7|10 years ago|reply
[+] [-] dhaas|10 years ago|reply
[+] [-] tilek|10 years ago|reply