There are countless flaws in this comparison, but the one that stands out to me is the simple buying vs renting issue. They sourced screws that would be permanently installed into something. They rented a couple hours on a server.
Renting is always cheaper than owning. That's a pretty universal rule. Complain about the red tape all you want, I have also felt the burn of the "approved supplier" list. That doesn't excuse an extremely faulty comparison, however. It just clouds the issue.
I understand coming up with a better comparison would be a little more work than the immediate but completely faulty on-hand cost of the server instance they spun up, but I know the cost of the computer I own and the cost of a server I rent and I wouldn't use them to make a comparison just because I'm too bothered to come up with a real comparison.
There is also the issue of the fact that the $2000 number was pulled out of seemingly thin air, the possibility that they use approved suppliers to avoid the issue of counterfeit or sub-standard parts, which have killed people when installed into regular things like bleachers that stay on the Earth. It's the freaking Canadarm, god forbid we splurge now so we don't waste millions later fixing it or kill someone. When the end result of the materialized idea is it's going to be launched into space, the costs probably aren't so black and white.
Interesting story, but with so many variables unaccounted for I have a hard time gleaning any sort of lesson from it. Other than the fact that building a piece of equipment that will be launched into space involves a fair bit of red tape, which I was already aware of and am honestly kind of thankful for.
The comment backlash against this anecdote is a little odd to me. No, it's not a perfect analogy. No, the cost comparison is neither fair nor complete nor comprehensive.
But it is an interesting connection to make, and there are interesting ideas that the comparison generates, so let's treat it like a parable rather than a critical business analysis.
If anything, one of the most striking comparisons is how work was done in a government-funded research lab versus how it is done in a startup. Own vs rent, meticulous simulation vs rapidly-iterated prototyping, waterfall vs agile, regulation vs free for all.
The superficial details of the story aren't actually that important, but there's a lot of meat to dig into.
People are complaining because the author ignores his own take-away: "Different time, different industry."
All of this "agile," "free for all," "start up" cuteness is okay when you're dealing with software, which generally only ever results in frustration or at worst lost profits. The standards and practices in industries where things can go wrong in epic, expensive, and dangerous ways is obviously quite different.
The first example includes cost of labour. The second example doesn't.
Two people playing with that cluster for a couple of hours cost more than $45.
Still, the author is right that when building physical products sometimes things are a lot more expensive than you think they're going to be because you're following some strict protocol.
Yes, you are absolutely right. But I think this is not the main point of the article.
The main point is that in a huge corporation with rules for purchasing and choosing suppliers, there's an insane amount of labor spent in administration even for purchasing even the most mundane things, as soon as it's something deviating from "the norm".
I've worked in a highly regulated industry, in a pretty big company, and my experience was the following: If you always order €1000 devices from a company, this company is registered in the internal databases, and had an audit from your quality assurance company. All traffic-lights will be green. You order the device, you, your boss, the financial responsible or whatever signs the document and they will bill one hour of work for that single-hand-move operation, which will be in the order of €100/$100. Total cost for your department of project: €1100 for tie €1000 device.
Now if you are in the lab, and think it's cool to use that Arduino shield for $10 on seedstudio for your lab-setup (which is probably a completely reasonable approach, as you only want to turn on/off that light/motor/... you normally would have to do manually for your experiment/measurement/test) the pain begins:
Seedstudio is not on the list of approved vendors, the QA team has to be persuaded that this is not a item that will ever reach a customers installation so they don't travel there to do an audit of the ISO9000 compliance (or whatever special regulation your partícular industry demands). Purchasing will complain that seedstudio has never signed your "terms of order" (and they will just throw away the 10-page-legalese anyway for $10...). And controlling will complain endlessly about the "proper inventory code" for a "seedstudio stepper motor shield". They'll also give you a sticker and make it a "IT inventory item" because it can be connected to a PC. In the end you'll have 5 people bill each 2 hours on that issue, and your project will be billed 52$100 + $10 = 1010$ of the Arduino shield. And you'll be known as "that hacker guy" that "constantly orders strange things from chinese companies" through the whole company.
Everyone new to that kind of big corporation tries this once, or twice, and then decides that it's just not worth it. In practice, you'll buy it on your private credit card, and hand it in as "travel expenses" on your next business trip. Or you'll have subcontractor already working with you that will order it for you, forward the device, and just put "Misc. manufacturing material. $50" on the next monthly $20'000 bill that on one will ever check.
Ordering from an approved supplier with a traceable chain of delivery for equipment used in a space program seems quite reasonable to me. There is a reason that people from the more traditional engineering disciplines (aerospace, chemical, civil, mechanical) are so anal-retentive about "proper engineering guidelines." It's because they deal with things that blow up, and when they do blow up shoot turbine blade shrapnel all over the place. Nearly everyone working in these fields has some story about how a little screw up in one out-of-the-way component caused some epic disaster that involved fire.
Yeah, sure in this case it was just some screws on a plastic housing and there was a 99.999% chance that everything would've been fine. But you don't want to make exceptions, because if you play fast and loose here and there, eventually all the prudent practices go out the window and you're no better than the software guys.
This is a huge point that many HNers probably don't get(but they think that they get). Releasing patches/bugfixes doesn't work so well on physical stuff. If Amazon AWS goes down, a group of people can figure out what went wrong, fix the issue and "reboot" the system and all that is lost is a bit of revenue and a portion of their reputation. When pieces break on heavy iron you don't have the time to figure out what went wrong and then fix it because someone is about to crash. Hence the difference between the traditional engineers and the software folks is the traditional engineers have to fix stuff before it breaks, software engineers can afford to fix stuff after it breaks.
This happens all the time in government. Business has the lucky position of constantly scrambling to cut costs, cut redundancy, cut inefficiency. Otherwise they'll be beaten by a new company that isn't tied down by self-induced paralysis. The government has no such incentive to do so. When faced with a choice between saving fifty billion dollars and having the assurance that everything is hunky-dory, the government will spend the fifty billion dollars without a second thought.
One of my duties, along with being a radio technician, is ordering replacement parts for the systems that we work on. Some of the stuff is expensive and reasonably so - for example, a circuit board with $200 bucks in components is $20,000 because it has to be professionally made and is basically a one-of-a-kind part. The supplier has to charge that just to recoup the costs of retooling. We accept this because we need very reliable parts on radar systems and are willing to pay for the assurance that these parts will be good.
On the other hand, I just ordered an audio cable through the FAA. Simple two-wire audio cable... $333.45. I could've bought the exact same thing from Radioshack for five bucks. Or Monoprice for one dollar.
At one point, Raytheon told us that we were not allowed to do any intermediate-level maintenance (anything involving fixing circuit cards) on their stuff. We were expected to send the bad cards to them, and they would give us replacements and bill us for $10,000 each. The chief warrant officer laughed, said "Fuuuck no," and told us that if we could easily fix it, do so.
Raytheon realized we weren't sending them any circuit cards and called the commanding general to get him relieved. He didn't get fired, but we started sending the cards to them for broken 2-cent resistors. For ten grand each. Same thing with $30,000 power supplies, etc.
Right now, the government has declared that getting office supplies from Staples is horrible. We're supposed to get them from Servmart, which sells them for five times the price... or more. Cheap-shit ballpoint pens for a dollar each when I can get Bics for $4.60 for 72 of them.
Wow - That is a really amazing question. I'll be wondering about it for a while...
The following is speculation.
On AliBarBar, you can find wholesale pans at source in china for $1 each. The price of pi like devices is about $30 dollars.
So, why do pans get so much more expensive as they cross the ocean?
1. Cost of postage
2. Protectionist import taxes. Here is Europeans list of things they are thinking of levvying punative taxes on. It's long. http://trade.ec.europa.eu/tdi/notices.cfm
3. Regulatory approval. The burden of bringing a new electronic device to market is pretty high. You need specialist facilities (or to pay a lot of money to a test lab) to prove electrical emissions are fine. This is why so many Chinese designs never make it to the UK. Alibaba is full of lovely android powered arm net-tops, but you note the ones you see in electronic stores are from the same limited range, even the cheap ones. This is why.
I presume the same is true for pans. So although designing a pan and bringing it to manufacture is simpler than manufacturing a new ASIC or even embedded computer, only a very few pan designs can be sold in the UK. It just isn't worth bringing another one through approval (unless it's much cheaper). And the approved designs can sell for much more.
One hears about western companies keeping as much of the 'value add' as possible out of China and India, possibly because it's so expensive to retrieve money once it's in there. I wonder if this also is part of it.
This is why as an intern at NASA I decided government work wasn't for me and I eventually ended in startup world. However, I think there is something to be said for creating within strict guidelines and procedures, it can sometimes lead to amazing engineering achievements.
This also shows in how quick you can iterate. The "REPL time" in mechanical engineering has decreased a lot with CAD software but it's still insanely long compared to software development.
I used to work in a lab with a researcher who designed complex mechanical systems with lots of rotating parts made from rare and expensive materials. Everytime he needed even the slightest modification he had to wait for at least three months to have a new part machined.
How a comment cost $10,000 and a car cost $5: I got you all to read this, wasting your time and electricity, computer costs, etc. In contrast, I got a cab yesterday and it only cost $5.
I don't generally insist that everything posted be transcendent, but I fail to see even a fair point being made in the post. If the point is "buying things for an aerospace corporation is more expensive than renting an hour on EC2", okay, but we sort of knew that already.
I work in hardware, and wanted to give my perspective on this.
Designing and building hardware is expensive, really expensive. If you don't take the time to follow proper procedures you are not only wasting your time, but often hundreds to thousands of dollars of parts you are never going to use. Going ahead without thinking can easily put you 2 or more weeks behind just because that is how long it takes your supplier to make what you ordered. You don't just quickly spin up an instance of a test, you have to buy or rent equipment that is often in the thousands to hundreds of thousands of dollars range.
I can see exactly why you wouldn't want to just go to Home Depot when working on equipment like what MDA makes. Say you go to Home depot and buy your fasteners, now the company has to pay for your time and reimburse the cost, which is almost certainly going to be as much or more to process than the courier charge ($30-$40) to ship the parts from the company you are making monthly payments to regardless. Then there is the additional risk of bringing in parts that could get mixed up with real production parts. There is a huge risk if you somehow get that Home Depot fastener mixed up with the special aerospace fastener you are using, the extra cost is worth it as insurance that "bad" fasteners aren't getting into the product. I can say from experience, it is ridiculously easy to get parts mixed up even when you are careful.
I fail to see how filling out $2000 in paperwork will stop you from getting screws mixed up. Because that's what the cost was, apparently. Not the actual equipment.
Those rules are important. Not every company in the world is a start up with <100 people. Once you have thousands of employees, you need controls. You can't have interns selecting suppliers or putting stuff into the bill of materials, are you serious with this?
Unfortunately we're just as much to blame for the red tape. It didn't get created out of a vacuum, it was created to protect people. I wrote a whole blog post called Why Does a Hammer Costs $5000 at http://www.followsteph.com/2011/11/22/why-does-a-hammer-cost...
[+] [-] shinratdr|13 years ago|reply
Renting is always cheaper than owning. That's a pretty universal rule. Complain about the red tape all you want, I have also felt the burn of the "approved supplier" list. That doesn't excuse an extremely faulty comparison, however. It just clouds the issue.
I understand coming up with a better comparison would be a little more work than the immediate but completely faulty on-hand cost of the server instance they spun up, but I know the cost of the computer I own and the cost of a server I rent and I wouldn't use them to make a comparison just because I'm too bothered to come up with a real comparison.
There is also the issue of the fact that the $2000 number was pulled out of seemingly thin air, the possibility that they use approved suppliers to avoid the issue of counterfeit or sub-standard parts, which have killed people when installed into regular things like bleachers that stay on the Earth. It's the freaking Canadarm, god forbid we splurge now so we don't waste millions later fixing it or kill someone. When the end result of the materialized idea is it's going to be launched into space, the costs probably aren't so black and white.
Interesting story, but with so many variables unaccounted for I have a hard time gleaning any sort of lesson from it. Other than the fact that building a piece of equipment that will be launched into space involves a fair bit of red tape, which I was already aware of and am honestly kind of thankful for.
[+] [-] icebraining|13 years ago|reply
Yes, but OP clearly says the boxes were never to be used in production, only testing.
[+] [-] mariuolo|13 years ago|reply
[+] [-] CognitiveLens|13 years ago|reply
But it is an interesting connection to make, and there are interesting ideas that the comparison generates, so let's treat it like a parable rather than a critical business analysis.
If anything, one of the most striking comparisons is how work was done in a government-funded research lab versus how it is done in a startup. Own vs rent, meticulous simulation vs rapidly-iterated prototyping, waterfall vs agile, regulation vs free for all.
The superficial details of the story aren't actually that important, but there's a lot of meat to dig into.
[+] [-] rayiner|13 years ago|reply
All of this "agile," "free for all," "start up" cuteness is okay when you're dealing with software, which generally only ever results in frustration or at worst lost profits. The standards and practices in industries where things can go wrong in epic, expensive, and dangerous ways is obviously quite different.
[+] [-] plam|13 years ago|reply
[+] [-] DanBC|13 years ago|reply
Two people playing with that cluster for a couple of hours cost more than $45.
Still, the author is right that when building physical products sometimes things are a lot more expensive than you think they're going to be because you're following some strict protocol.
[+] [-] cnvogel|13 years ago|reply
The main point is that in a huge corporation with rules for purchasing and choosing suppliers, there's an insane amount of labor spent in administration even for purchasing even the most mundane things, as soon as it's something deviating from "the norm".
I've worked in a highly regulated industry, in a pretty big company, and my experience was the following: If you always order €1000 devices from a company, this company is registered in the internal databases, and had an audit from your quality assurance company. All traffic-lights will be green. You order the device, you, your boss, the financial responsible or whatever signs the document and they will bill one hour of work for that single-hand-move operation, which will be in the order of €100/$100. Total cost for your department of project: €1100 for tie €1000 device.
Now if you are in the lab, and think it's cool to use that Arduino shield for $10 on seedstudio for your lab-setup (which is probably a completely reasonable approach, as you only want to turn on/off that light/motor/... you normally would have to do manually for your experiment/measurement/test) the pain begins:
Seedstudio is not on the list of approved vendors, the QA team has to be persuaded that this is not a item that will ever reach a customers installation so they don't travel there to do an audit of the ISO9000 compliance (or whatever special regulation your partícular industry demands). Purchasing will complain that seedstudio has never signed your "terms of order" (and they will just throw away the 10-page-legalese anyway for $10...). And controlling will complain endlessly about the "proper inventory code" for a "seedstudio stepper motor shield". They'll also give you a sticker and make it a "IT inventory item" because it can be connected to a PC. In the end you'll have 5 people bill each 2 hours on that issue, and your project will be billed 52$100 + $10 = 1010$ of the Arduino shield. And you'll be known as "that hacker guy" that "constantly orders strange things from chinese companies" through the whole company.
Everyone new to that kind of big corporation tries this once, or twice, and then decides that it's just not worth it. In practice, you'll buy it on your private credit card, and hand it in as "travel expenses" on your next business trip. Or you'll have subcontractor already working with you that will order it for you, forward the device, and just put "Misc. manufacturing material. $50" on the next monthly $20'000 bill that on one will ever check.
[+] [-] Radim|13 years ago|reply
The cost and value of using the bought products (screws/servers) is not included in either example.
[+] [-] rayiner|13 years ago|reply
Yeah, sure in this case it was just some screws on a plastic housing and there was a 99.999% chance that everything would've been fine. But you don't want to make exceptions, because if you play fast and loose here and there, eventually all the prudent practices go out the window and you're no better than the software guys.
[+] [-] olympus|13 years ago|reply
This is a huge point that many HNers probably don't get(but they think that they get). Releasing patches/bugfixes doesn't work so well on physical stuff. If Amazon AWS goes down, a group of people can figure out what went wrong, fix the issue and "reboot" the system and all that is lost is a bit of revenue and a portion of their reputation. When pieces break on heavy iron you don't have the time to figure out what went wrong and then fix it because someone is about to crash. Hence the difference between the traditional engineers and the software folks is the traditional engineers have to fix stuff before it breaks, software engineers can afford to fix stuff after it breaks.
[+] [-] omegaham|13 years ago|reply
One of my duties, along with being a radio technician, is ordering replacement parts for the systems that we work on. Some of the stuff is expensive and reasonably so - for example, a circuit board with $200 bucks in components is $20,000 because it has to be professionally made and is basically a one-of-a-kind part. The supplier has to charge that just to recoup the costs of retooling. We accept this because we need very reliable parts on radar systems and are willing to pay for the assurance that these parts will be good.
On the other hand, I just ordered an audio cable through the FAA. Simple two-wire audio cable... $333.45. I could've bought the exact same thing from Radioshack for five bucks. Or Monoprice for one dollar.
At one point, Raytheon told us that we were not allowed to do any intermediate-level maintenance (anything involving fixing circuit cards) on their stuff. We were expected to send the bad cards to them, and they would give us replacements and bill us for $10,000 each. The chief warrant officer laughed, said "Fuuuck no," and told us that if we could easily fix it, do so.
Raytheon realized we weren't sending them any circuit cards and called the commanding general to get him relieved. He didn't get fired, but we started sending the cards to them for broken 2-cent resistors. For ten grand each. Same thing with $30,000 power supplies, etc.
Right now, the government has declared that getting office supplies from Staples is horrible. We're supposed to get them from Servmart, which sells them for five times the price... or more. Cheap-shit ballpoint pens for a dollar each when I can get Bics for $4.60 for 72 of them.
Your tax dollars at work, gents.
[+] [-] ctdonath|13 years ago|reply
Someone explain why a frying pan and a Raspberry Pi cost the same.
[+] [-] shubb|13 years ago|reply
The following is speculation.
On AliBarBar, you can find wholesale pans at source in china for $1 each. The price of pi like devices is about $30 dollars.
So, why do pans get so much more expensive as they cross the ocean?
1. Cost of postage
2. Protectionist import taxes. Here is Europeans list of things they are thinking of levvying punative taxes on. It's long. http://trade.ec.europa.eu/tdi/notices.cfm
3. Regulatory approval. The burden of bringing a new electronic device to market is pretty high. You need specialist facilities (or to pay a lot of money to a test lab) to prove electrical emissions are fine. This is why so many Chinese designs never make it to the UK. Alibaba is full of lovely android powered arm net-tops, but you note the ones you see in electronic stores are from the same limited range, even the cheap ones. This is why.
I presume the same is true for pans. So although designing a pan and bringing it to manufacture is simpler than manufacturing a new ASIC or even embedded computer, only a very few pan designs can be sold in the UK. It just isn't worth bringing another one through approval (unless it's much cheaper). And the approved designs can sell for much more.
One hears about western companies keeping as much of the 'value add' as possible out of China and India, possibly because it's so expensive to retrieve money once it's in there. I wonder if this also is part of it.
[+] [-] guelo|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] pbo|13 years ago|reply
I used to work in a lab with a researcher who designed complex mechanical systems with lots of rotating parts made from rare and expensive materials. Everytime he needed even the slightest modification he had to wait for at least three months to have a new part machined.
[+] [-] StavrosK|13 years ago|reply
I don't generally insist that everything posted be transcendent, but I fail to see even a fair point being made in the post. If the point is "buying things for an aerospace corporation is more expensive than renting an hour on EC2", okay, but we sort of knew that already.
[+] [-] starky|13 years ago|reply
Designing and building hardware is expensive, really expensive. If you don't take the time to follow proper procedures you are not only wasting your time, but often hundreds to thousands of dollars of parts you are never going to use. Going ahead without thinking can easily put you 2 or more weeks behind just because that is how long it takes your supplier to make what you ordered. You don't just quickly spin up an instance of a test, you have to buy or rent equipment that is often in the thousands to hundreds of thousands of dollars range.
I can see exactly why you wouldn't want to just go to Home Depot when working on equipment like what MDA makes. Say you go to Home depot and buy your fasteners, now the company has to pay for your time and reimburse the cost, which is almost certainly going to be as much or more to process than the courier charge ($30-$40) to ship the parts from the company you are making monthly payments to regardless. Then there is the additional risk of bringing in parts that could get mixed up with real production parts. There is a huge risk if you somehow get that Home Depot fastener mixed up with the special aerospace fastener you are using, the extra cost is worth it as insurance that "bad" fasteners aren't getting into the product. I can say from experience, it is ridiculously easy to get parts mixed up even when you are careful.
[+] [-] Dylan16807|13 years ago|reply
[+] [-] kfk|13 years ago|reply
[+] [-] FollowSteph3|13 years ago|reply
[+] [-] dschiptsov|13 years ago|reply