I started a company that was asked to write a proposal to make what became the app that failed last night. I declined because it was outside of our core skillset. You can read my prior comments or look at my profile to validate this. If you want my perspective as someone who started a Democratic software company, you should keep reading.
This app emerged from a mandate to make the caucus more accessible and transparent. It was well-intentioned but underfunded and lacked comprehensive organizational buy-in. Introducing tech can help but you have to spend tons of money to make it reliable and usable, then you have to spend more to train everyone in using it. This is a problem organizations of all sizes and shapes face when making massive IT changes.
Shadow is a firm that makes custom software for Democrats and progressives. It has an unnecessarily sinister name. There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because you are some combination– you choose– of secretly evil, secretly working for Bernie, secretly working for The Establishment/Hillary (per someone's unpersuasive Imgur post below), or secretly working for Buttigieg.
Others have noted that Shadow also made software for the Buttigieg campaign. If you take my claims above as true, this should be unsurprising to you: a hard market where everyone hates you and no one has money to pay you is not attractive to enterprising software engineers, so there are few firms available to choose.
I don't know whether I heard correctly or if the number was accurate, but I heard someone throw out $60k as the amount paid for the app development this morning on the radio.
That sounds like a lot of money to a lay-person, but assuming $100/hr, which seems like at least a reasonable ballpark rate, that breaks down to 600 man-hours, or 15 man-weeks.
That is a very tight deadline to turn out a critical app like this. Especially since I assume it wasn't just a single person doing the coding.
People don't realize that software is expensive and a custom built application is probably not the correct choice for a single-use product. I'm sure whoever was in charge felt that $60k was a quite generous offer to build this. In reality it's basically nothing.
I used to have acquaintances approach me and say, "Hey I've got this great app idea you could help me with."
The response I found shut them down real quick was, "Great. Do you have $100k lying around to get the first beta out the door?"
The problem didn’t come from making software for the Buttigieg campaign but the CEOs wife was on the Buttigieg campaign and he was a supporter.
Also Shadow was openly hostile to the Sanders campaign in numerous encounters.
It’s not as bad as the CEO of a major voting software company saying he’ll “bring home the votes for Republicans” (Diebold, 2004) a but this is a clear case of conflict of interest.
>There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because...
In other words it's exactly the kind of business that attracts activist types who have non-monetary motivations and it could therefore present an opportunity for tampering, particularly since any discrepancies can be dismissed as bugs from underfunding and rushed development.
Whatever is actually going on, the system is clearly insanely broken.
>Shadow is a firm that makes custom software for Democrats and progressives.
If I can ask, why the specific nature for their clientele? Isolating your solutions to a specific political party seems strange to me especially if the emphasis on making money. If anything, money normally supersedes political leanings at the end of the day which makes the whole fiasco even stranger to me. Wouldn't you want to provide a solution for both parties by default? Not doing so seems nefarious on it's own.
It sounds like pairing products with political leanings is not a good business decision? Which may explain where all the 'hate' is coming from - it's just bad business and sows the seeds of doubt in their credibility in the minds of many.
One underlying question is why there was an app for this in the first place, when election process activists basically say do it on paper and by hand with people and audits.
Anecdotally, nations doing it that way seem to report faster than when software and electronics become involved.
Can you comment on whether the proposal was part of the the SBIR / STTR program? Just curious as to the scope of companies that were participants in the proposal process.
As an aside - these instances of underfunded and underdeveloped services, specifically in the political sphere, are really painful reminders of the flip side to the idealistic technocratic future that I think a lot of folks around here assume to be an inevitability.
Curiously... Any take on Michael Slaby, Timshel (defunct), The Groundwork (defunct), the dozens (up to 70 at one point, many from SV/NYC) of engineers that worked on it for roughly two years in close contact with HFA 2016 (The Groundwork was right down the street from HFA 2016 HQ), and the $700,000 that HFA 2016 paid for that work? More generally, do you think there is a problem with for-profits which focus on political organizations under the guise of "focusing on non-profit"? How often do you think illegal in-kind contributions take place through these companies to political campaigns?
Why is any sort of "Democratic software company" even remotely necessary? Why can't they pay for white-labeled solutions? Or pay an established consulting agency to build things?
Please educate me on something. I fail to understand why "making custom software for <insert party here>" is something anyone would want to do in the context of owning and growing a real business. This seems fraught with all kinds of issues.
First of all (and it seems I've been using this word a lot lately) this turns software development into a religion rather than a business with solid business strategy. The equivalent in other businesses, like the entertainment industry, are people who do work for stars for nothing or nearly nothing because they think it will lead somewhere. All they get is a weird and abusive form of servitude (I've seen this first hand) and no future.
Why would someone not write software for politics instead and make part of their value proposition that they are not biased (and, of course, take the steps necessary to actually deliver that).
We have a close relative in South America who owns a company that makes and markets voting hardware and software. They are not aligned with any specific political party, movement or cult. They sell to everyone, in and outside South America and have a very nice business.
Can you share any more about the proposal process? Where was the RFP posted? What were the evaluation criteria? I was kind of assuming that this was more of a handshake deal or a cold sell or something, surprised to hear it went through a structured bid of some kind.
Did they make the text-banking app for Buttigieg, or merely sell them SaaS? Biden, Clinton, other Dem campaigns, and the DNC have all used Shadow's software.
> I declined because it was outside of our core skillset.
Um, isn't this app allowing people to enter data, and sending that to an API? How could that be out of anyone's core skillset? Almost every single app must do that.
Checked their LinkedIn and the employees who work there. The founder is non-technical and there are two developers: one who is a "back-end intern" and the other is a front-end developer (both are fresh from bootcamps).
For a company of its size, it certainly got a good amount of C-titled people.
CEO, COO, CTO...
Here's the thing I don't understand about them, I thought startups were averse to this? Am I wrong about startups or am I wrong about them being a startup?
What did this app do exactly? It was just a reporting app, I'm assuming that the source data collected, albeit perhaps in different formats, lived somewhere else and that there wasn't a lot of data or a lot of variations. From my understanding it seems there was an Auth0 redirect issue, but why wasn't this just a night of taking the source data and doing scripting then shipping out the reports via a secure DropBox type service? We've all be there where an ETL job fails, since it isn't critical (e.g., financial transaction), it wasn't tested every which way and we just had to some scripting.
Even if it is, say, 500TB of data, in 300 different formats usually those formats aren't drastically "different." Maybe I'm not understanding what the application was supposed to do to not understand why this wasn't solved every quickly. Or maybe given the timeline it was solved quickly once the right people got involved and figured out what needed to be done.
to be fair, they needed to scale to roughly a thousand concurrent users (where concurrent is used very loosely here). so I wouldn't say that scaleability was really a requirement. 1000 users is nothing.
I'm all about Hanlon's razor but naming the ticking time-bomb of a voting software startup "Shadow Inc." seems a bit too on the nose to be pure stupidity.
Everything about this reeks of a publicity stunt to "ruin" electronic voting in the public eye.
This app failed because no one did enough testing and enough training. What I saw was a relatively simple app to input some data. The problem seems to have been that they never tested it at scale and they assumed that all the users would be able to download it and use it.
My experience is that customers hate to pay for testing once they see the product running. They assume that it's done. I had a customer tell me that if I did the programming right it should always work and testing should be minimal. I had to explain to him that that's not the case with software and testing is one of the most important parts of the software development cycle. He felt it was a waste and that I was looking to add extra costs for no reason.
I bet there was not enough money. People seem to feel that a few grand will cover the costs. They figure 10k is an outrageous amount. What they don't seem to understand is that it will barely cover the costs of planning the app.
Both the developers and the people that approved the app for use need to take responsibility. Too bad since it could have saved a lot of money and time in the long run.
BTW, this could have been a Google Form with a spreadsheet as a back end. But user training would have still been an issue. You can't get around that.
This should really be getting more attention, here is an alternative breakdown: https://imgur.com/gallery/ycOC0HX.
The jest of it is one of the most important institutions in the United States (the Democratic National Committee) uses a highly nepotistic and incompetent system for managing IT which leads to colossal failures in marketing, canvassing, and security. Not to mention massive PII violations as millions of emails, phone numbers and SSNs, are passed around in plain-text via CSV files.
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously try shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
There's no reason why it has to be this way other than DC careers being an absolute rat race. Elections happen literally like clock work. There is no need to get an app out in 2 months; there will always be the next election if the app isn't ready to ship. Plenty of elections have happened smoothly without your app; it isn't necessary.
However, if you are only a campaign manager for six months or whatever and have to prove your worth or else move back in with your parents and spend the rest of your working life fetching coffee and slowly paying for your $200k georgetown political science degree, then yeah that app better be out in 2 damn months.
Rat races don't lead to quality, they lead to desperate people taking shortcuts and making mistakes, and fodder for managers to point to on a slide when they have to appear in front of superiors. Things that woudn't otherwise happen if pressure was lowered and we thought about things like burnout and well being rather than whatever meaningless work-related metric is hot right now. This sort of failure is inevitable in fields operating like this.
Well, you aren't technically wrong here, but I disagree with the sentiment. The RNC leveraged a the same kind of data used by the Obama team in '12 to squeak out an Electoral College win in `16.
I think it's easy to get overly ambitious with civic tech and deliver expensive garbage. However, when people stay focused and deliver then I think the results speak for themselves. While the '12 and '16 elections didn't turn on candidates tech strategies alone, it's clear that they contributed.
This seems overly paranoid to me - wouldn't we want people with experience in the civics tech space to be leaders at a company working in the civics tech space? Besides, wouldn't most of the good Bernie tech people... go back to work for Bernie? There's a reason why the Hillary people need new jobs.
I don’t agree with the use of an app here for many reasons. That said, such an app would basically need secure authentication and a form to upload some pre-templated numbers.
It seems pretty hard to screw that up so badly but clearly it’s quite easy to make a complete dumpster fire from those requirements.
According to multiple reports, the vote counts entered into app and sent were not what was received by HQ. How do we know that the company wasn't paid to change the numbers on the backend? How would the numbers change by themselves? I'll be called a tinfoil-hatter for assuming malice.
This is why we need to start using Byzantine Fault Tolerant distributed systems to vote via our mobile phones. No need for a voting holiday or standing in line. Bigger turnout, too. If it's secure enough for banking apps, why not for opt-in voting via app.
Maybe it's too hard to move to electronic voting nationwide. But every organization has governance and could use an electronic voting system based on BFT consensus of mutually distrusting parties. Vote using an app, it gets stored "on-chain", then you can check it on another app.
Wow! That provides some great info on this. Still leaves me wondering if the end result wasn't by design though. It's kind of hard to buy into the "Opps" explanation.
In particular it looks like Firefox Fenix/Preview (or possibly Reference Browser), not Fennec (regular Android). Only Fenix and Reference have bottom URL/tab bars. Fenix has a purple "Try Again" button, I don't know about Reference Browser, and Fennec has a gray button.
One possible issue is that Fennec/Fenix doesn't open http URLs in external apps by default (unsure about custom protocols), whereas Chrome does.
[+] [-] nooron|6 years ago|reply
This app emerged from a mandate to make the caucus more accessible and transparent. It was well-intentioned but underfunded and lacked comprehensive organizational buy-in. Introducing tech can help but you have to spend tons of money to make it reliable and usable, then you have to spend more to train everyone in using it. This is a problem organizations of all sizes and shapes face when making massive IT changes.
Shadow is a firm that makes custom software for Democrats and progressives. It has an unnecessarily sinister name. There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because you are some combination– you choose– of secretly evil, secretly working for Bernie, secretly working for The Establishment/Hillary (per someone's unpersuasive Imgur post below), or secretly working for Buttigieg.
Others have noted that Shadow also made software for the Buttigieg campaign. If you take my claims above as true, this should be unsurprising to you: a hard market where everyone hates you and no one has money to pay you is not attractive to enterprising software engineers, so there are few firms available to choose.
[+] [-] war1025|6 years ago|reply
That sounds like a lot of money to a lay-person, but assuming $100/hr, which seems like at least a reasonable ballpark rate, that breaks down to 600 man-hours, or 15 man-weeks.
That is a very tight deadline to turn out a critical app like this. Especially since I assume it wasn't just a single person doing the coding.
People don't realize that software is expensive and a custom built application is probably not the correct choice for a single-use product. I'm sure whoever was in charge felt that $60k was a quite generous offer to build this. In reality it's basically nothing.
I used to have acquaintances approach me and say, "Hey I've got this great app idea you could help me with."
The response I found shut them down real quick was, "Great. Do you have $100k lying around to get the first beta out the door?"
[+] [-] r00fus|6 years ago|reply
Also Shadow was openly hostile to the Sanders campaign in numerous encounters.
It’s not as bad as the CEO of a major voting software company saying he’ll “bring home the votes for Republicans” (Diebold, 2004) a but this is a clear case of conflict of interest.
[+] [-] allovernow|6 years ago|reply
In other words it's exactly the kind of business that attracts activist types who have non-monetary motivations and it could therefore present an opportunity for tampering, particularly since any discrepancies can be dismissed as bugs from underfunding and rushed development.
Whatever is actually going on, the system is clearly insanely broken.
[+] [-] whylie|6 years ago|reply
If I can ask, why the specific nature for their clientele? Isolating your solutions to a specific political party seems strange to me especially if the emphasis on making money. If anything, money normally supersedes political leanings at the end of the day which makes the whole fiasco even stranger to me. Wouldn't you want to provide a solution for both parties by default? Not doing so seems nefarious on it's own.
It sounds like pairing products with political leanings is not a good business decision? Which may explain where all the 'hate' is coming from - it's just bad business and sows the seeds of doubt in their credibility in the minds of many.
[+] [-] dv_dt|6 years ago|reply
Anecdotally, nations doing it that way seem to report faster than when software and electronics become involved.
[+] [-] ihunter2839|6 years ago|reply
As an aside - these instances of underfunded and underdeveloped services, specifically in the political sphere, are really painful reminders of the flip side to the idealistic technocratic future that I think a lot of folks around here assume to be an inevitability.
[+] [-] birdyrooster|6 years ago|reply
[+] [-] atwebb|6 years ago|reply
https://simple.wikipedia.org/wiki/Hanlon%27s_razor
[+] [-] Benjammer|6 years ago|reply
[+] [-] robomartin|6 years ago|reply
First of all (and it seems I've been using this word a lot lately) this turns software development into a religion rather than a business with solid business strategy. The equivalent in other businesses, like the entertainment industry, are people who do work for stars for nothing or nearly nothing because they think it will lead somewhere. All they get is a weird and abusive form of servitude (I've seen this first hand) and no future.
Why would someone not write software for politics instead and make part of their value proposition that they are not biased (and, of course, take the steps necessary to actually deliver that).
We have a close relative in South America who owns a company that makes and markets voting hardware and software. They are not aligned with any specific political party, movement or cult. They sell to everyone, in and outside South America and have a very nice business.
[+] [-] yazboo|6 years ago|reply
[+] [-] bradgessler|6 years ago|reply
[+] [-] BLanen|6 years ago|reply
[+] [-] saluki|6 years ago|reply
[+] [-] jadell|6 years ago|reply
[+] [-] akhilcacharya|6 years ago|reply
[+] [-] charred_toast|6 years ago|reply
[+] [-] megablast|6 years ago|reply
Um, isn't this app allowing people to enter data, and sending that to an API? How could that be out of anyone's core skillset? Almost every single app must do that.
[+] [-] 3uclid|6 years ago|reply
...Yikes.
[+] [-] thedance|6 years ago|reply
[+] [-] sct202|6 years ago|reply
[+] [-] Aperocky|6 years ago|reply
CEO, COO, CTO...
Here's the thing I don't understand about them, I thought startups were averse to this? Am I wrong about startups or am I wrong about them being a startup?
[+] [-] skrowl|6 years ago|reply
At least now I don't feel bad about when I test in production. (just kidding, I didn't feel bad before).
[+] [-] LargeWu|6 years ago|reply
[+] [-] geoff-dot|6 years ago|reply
Even if it is, say, 500TB of data, in 300 different formats usually those formats aren't drastically "different." Maybe I'm not understanding what the application was supposed to do to not understand why this wasn't solved every quickly. Or maybe given the timeline it was solved quickly once the right people got involved and figured out what needed to be done.
[+] [-] altitudinous|6 years ago|reply
I would have written an app that doesn't work for only $25000
[+] [-] wizzard|6 years ago|reply
[+] [-] meshko|6 years ago|reply
[+] [-] padseeker|6 years ago|reply
[+] [-] JackRabbitSlim|6 years ago|reply
Everything about this reeks of a publicity stunt to "ruin" electronic voting in the public eye.
[+] [-] jessaustin|6 years ago|reply
https://thegrayzone.com/2020/02/04/pro-israel-buttigieg-seth...
[+] [-] pmoriarty|6 years ago|reply
Going back to voting exclusively by ink on paper and hand counting is imperative.
[+] [-] WheelsAtLarge|6 years ago|reply
My experience is that customers hate to pay for testing once they see the product running. They assume that it's done. I had a customer tell me that if I did the programming right it should always work and testing should be minimal. I had to explain to him that that's not the case with software and testing is one of the most important parts of the software development cycle. He felt it was a waste and that I was looking to add extra costs for no reason.
I bet there was not enough money. People seem to feel that a few grand will cover the costs. They figure 10k is an outrageous amount. What they don't seem to understand is that it will barely cover the costs of planning the app.
Both the developers and the people that approved the app for use need to take responsibility. Too bad since it could have saved a lot of money and time in the long run.
BTW, this could have been a Google Form with a spreadsheet as a back end. But user training would have still been an issue. You can't get around that.
[+] [-] tmpz22|6 years ago|reply
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously try shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
[+] [-] asdff|6 years ago|reply
However, if you are only a campaign manager for six months or whatever and have to prove your worth or else move back in with your parents and spend the rest of your working life fetching coffee and slowly paying for your $200k georgetown political science degree, then yeah that app better be out in 2 damn months.
Rat races don't lead to quality, they lead to desperate people taking shortcuts and making mistakes, and fodder for managers to point to on a slide when they have to appear in front of superiors. Things that woudn't otherwise happen if pressure was lowered and we thought about things like burnout and well being rather than whatever meaningless work-related metric is hot right now. This sort of failure is inevitable in fields operating like this.
[+] [-] ch4s3|6 years ago|reply
Well, you aren't technically wrong here, but I disagree with the sentiment. The RNC leveraged a the same kind of data used by the Obama team in '12 to squeak out an Electoral College win in `16.
I think it's easy to get overly ambitious with civic tech and deliver expensive garbage. However, when people stay focused and deliver then I think the results speak for themselves. While the '12 and '16 elections didn't turn on candidates tech strategies alone, it's clear that they contributed.
[+] [-] tptacek|6 years ago|reply
[+] [-] sjm-lbm|6 years ago|reply
[+] [-] code4tee|6 years ago|reply
It seems pretty hard to screw that up so badly but clearly it’s quite easy to make a complete dumpster fire from those requirements.
[+] [-] meristem|6 years ago|reply
Ok, did anyone actually design the app?
[+] [-] fuqmachine|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] EGreg|6 years ago|reply
Maybe it's too hard to move to electronic voting nationwide. But every organization has governance and could use an electronic voting system based on BFT consensus of mutually distrusting parties. Vote using an app, it gets stored "on-chain", then you can check it on another app.
[+] [-] trianglem|6 years ago|reply
[+] [-] smoyer|6 years ago|reply
[+] [-] abetlen|6 years ago|reply
My favorite screenshot is the last one, it looks like a generic mobile Firefox error for a misspecified URL.
[+] [-] dang|6 years ago|reply
[+] [-] oblib|6 years ago|reply
[+] [-] _fbpt|6 years ago|reply
One possible issue is that Fennec/Fenix doesn't open http URLs in external apps by default (unsure about custom protocols), whereas Chrome does.