top | item 26597293

In Stockholm there's currently an IT battle underway

566 points| div3rs3 | 5 years ago |twitter.com

231 comments

order

snicksnak|5 years ago

The changes are pretty ridiculous [1], apparently the gobernment's devs kept deliberately renaming a specific header by adding and increasing a number at the end, from `xsrf-token` up to `xsrf-token11` [0], clearly only with the intend to break their rival free market app.

[0] https://twitter.com/oppnaskolplatt/status/137505230118290637...

[1] https://github.com/kolplattformen/embedded-api/commit/b61122...

yoelo|5 years ago

The devs don't work for the Stockholm region nor the government. They are contractors. Stockholm is notorious for terrible fragmented outsourcing programs

ZephyrBlu|5 years ago

It's hilarious that this is actually happening.

bitterblotter|5 years ago

They should just fuzzy match the headers!! Just kidding. This is ridiculous

mwidell|5 years ago

Maybe Öppna Skolplattformen should just change their code to identify the token by its contents, no matter the parameter name?

johngalt|5 years ago

It's amazing how often giant IT projects like these go off the rails. Exploding costs and garbage implementations. If you start the sentence with: "A 5+ year government IT project that cost over 100 million-" I already know how the story ends.

How do we improve this common scenario? What are the root causes?

The common themes are:

1. Lack of technical project competence at the decision maker level.

2. Scope creep. Where the one true system has to do everything.

3. A 'one-pass' approach where everything is expected to be delivered as a working system at the end of the project.

Even fixing two of these gives us a solid shot at a successful project.

908B64B197|5 years ago

4. The requirements are often flat out wrong.

The Phoenix payroll system comes to mind, the Canadian government tried to shift the blame to IBM, but have their hands tied since IBM delivered exactly what was in the contract. It's just that what the government decided to put in the contract has little to do with how they really do payroll.

jordanbeiber|5 years ago

Projects are necessary for things that are built to last and not change much during their lifetime:

- a bridge

- airplanes

- most houses

- etc

Hardware comes to mind - it’s all hardware on the list, basically.

Software (outside certain realms ofc) like this? I’ve been, like many others here, doing this software thing for 20+ years now.

Big and small, I’ve basically never seen anything spawned from a project-driven organization actually deliver great results.

Most software is supposed to change, indefinitely - that’s the point!

Everyone in this day and age should know that requirements change over even short periods of time, so why even bother trying to pin them down in detail up front - you’re going to do everyone involved a disservice.

There is something to this agile thing and a “project” is it’s anti-pattern.

Not mention how much a quick feedback loop will learn you about the operation side of things.

Operations and change, it’s all you can build for and that is best done one step at a time.

(This joke of a platform is spread across multiple (5?) vendors/partners no less. A couple of them probably started just for this, backed with vc funding. It’s most likely a glorious mess!)

End rant.

ilaksh|5 years ago

There should be laws the require the use of open distributed extensible protocols.

That means the government comes up with a way for the parties to collaborate and just enforces that. So the actual implementation is open to competition.

Things like communication protocols or extensible APIs or schemas for exchanging APIs.

peteretep|5 years ago

Also internal company politics. What would fix much of this, for many projects, is a very experienced product person with final say over every feature, reporting directly to the CEO / board, and outranking everyone else in terms of decision making.

anvemaha|5 years ago

In case anyone wants to read some research on the success rate of IT projects Standish Group's Chaos Reports are a good place to start.

I read the 2014 one as a part of our project management uni course, but couldn't find it with a simple Google search so here's the 2015 one:

[pdf] https://www.standishgroup.com/sample_research_files/CHAOSRep...

cfn|5 years ago

I once interviewed with a company that had been hired by the UK government to create a system for one of their agencies. They told me that the system had 1000 requirements (literally) had already been implemented once and failed but they wanted to have another go. Boggles the mind (and I refused their offer).

throw0101a|5 years ago

> It's amazing how often giant IT projects like these go off the rails. Exploding costs and garbage implementations. If you start the sentence with: "A 5+ year government IT project that cost over 100 million-" I already know how the story ends.

People don't notice when things go right.

What about all the 5+ year, $100M projects you didn't hear about, because they never made the news, because the project went smoothly?

Even in general conversation we tend to vent about how bad our day/week was, and not how awesome something went:

* https://en.wikipedia.org/wiki/Negativity_bias

DelightOne|5 years ago

D4. Not having the necessary data available through (good) APIs for everybody to use.

AdrianB1|5 years ago

Cities are ran by politicians that in most cases are too incompetent to do anything better with their lives. When incompetent leaders make decisions, the incompetence is flowing to most, if not everything they do, including this kind of projects. There is no immediate fix for this situation, unless the decision would be made by some kind of city manager or board of experts that are recruited based on competence and held accountable for the results (up to prison, if needed). The old story with "politicians pay with their mandate, they will not be reelected if they fail" is a story for toddlers, that is not a punishment on par with the damages they make, will the Stockholm mayor pay back 100 million dollars?

zhte415|5 years ago

Just have everyone do PMP. Project management problems solved.

cromka|5 years ago

"The city's local government, @Stockholmsstad, spent 1 billion Swedish crowns (100 million dollars)"

"@Stockholmsstad are now acting like angry toddlers."

I'd read between the lines. I don't think it's a hurt ego, not with these numbers. Someone is (continued) to be paid for this to be happening. This reeks corruption.

hutzlibu|5 years ago

Well, angry toddlers are also often angry when someone takes something they believe to be theirs. And toddlers often think the whole world is theirs and theirs alone and everyone is here for their amusement .. and if reality tells them different, they can get very mean, too. So - I think the analogue holds somewhat.

Covenant71|5 years ago

Most of the people responsible for starting the journey have left the local government. I think the big problems are - the fact that they have been fined for not handling personal information in a correct way combined with - not enough technical knowledge to know what their current solution enables - the scope of the problems with the system are so big that they do not know what to do and they feel like they get attacked from everywhere and do not have the professionalism to see the bigger picture - they can't afford to scrap the project and start from the beginning

nix23|5 years ago

>This reeks corruption.

Absolutely correct, it's smells exactly like the kill-LiMux "Project" from Munich.

But Sweden has a pretty big history being corrupt, just think about Assange, Olof Palme...and the bad russian sub story.

xrd|5 years ago

If open source advocates banded together and "sold" this story to local governments, there would never be these kinds of boondoggles. I'm reminded of the same thing that happened in Oregon. Oracle came in with a low ball price, then extracted hundreds of millions of dollars out of the state for a POS health care system.

https://www.oregonlive.com/politics/2016/09/post_183.html

airhead969|5 years ago

That's how BDUF software consulting works. Design something unusable but make it cost as many billable hours as possible. That way, it has to be thrown away and it's job security for the consulting industry to keep a desperate customer throwing money at non-solutions.

A better approach is agile with customer's actual employees guiding small changes all along from the beginning, and customer's project managers/budgeting managers tracking progress, requirements, and costs.

duxup|5 years ago

Is open source really a place you would find what I imagine is an enterprise 'do everything' education related IT application?

I like the idea but that's not a product I think open source folks ventures to build much?

trhway|5 years ago

>The city's local government, spent 1 billion Swedish crowns (100 million dollars) over 7 years to build a massive do-everything school IT platform, which students, teachers and parents are forced to use. And their app is awful. Slow, buggy, almost unusable. One parent managed to access other people's private data, leading to @Stockholmsstad being fined for bad security.

where is juicy details like who did it? Oracle? IBM? ..?

> So a few parents decided, since the data is basically their data, to build their own better version. A couple of months later, @oppnaskolplatt was ready.

Bad precedent that must be squished ruthlessly. Otherwise next time they would decide to have their own better roads, police, government... that slippery slop of "we the people".

marvin|5 years ago

In Scandinavia, contracting agencies are king. We've brewed up many local varieties of these, that are much smaller, probably a bit more skilled and generally don't deliberately take money from customers that clearly don't know what they're doing.

Directly employing most of one's required software engineers is largely a very new phenomenon, and not yet widespread.

In Norway, you'd have e.g. Evry, Itera, Bouvet, Miles, Computas as well as international companies like Steria, Accenture and CapGemini. They contract out developers at ~$125 an hour and pay a regular middle-class salary. A large portion of software engineers are employed at a company like this.

renewiltord|5 years ago

Government organizations are terrified about open source initiatives. There's something strange about it.

You could volunteer to do all the work and they'll still oppose you at every turn.

My hope is that we'll reach a stage where citizen participatory programming is normal for all. Where my dad could offer a PR to fix a typo on a government page casually as he browses it.

I have a feeling we're not far off but you need it to happen in a place with low entrenched interests but with sufficient enlightenment.

I think big US cities have the latter but not the former, and authoritarian developing nations lack both the former and the latter. So maybe smaller Western nations like Estonia.

Or, my biggest hope, sufficiently advanced townships in America.

callesgg|5 years ago

It is hard to blame other people when you are the only one to blame. When you decide to use a open source project it is your fault if it fails not the open source project, it often says so in the licensing.

While that is always true in reality(you are always responsible for your actions), it is not legally obvious when you buy commercial products(you can blame the manufacturer).

While people are often reluctant to accept that. I find that it is often what people’s arguments in this regards can be boiled down to.

progre|5 years ago

I don't know, when our libraries procured new state wide library software (each city had it's own before) they went with Koha (open source). This means that they are not really paying for development, only installation integrations and support. This opened up the bidding to much smaller players and was much cheaper over all.

alkonaut|5 years ago

It’s still “no one got fired for buying IBM” at play.

I can see why you’d be defensive and make a project together with the firm that is the biggest player in software for the public sector (although their reputation is poor).

What I don’t understand is why the project is a Big Bang release type thing. Or why the contract can’t have clauses about openness or interoperability? If the supplier is scared by that or charges more for it - switch. Having source visible or exposed APIs doesn’t mean they have to accept PRs (although that would be great PR)

rbjorklin|5 years ago

I don’t understand how it is allowed to use tax money for closed source projects in the first place.

globular-toast|5 years ago

It shouldn't be. The US has the right idea with public domain, but that concept doesn't really exist anywhere else in the world and, for some reason, doesn't really exist for software at all.

drno123|5 years ago

It is fascinating how those expensive IT companies that get enormous government contracts, with huge profit margins, always hire shitty programmers.

Lionga|5 years ago

I dont think the programmers are the problem. The problem are companies getting government contracts are good at exactly that getting government contracts. That mostly involves doing bribes without them beeing official corruption.

They are horseshit at anything after that

thrower123|5 years ago

Typically the way this works is that the A-listers are trotted out for the initial dog-and-pony show and prototype, then they move on to the next boondoggle and are backfilled by a rotating cast of Elbonians billing the same rate to flesh out the details.

And that's where things go sideways.

indymike|5 years ago

Behold the power of cost plus contracts.

askvictor|5 years ago

The government's system sounds just like the "Ultranet" (as it was comically named) in Victoria, Australia a few years back which similarly aimed to be the one system to rule them all in government schools. Eventually went to the scrapheap (along with a few bureaucrats who were charged for corruption). During the tender process a few of us were toying with putting in a tender using Moodle, but glad we didn't waste our time as it was stitched up from the start.

AussieWog93|5 years ago

Ctrl-F'd for that exact phrase. I remember that system being introduced - I was in high school at the time and my mum was a teacher. It was down for the entirety of the single government-mandated training day, and when it finally came up everyone could tell from a mile off that it was a steaming pile of shit. I don't think we ever used it after the initial session, ever.

hpkuarg|5 years ago

I wonder how Swedes maintain their reputation of having a well-run, corruption-free public sector?

yoelo|5 years ago

Corruption and inefficiency of this kind is a relatively new phenomenon, stemming from a regime of letting private contractors run the whole show. A fun fact is that Stockholm has one of the most expensive buildings in the world: Nya Karolinska, which has cost (converted from Swedish currency) around 2 billion dollars. The reason behind this is that construction was run as a public-private partnership. Meaning you shove loads of cash into private hands while you get none of the benefit.

alkonaut|5 years ago

Among other things, that we get angry in public when this happens. Hopefully that someone will see at least some consequence. That we are even surprised and upset about this I think is a good sign. I’m sure there are places where people would shrug even if the brother of the politician in question was running the firm that got the contract. Thankfully we have very little of that.

odiroot|5 years ago

Does any country have a corruption-free public sector?

wmf|5 years ago

This doesn't sound like corruption to me and does any government have good IT? Maybe Estonia?

gher-shyu3i|5 years ago

By ignorant media in the US that hand picks what fits the narrative.

cerved|5 years ago

Hanlon's razor. The problem with Swedish public financed IT, is a) the requirement by anti corruption legislation to choose the cheapest solution b) the lack of competence by the people in charge of writing public tendors

AdrianB1|5 years ago

Who said they do?

wrnr|5 years ago

This happens in every city in EU. Yesterday I was checking out the mandatory drone tracking solution of my city provided by a local supplier. Don't think it a network of radars connected to a surface-to-air-missile installation, no, a lame skin on google maps where you upload a flight plan based on a pdf you need to find for yourself on another website.

a3n|5 years ago

> You can understand why they arent keen to allow it to continue as its using BankID to authenticate. Would you install a 3rd party app to access your bank account?

Isn't that how Mint works?

https://en.m.wikipedia.org/wiki/Intuit_Mint

hugolundin|5 years ago

Additionally, that tweet is wrong. BankID is an authentication system, and won’t give the 3rd party access to your bank account

aronatom|5 years ago

This 3rd party app in question ( mobilt bankid) is also used to access everything in Sweden ( taxes, banks, social security applications .. )

lordnacho|5 years ago

Perhaps the thing to do is to offer up the government project to an open source initiative.

"We need a school comms platform. It needs to have messaging and scheduling. People need to be authenticated (duh)."

Now ordinarily I'd say "WTF who would build that for free?" but by the looks of it someone has done substantial work for free already.

Heck, you could probably get free work from the kids themselves. There's plenty of people in education who would want to do odd jobs on it.

Now maybe pay up for a few senior devs and a PM, so that someone is at least responsible for it, with their income tied to it. But make it a small group, for the same reason.

If there's suggestions, or something breaks, there's a place to report that. End of the day, it's a platform for the people by the people.

Sounds like a great way to get a community to build its own infrastructure?

dmurray|5 years ago

> Now ordinarily I'd say "WTF who would build that for free?" but by the looks of it someone has done substantial work for free already.

There's a Cunningham's Law parallel here: the best way to get a good free open source system is to first build a terrible expensive proprietary one with shady business practices, and let the frustrated users do the rest.

core-questions|5 years ago

I love this idea, but it seems that in general bureaucracies are not agile or freethinking enough to be willing to risk doing something like this, even though in the long run it might be amazing. Imagine if an easy-to-administer system of this ilk was freely available and was picked up by school districts around the world; that would be an incredible pool of talent to work with. Especially if the product can be a little bit fun and whimsical, being not-for-profit, compared to working on something dry and corporate.

Toutouxc|5 years ago

> Heck, you could probably get free work from the kids themselves. There's plenty of people in education who would want to do odd jobs on it.

> Now maybe pay up for a few senior devs and a PM.

> platform for the people by the people

I think you're overestimating the abilities of "kids" and non-professional devs and underestimating the complexity of running that kind of show (thus underestimating the ratio of professionals/community necessary to pull that off). Notice how most successful open-source projects are in fact supported by tech companies and worked on by professional devs on those companies' payrolls, and how, despite that, most open-source projects still end up kind of chaotic.

lampe3|5 years ago

Germans: what? you guys have an app?

Believe me its way worse here in Germany.

I would consider this a luxury problem...

Lionga|5 years ago

What? You guys build airports withouth 14 years of delay and 500% of the planned cost?

German government is the european leader in corruption , even covid was used by many of them to get kickbacks for masks for elders and wasted 2 Billion € there.

And like the corrupt "Flintenuschi" Ursula von der Leyen they get promoted to President of the European Commission for that if they manage to delete all evidence from their phone after beeing caught.

GauntletWizard|5 years ago

if only there were some name for being dependent on a group that has shown you only abuse...

ToFab123|5 years ago

Reminds me the old "war" between AOL Instant Messenger and MSN Messenger. I think I recall that one the same day there was around 50 updates to those programs. Each update breaking something from the other and then one blinked and the stream of updates stopped and the 2 apps no longer spoke to each other.

https://www.istartedsomething.com/20140420/msn-messenger-and...

gopaz|5 years ago

Not really sure what this platform contains, a backend, a site and an app?

It sounds like if I would get 1% of the price to build this, I would laugh all the way to the bank.

tibu|5 years ago

Exactly the same is happening in Hungary. There was a huge educational system developed for schools from extremely huge budget but both the backend and front-end is crap. There was an alternative front-end built similar to the Swedish way but they were forced to shut it down.

Sirikon|5 years ago

In Spain this is called a normal day

say_it_as_it_is|5 years ago

U.S. Government sweetheart deals with IBM aren't much better.

tu7001|5 years ago

Why such app at all, and how can gov may force people to use it?

hutzlibu|5 years ago

Mandatory education system?

waspight|5 years ago

Is there a possibility to use OTA updates with expo since you are already using react native? To quicker get the header fixes out?

flerp|5 years ago

Fantastic! Looks amazing as well

duxup|5 years ago

They paid to develop an entirely original school app to "do everything"?

Jebus....

brighton36|5 years ago

Government procurement is fundamentally broken, in the software world. The incentives of procurement seems to ensure that the worst software is produced, at a significantly greater cost than free alternatives. I blame this on the rise of administrative costs in the universities (which trickle into enterprise) and the inherent disagreeableness (with outsiders) amongst the price's law coalition.

908B64B197|5 years ago

[deleted]

stevenbedrick|5 years ago

Funny thing about that quote- it's taken from a longer speech that Reagan gave, in which he talked about how his government had "committed record amounts" of agricultural assistance:

> I think you all know that I've always felt the nine most terrifying words in the English language are: I'm from the Government, and I'm here to help. A great many of the current problems on the farm were caused by government-imposed embargoes and inflation, not to mention government's long history of conflicting and haphazard policies. Our ultimate goal, of course, is economic independence for agriculture, and through steps like the tax reform bill, we seek to return farming to real farmers. But until we make that transition, the Government must act compassionately and responsibly. In order to see farmers through these tough times, our administration has committed record amounts of assistance, spending more in this year alone than any previous administration spent during its entire tenure. No area of the budget, including defense, has grown as fast as our support for agriculture.

From this 1986 speech: https://www.reaganfoundation.org/media/128648/newsconference...

bjoli|5 years ago

This is unrelated, but how in the world can he be held in such high regard in the US? Over here (Europe) he was widely regarded as a clown during his presidency, and the whole Iran/contra deal, together with the pardons by Bush senior, didn't really do his legacy any favours.

Just a week ago a friend of mine (who does not share my views, and is generally positive of Reaganomics) said something along the lines of: "not only was Reagan a clown, he surrounded himself with crooks"(regarding Bush and Meese).

Can someone explain it like I'm 5?

hutzlibu|5 years ago

Well, but does this

I'm from [name of big corporation] and I'm here to help.”

really sound any better?

I guess any big entity with lots of power has the same mechanism. Arrogance, intransparency and ignoring or misunderstanding (or knowing better) of the real needs on the people on the base.

Twixes|5 years ago

IMO the problem here is exactly that the project was outsourced. Outsourcing is likely to lead to crap products and with clueless public funding the costs end up astronomical relative to the quality of the result. In-house software expertise could result in better quality (this budget could definitely afford a lot of great engineers) and more controllable costs.

vangelis|5 years ago

Reagan would love this public-private partnership, no? The government is too incompetent to do things in house obviously, so they hired contractors!

ta988|5 years ago

Some would say that government services are regulating themselves.

bsder|5 years ago

[deleted]

lenkite|5 years ago

This is clearly malicious, definitely illegal and should be brought to a court of law.

Frankly, I see jail time for the contractors doing this.

pjc50|5 years ago

An expert on Swedish law? Got a cite for what they might be breaking?

eCa|5 years ago

IIUC, the open source app uses an unofficial/private api to communicate with the city’s application backend? Regardless of anything leading to the open source app being built, I struggle to take their side here.

Looking at the code, it appears that they authenticate against the api? So it’s a third-party app using an api, against the first-party’s expressed wishes, to read and/or manipulate student data?[1]

Of course the city will defend against that.

[1] Correct me if I’m wrong here.

lolinder|5 years ago

I understand the same, but remember that this is a city government that built this app, not a private company. The app belongs, by rights, to the taxpayers who funded it, and for the city (or their contractors) to be actively working to prevent them from building a better version of it is clearly wrong. Publicly-funded applications should be required to be open source (barring national security concerns), but at the very least they shouldn't actively prevent open alternatives.

dvdhnt|5 years ago

I feel like this is no different than using multiple web browsers to access my own generated content.

The app uses your device, your electricity, and your credentials. The API is built using your tax money.

I don’t see a problem.

reader_mode|5 years ago

>Of course the city will defend against that.

Why would that be a given.

I can understand it in scenarios where it messes up a monetisation strategy - but this bein publicly funded it's the opposite of what I'd expect.

utrechtsouth|5 years ago

It looks like users use a different frontend than the one provided by the government. So it's users changing their own data.

So that leaves a defense against... their own users? Makes no sense.

Plus the tax payers funded the platform so it's not like the government has any legitimate interest in protecting the product itself like a private business might.

bsder|5 years ago

While I love the David vs Goliath here, I'm going to point out the thing that open source folks just love to fluff over:

Who is going to maintain that app over time?

Maintenance sucks and is expensive.

I'm going through this right now with a security system for a non-profit. The old system is open source and works--but it's 10 years dead.

So, they'd like to add these couple features. Who is going to develop that? Who is going to pay for that? What happens 10 years from now?

So, they can pay money for a commercial solution which is "Somebody Else's Problem(tm)" or they can go with a bespoke system that becomes their problem.

Maintenance is a cost that open source never accounts for.

lucb1e|5 years ago

Proprietary route:

input: 5+ years and 100 million dollars

output: crappy system

maintenance: still necessary, hire anyone but probably the expensive guys from before

The route that these parents demonstrated:

input: a few months and presumably a whole lot less money

output: something that seems to be well-liked

maintenance: still necessary, hire anyone

Regardless of "but who's going to maintain it", the benefits should be clear here. People don't work on open source code because they get paid to and shrug when they do something useless or even detrimental just because the boss says so. They work on it for a passion. Now if you hire a company to write the open source code, you kind of lose that benefit, but if the development is out in the open, the public can at least keep track of it and say "but this doesn't make sense" or "let's get a working system before we spend another 4 years over-engineering and bloating this". This open model is how the corona tracker was developed in the Netherlands and it worked super well. The question is now whether the government will dare to do it again with the next IT project.

detaro|5 years ago

Of course open source also needs maintenance, but maintenance isn't suddenly cheaper because you bought a big commercial solution. The difference is that for open-source, you are not reliant on the original provider to maintain it, but can pick from a large range of open-source dev shops and consultancies to help as long as the size of the system is tractable. (or if it's a big thing, hire your own people and bring in external support as needed)

viraptor|5 years ago

> Who is going to maintain that app over time?

This is not a "current proprietary system" vs "open system maintained by volunteers". They can basically adopt the open system which is better and pay whoever they want the same account of money to maintain it. They just need to shift which system is being maintained. (And may even save money that way)

mannerheim|5 years ago

Yes, free software projects can stop being maintained. But proprietary products can go out of business. Or they might shut the service down (witness the Google Graveyard). At least with free software you can hire someone else to maintain it if it stops being maintained.