top | item 36095583

(no title)

PaulAJ | 2 years ago

Nothing specific, but I've seen some big systems from the inside, and I know the kind of thing that leads to failures:

* Back in the 60s they got a big IBM computer to do some stuff. Then later on they needed to do other stuff. The old computer was too expensive and difficult to replace, so they got a new VAX or something to do the new stuff and talk to the old mainframe. Then some PCs got added to do more stuff, and so on. Today the back end consists of many different systems of different ages all talking to each other using different protocols that were designed against different requirements. Newer systems are forever being patched and updated to cope with new requirements, while the code for old requirements lurks waiting to be accidentally reactivated. Each of these systems has its own specialists for care and feeding, but nobody fully understands the whole thing. When something goes down there are not many people who can diagnose the fault and get it back up.

* Government contracts have lots of rules around them to ensure value for money and prevent corruption (see the UK COVID PPE fiasco for what happens when you try to cut these rules out). But the size and complexity make even bidding for a big contract very expensive and complicated, so it tends to be the preserve of a few big companies who chose to specialise in it. Their core competence is winning these contracts, not delivering on them later.

* These rules mean that everything has to be specified in detail up front, so that everybody knows what is supposed to happen. But this makes the whole thing horribly inflexible. As new requirements emerge from the woodwork there is a continuous process of renegotiation.

* The UK civil service is based around the "cult of the gifted amateur". Senior managers are rotated around departments every few years. So the person who kicks off a project is rarely the person who sees it through. Everybody gets to blame someone else for failure.

* When one of these big contractors fails to deliver, the Government has to chose between suing to get their money back (or some of it) in a few years, or getting the at least part of the system they actually need at a higher price. The government doesn't need the money, it needs the system. So the contractor gets to carry on regardless of failure.

* Humans are very bad at managing small risks with large consequences. Many big disaster stories have at their heart someone who decided that the risk was too small to be bothered with.

discuss

order

No comments yet.