oilman's comments

In 99.99% of real-world scenarios, the rig would have other options to bootstrap a black start—like fully charged air tanks, backup power from a support vessel, or even emergency battery systems. The hand-cranked air compressor is really a last resort tool. We test it during commissioning to prove it could work, but in most cases, it’s never used again in the rig’s working life. It’s there for the rarest situations—like if a rig was abandoned during a hurricane, drifted off station, and someone somehow ended up back onboard without normal support. It’s a true "everything else failed" kind of backup.

In another life I worked as an engineer commissioning oil rigs and I’ve seen how tricky even a small-scale black start can be. On a rig, we simulate total power loss and have to hand-crank a tiny air compressor just to start a small emergency generator, which then powers the compressors needed to fire up the big ~7MW main generators. It's a delicate chain reaction — and that's just for one isolated platform.

A full grid black start is orders of magnitude more complex. You’re not just reviving one machine — you’re trying to bring back entire islands of infrastructure, synchronize them perfectly, and pray nothing trips out along the way. Watching a rig wake up is impressive. Restarting a whole country’s grid is heroic.

As an engineer who has spent time commissioning electrical systems on oil rigs, I've had the opportunity to witness some truly fascinating tests. However, out of all the tests I've seen, the black start test is by far the most intriguing.

Similar to the article, but at a smaller scale, it takes power to start up one of the ~7MW main generators on an oil rig. During commissioning, a black start test simulates a scenario where all power sources on the rig are exhausted, and the emergency backup systems must be used to bring the rig back to life.

Starting the main generators on an oil rig is no easy feat. It takes a significant amount of power to get these massive machines running, and it's not as simple as pulling a cord like you would with a lawnmower. In normal circumstances, the rig uses tanks of compressed air to start the generators, but during a black start test, these tanks are assumed to be empty.

So, how do you start the main generators in this situation? The answer is with a special emergency hand-cranked air compressor. By cranking (and cranking, and cranking) this compressor, you can generate enough air to start a small air compressor, which in turn is used to pump up the air tank and start the 1.5MW emergency generator. Once the emergency generator is running, it can be used to power the compressors that fill the large tanks needed to start the main generators.

Watching this process unfold is truly a unique experience. To see a massive oil rig slowly come to life, all thanks to one person cranking away at a small air compressor, is truly impressive.

This seems like great advice, and it resonates with me. I've seen people make a big impact by following this path.

> It’s ok to spend some of your time on snacks to keep yourself motivated between bigger accomplishments, but you have to keep yourself honest about how much time you’re spending on high-impact work versus low-impact work. In senior roles, you’re more likely to self-determine your work and if you’re not deliberately tracking your work, it’s easy to catch yourself doing little to no high-impact work.

In my own personal experience, that boost of actually accomplishing something right now instead of slowly starting the process of impactfully pushing another rock up a hill is very tempting.

Does anyone have any experience or recommendations for effectively tracking your own work and putting yourself in the right headspace to tackle these more long lived impactful tasks? This mental game seems to me to be one of the huge factors that determine outcomes.

I tend to have some challenges with attention at the best of times. My interests tend to run hot and cold. I can make a huge impact and move a project significantly forward when I get into it and hyperfocus on it. But other times managing to focus my attention on a tasks that I know would be high impact is mental torture.

I was once working for a small company building electrical equipment. We mostly worked on "medium voltage" equipment, you know 2400 to 69000 VAC.

For one project we had large banks of ultracapacitor in a cabinet. Fully charged it was around 1200 VDC. This thing was in the prototyping stage, and we were testing a control system on a Saturday morning.

So we charge it using a large AC/DC converter, fully charged, everything worked beautifully. We start a discharge cycle converting the DC back to AC. Uh oh, it starts pulling way too much current. Flames start to shoot out of the AC/DC converter. Fuck. BANG. Fuse blown.

We assess the damage... the AC/DC unit is totally shot. And someone (me) is going to have to analyze what caused the failure. Otherwise everything with the capacitor cabinet seems okay, but the thing is still charged to 1090 VDC and the fuse is blown. Check with the mechanical engineer that designed the cabinet. Turns out the fuse can't be changed (can't be accessed) while the cabinet is charged and the cabinet can't be discharged because the fuse is blown. Well that isn't good.

The only thing we could do was discharge it into a load bank (think large toaster) by connecting something directly to the copper busbar live at 1090 VDC. So one of the commissioning guys volunteered. He put on some high voltage gloves, stood on a plastic mat, and connected some jumper cables someone had in their car to the bus bar. He stepped back and someone else threw the switch on the load bank and it discharged without incident.

There were some design revisions after that.

Perhaps the solution is more reporting instead of less reporting. Some sort of real time (or hourly, daily, whatever) metrics about a company instead of quarterly reports. I feel like a certain frequency makes it harder to game, and much more routine, so people aren't as likely to make decisions that are detrimental to the long term. I think faster reporting and quicker feedback loops are the way the world is going, why not for public markets?

If there are any of the moderators reading this, I would love to know what this post did wrong to get shuffled to the very bottom of the conversation so I don't do it again. There must be some mod magic involved as it has a number of up-votes.

A quote from a letter from Chairman Khrushchev to President Kennedy during the Cuban missile crisis.

I see, Mr. President, that you too are not devoid of a sense of anxiety for the fate of the world understanding, and of what war entails. What would a war give you? You are threatening us with war. But you well know that the very least which you would receive in reply would be that you would experience the same consequences as those which you sent us. And that must be clear to us, people invested with authority, trust, and responsibility. We must not succumb to intoxication and petty passions, regardless of whether elections are impending in this or that country, or not impending. These are all transient things, but if indeed war should break out, then it would not be in our power to stop it, for such is the logic of war. I have participated in two wars and know that war ends when it has rolled through cities and villages, everywhere sowing death and destruction.

I worry that the current nationalist trend that seems to be simultaneously happening in many countries is going to lead down the long path to war. I don't see many leaders today that I think would have the diplomatic resolve to write a letter like this. I feel people are interpreting others actions in the least charitable way which causes all kinds of rifts. The deeper these rifts grow, the more likely we are to see someone ignite the spark of war. I honestly don't know the reasons, or if maybe its just my perception that has changed as I've gotten older.

In a lot of industrial sites software security is a joke. Embedded systems tend to use very old, well proven technology, which in itself isn't a problem, it fits the market well, but the side effect is that security isn't always properly considered as it wasn't a concern when the software/hardware was developed.

I was involved in a project a few years ago delivering a series of monitoring systems running Windows XP to a brand new 700 million dollar oil rig. This was at the request of the client, they had software they needed that would only run on Windows XP. They had a fit when we had trouble sourcing Windows XP licenses. The expectation is that these systems will have a 20 - 30 year life.

It used to be that keeping every air gapped was enough, but organizations want easier monitoring, so more systems are being networked in an ad-hoc way without a lot of thought about security.

I expect we are going to see more things like this happening in the future until we start taking security in systems / embedded space more seriously. And even then there will be exploits of older systems for years afterwords since the replacement cycle is so long.

I wonder what a secure embedded system even looks like when I think about it. The environment isn't suitable to the kind of continuous patching that is done in the web world, but exploits will be found and dependencies will need to be updated. How do you square keeping things up to date with stringent testing requirements in systems that can kill people. Many of these systems / plants are unique, there is only one plant like it in the world, so testing becomes very hard.

The attitude of those in the oil industry does not help this at all. I work in oil and talking politics to those I work with just makes me sad. There is a lot of anti-government hate. People are still resentful about the National Energy Program and that ended 30+ years ago. Its kind of a mess.

I think its the notion that all the provinces are competing against each other, and that under any oil program some provinces will win and other lose that really holds us back. No one can see that we can all win by collectively working together. It doesn't help at all that there are now moneyed corporate interested involved that are not interested in seeing any changes take place.

I must be a bit dense today, what irony?

This sounds A LOT like myself. I've never really considered going to a psychiatrist or neurologist, I thought my experience was somewhat normal. Do you mind elaborating some about what sort of medication/condition are you talking about?

I have personal experience with (4). I work for a company that creates systems that are used on oil rigs. I came about this job in a round about way. I used to work in SF at a big name tech company, but I'm from a small town with no tech community whatsoever. My parents got sick, and I wanted to live closer, so I made the move home to support them. Found this job though connections and here I am.

The systems we create are complicated and require automation (PLCs, embedded systems, PCs) to function, so the company requires programmers. The money is fantastic for two reasons: they have a really hard time finding people and money for oil companies shoots out of the ground (most of the time).

The work is different from most systems someone would deal with at a startup. The systems range in age from new to 20 years old. Object oriented is a new and scary thing. I write and debug a lot of pascal and C. Also do a lot of sysadmin work on ancient systems, token ring networks etc.

The biggest difference however is work environment. Most rigs have an absolutely awful (think dial up) satellite internet connection, so there is no remote work. Everything gets done on site. This leads to a lot of travel, which is where you make the big money if you are willing to do it. The work environment is a lot different from a startup: Folding chairs and a table in the corner of a room if you are lucky, sitting on the floor if not. You get to see some interesting and scary parts of the world, police escorts from the airport, body guards on the way to the rig in really bad places, which can be very isolating as well as traveling for months at a time.