top | item 26949604

(no title)

yaml-ops-guy | 4 years ago

How do the conversations typically go when asked "did you know this could happen"/"did we see this coming" or has no one bothered to ask that question yet during the RCA process?

discuss

order

jcims|4 years ago

Generally not well.

If you haven’t been shot down by an expendable in the past, the best course of action is to find the most complementary position historically taken by leadership, twist to your objective and then credit them publicly with the vision.

The gap can be reconciled privately later.

lostcolony|4 years ago

"that no one is willing to prioritize"

You missed that part. "Here is the email chain it was discussed in, including the bit where I raised the concern of what could happen" is generally pretty solid CYA.

yaml-ops-guy|4 years ago

I actually did not miss that part, but thanks for calling it out regardless. The question was instead borne out of curiosity as to know how the follow up and subsequent discussions fare even with CYA in pocket.

What has your experience been? Have you had success reprioritizing necessary and critical fixes that were previously not in scope postmortem?

8note|4 years ago

Sounds like that's solved by having the list as an action item.

Yes, we knew about it. Also, here's all these other things we know about that will result in similar catastrophes

kaetemi|4 years ago

That's when you show the other hundred lists of solutions to things that could go wrong.

yaml-ops-guy|4 years ago

I see.

Maybe I'm overthinking it (this happens sometimes, I'm working on it), but waiting around for a security crisis to happen when a solution is in sight, even when facing difficulty prioritizing the work seems dangerous to one's career.

But I admit not knowing a lot of things most engineers just assume are a given, lately.