[eightnoteight]

> “Oscar, do you mind sharing your screen so Deepak and Deanna can see the weird log messages too?”

it seems so obvious from an Incident Commander perspective but so much goes into this workflow during an incident

* what if the person is a fresher, you are asking him to share screen, debug and perform actions in front of 100 people in the incident call and the anxiety that comes with it

* While IC has much more practice with handling fires continuously, for instance, if there is a fire every week in a 50-team organisation, a specific team would only be seeing their first incident once a year

* Self-consciousness/awareness instantly triggers a flight or fight response from even the most experienced folks

I don't know how other industries handle such a thing, I'm pretty sure even in non-tech there would be a hierarchy for the anomaly response and sometimes leaf level teams might be called to answer questions at top level of the incident response (like a forest fire response, might have a state wide response team and they pulling local response team and making them answer questions) probably they get much more time to prepare than in tech where its a matter of minutes

[chris_wot]

In a previous job, I had a critical incident crop up and we were dealing with the offshore parent company. All the senior management had been cc’ed into the emails about the problem.

Result: nobody was willing to say anything for fear of looking bad in front of those people. This was frustrating to say the least.

I solved this by replying all, but I took out all the senior people. I said something along the lines “hey guys, I’m the guy who needs this fixed. I can see you are la working hard. I’m removing a number of people from the cc lost and we will communicate with them in a seperate email. Just keep me up to date with how it’s going and tell me what you need from my end.”

This worked wonders. They worked the issue, and though it took some time it was to be expected.

When it was solved, I found the original email, replied all (including management) and explained that the problem was solved, and made a point of highlighting the excellent work the team fixing the problem had done on resolving the issue.

I never had any issues with the patent company’s dev team after that :-) in fact, they went through our incident reports and fixed 80% of the longstanding issues within the next week! Which I wasn’t expecting…

Moral of the story - take as much pressure off the incident team as you can.

[ChrisMarshallNY]

Thanks so much, that was good, practical, wise, in-real-life experience.

[throwanem]

> 100 people in the incident call

Well, there's your first problem...

[eightnoteight]

I took a high enough number to showcase the problem, for a fresher it doesn't change much even if that number is as low as 15 or 20, or even if 5 people that they don't know or at higher levels

also I feel like, the number of people that hop on the incident call are almost always related to the category of the incident, sure you can always break out to a separate room, but often the person would have already realised the impact and the weight of the incident

[ttymck]

Surely you'd want to instead share a link to the logs being investigated so others can investigate concurrently, instead of having 2 backseat drivers observe someone observing logs.

[brazzy]

Depends. In some situations it would in fact be better to have everyone discuss one person's shared screen, instead of having to constantly coordinate what they are talking about.

[c6400sc]

Ideally, wouldn't the IC's / Group of ICs' responsibility to introduce blameless culture before the incident, right?

I've worked in blameful places, always without ICs; just shouting HIPPOs.

I hope that an org evolved enough to create IC roles would back that up with culture, but I could be wrong.

[pjc50]

Indeed - in that kind of environment an important role is "managing upwards", preventing the people who are actually doing the work from being overwhelmed by constant requests for status and explanations.

[wil421]

What is a fresher?

[hibbelig]

Recently graduated, just entered the workforce.