top | item 42468745

(no title)

asciimike | 1 year ago

Not sure why it's getting downvoted, definitely experienced Code Yellows, Code Reds and Code Purples at Google. Red is (obviously) worse than Yellow and IIRC was a total code freeze for a period of time. IIRC there was a Code Red around memory at some point because the supply chain was literally so backed up that google couldn't get enough DIMMs and reasonably sized services had to stop deploying because there wasn't enough compute capacity.

Purple is/was "developer experience is so bad we need to stop developing new functionality and make the current functionality usable."

discuss

order

vitus|1 year ago

The rough hierarchy today:

- Code red: the situation is actively causing active business harm.

- Code yellow: the situation will cause irreparable business harm if not addressed in the next 3-6 months.

- Code purple: the situation will cause business harm if not addressed in the next year.

- Code green: things are not at risk of causing problems, but we still want to make sure we make progress.

At Google, all of these priority codes need senior VP signoff, which is to say that it is actually an existential threat to one of our main product areas (e.g. Search).

I only remember the RAM crunch (2020 or maybe 2021) being a code yellow, but it's possible it was downgraded after the first month or two.

Code reds don't always have to be met with a total code freeze, but they generally do preempt all work outside of incident response.

The point of a code yellow should not be to punish the team, and an appropriately-declared code yellow should be met with significant introspection from leadership about how we got into this mess and what we need to do to prevent us from getting into this mess in the future. It's a blunt tool that allows the organization to dictate that it's going to drop its existing commitments on the floor because they are simply less important than fixing the systemic problem.

You don't need a code yellow to try multiple things in parallel, or to ship a prototype without worrying about scale. A startup certainly doesn't need a code yellow to empower individuals to wear multiple hats. And if your team is spending 50-75% of its time on keep-the-lights-on work, then your systems are being held together by duct tape, and this is simply not sustainable.