top | item 39878103

(no title)

johnnyAghands | 1 year ago

Wow, mad jelly their CI/CD and monitoring proceses are robust enough to trust a major rollout in December. That's a pretty badass eng culture

That being said, still some unanswered questions:

- If the issue was ipv6 configuration breaking automated cert renewals for ipv4, wouldn't they have hit this like.. a long time ago? Did I miss something here?

- Why did this take 90 minutes to resolve? I know it's like a blog post and not a real post-mortem, but some kind of timeline would have been nice to include in the post.

- Why not move to DNS provider that natively supports ipv6s?

Also I'm curious if it's worth the overhead to have a dedicated domain for scripts/packages? Do other folks do this? (excluding third-parties like package repositories).

discuss

order

Thorrez|1 year ago

>- If the issue was ipv6 configuration breaking automated cert renewals for ipv4, wouldn't they have hit this like.. a long time ago? Did I miss something her

AIUI, they switched to their current setup 90 days prior to the outage. The initial cert they installed during their migration lasted 90 days. So 90 days after the migration, they had an outage.

PokestarFan|1 year ago

They're using Vercel, which lacks IPv6 support.