GitHub Outage

[mre]

Just noticed issues with Github handling requests and overall flakiness. Getting a lot of status code 500 errors and decided to open this thread for status updates.

[omn1]

Even though the status page (https://www.githubstatus.com/) shows no issues, I'm still getting the occasional 500. It seems to be happening quite irregularly. They are possibly facing a lot of load.

[adamscybot]

In my experience its often the case the status site does not reflect reality -- until someone intervenes.

[rozenmd]

I monitor GitHub externally here: https://github.onlineornot.com/

Seems like a huge spike in load.

[bowsamic]

Status pages are absolutely useless. I've never seen them accurately reflect an outage

[MattIPv4]

Been running into many unicorns for the last few minutes, had a moment where it came back but seems to be down again. Even the unicorn image won't load on the unicorn page.

[omn1]

Which is weird because the unicorn is an inlined image (png), encoded in base64. Seems like they broke it.

[qwertox]

I haven't pushed to GitHub in over a year. Now I'm setting up a new page on github.io with a new repo and GitHub goes 500 just when I try to push.

Those GitHub badges... they are as ugly as it gets.

[naikrovek]

well, I for one would like more unrequested critique of artwork on a code sharing website. ಠ_ಠ

[irajdeep]

> Those GitHub badges... they are as ugly as it gets.

Bingo. Not everything in this world needs to be gamified.

[blueflow]

Do we need to create a HN post for every outage? It happens every other week.

[capableweb]

In the beginning of status pages, most of them were automatic one way or another, or engineers quickly threw up "We know of the problem, stay tuned" messages there.

But soon after, legal/executive team got ownership of them apparently, and the status pages are no longer automatically showing downtime/response time and notice about when things are actually down can take a while.

So I think it's nice that there is at least one place where I can see if it's a problem on my end, or if it's global. It helps to remove some frustration at least.

[RcouF1uZ4gsC]

What else are you supposed to do when you can’t work because you don’t have access to your source code.

[corford]

Yep, noticed it with comments on an issue (had timeouts while submitting but it eventually went through).

Now 30 mins later, i've refreshed the issue and see that my reply and the comment I was replying too (by another user) are both gone. Hopefully, it's eventually consistent and these comments will re-appear later.

[kid64]

It's completely down for me. Status page says "all systems operational".

[fredrikaverpil]

The service seems very flaky right now. Even the unicorn isn't loading properly.

[sivapil]

Still getting this same error for past 10-12 hours. Tried in different times.

{ "code": 500, "message": "internal server error" }

Does anyone have luck? Any workaround to fix it?

[jmartens]

According to Metrist monitoring (disclosure: I work there), the errors were very rare, and didn't happen enough for us to call the product "down." Looks like around 1% of requests.

[WFHRenaissance]

I'm unicorning hard rn

[bilalq]

How reliable are Github cron action workflows? I set one up to run every 15 minutes recently, but it seems to actually be running closer to once an hour.

[view]

I'm trying to clone a repo at a whopping 6KB/s from Kenya.

EDIT: Seems to be a routing issue. I've enabled a UK VPN and it's working fine now.

[cherryblossom00]

‘No server is currently available to service your request. Sorry about that. Please try refreshing and contact us if the problem persists.’

[bityard]

Interesting how outages like this seem to happen mostly on Monday^w Tuesday mornings.

[HJain13]

Its Tuesday my man :)

[kleinsch]

If you have a risky commit, esp with a three day holiday weekend, you wait to land it until after the weekend.

[planxty]

it is tuesday my dudes :P

[max23_]

I get an error saying the action can't be performed when trying to star a repo.

[azeemh]

this is what happens when you sell important community infrastructure to M$FT

[dave4420]

This predates them getting bought by MS. GitHub was notoriously flaky from the very beginning.

[adamscybot]

Its back

[alexandremonjol]

Seems to be back right ?

[isusmelj]

Same issue. Site is also very non-responsive.

[tambourine_man]

They’ve been having issues since yesterday.

[jmartens]

I'd say since Friday night Pacific time.

[ohmahjong]

Time for some deep thumb-twiddling

[JSDevOps]

Yeah I noticed it earlier.

[planxty]

Seeing the same.

[maxcan]

Github outages are the bored engineer's equivalent of getting a surprise snow day when you were in school, full of unbridled joy.

For engaged, happy engineers its the equivalent of getting a surprise snow day when you are grown up and have to go dig your car out of the snow and its a normal day just with extra steps.

[ramigb]

I can be both bored and engaged, don't test me buddy /s

[lysergia]

> equivalent of getting a surprise snow day

Not if you self-host Git

[naikrovek]

Not really. I'd rather be getting stuff done. ¯\_(ツ)_/¯

edit: oopsie I misread.

[wheelerof4te]

You still can use git, you just can't push the code.

Not a huge problem, unless it lasts for hours or gasp, days.