top | item 38133297

We are investigating reports of degraded performance

116 points| contingencies | 2 years ago |githubstatus.com | reply

141 comments

order
[+] droptablemain|2 years ago|reply
My repos were coming up as 404s. I was wondering if I had been canned...
[+] __MatrixMan__|2 years ago|reply
The "nothing to see here" approach to access control has a lot of weird culture-consequences. I wish software would just address me like the peasant that I am, rather than trying to gaslight me into believing that my artificially limited world is the whole one.
[+] ellisv|2 years ago|reply
Not mutually exclusive - GitHub could be down AND you could be canned!
[+] dekhn|2 years ago|reply
Well, that explains a problem in an interview I was just administering (the interview code sample downloads a file from github).

Guess I can't treat github like a CDN.

[+] taf2|2 years ago|reply
Ah I was thinking it was southwest wifi!
[+] SoftTalker|2 years ago|reply
I like how "Every request returns a 404" == "degraded performance"
[+] agilob|2 years ago|reply
It should load faster if there's nothing to load
[+] CodeCompost|2 years ago|reply
Degraded in relation to 99.90÷ spread across the year.
[+] layer8|2 years ago|reply
At least it did return a response.
[+] WolfeReader|2 years ago|reply
I cannot wait for Gitea, Forgejo, and GitLab to start federating with each other via ActivityPub. Then we can all take one more step away from a corporate-controlled internet.
[+] perihelions|2 years ago|reply
- 'one more step away from a corporate-controlled internet"

Downvote me to grey-world if you like, but I think everyone's crazy to put all their code infrastructure in the hands of fucking Microsoft. Especially literal free open-source software. Who do you think Microsoft is? What do you know of Microsoft's history and their core values (they're "embrace, extinguish & exsanguinate"). It's like giving fucking Sauron safekeeping of your power-rings in Mordor, oh we have great infrastructure for safe ring storage here, very secure, the orcs are really expert guards.

[+] rglullis|2 years ago|reply
That would surely be nice and helpful, but why do we need to wait for it?

Even my open source projects in github are just mirrors from the "real place" of work: gitlab or my own gitea instance. If github is down, it is a minor inconvenience but I can still work.

[+] asdff|2 years ago|reply
Or just host your own git repo and cut out the dependency entirely.
[+] Eji1700|2 years ago|reply
Once again proving that Monday is "whenever you push to production"
[+] Animats|2 years ago|reply
Github is becoming too important. A routine backup system is needed, something like archive.org, that backs up important Github projects.
[+] swatcoder|2 years ago|reply
I know this a greybeard's fantasy and that most people working today were trained not to bother, but: important things should not have GitHub as a failure point.

Hobby projects and today's work? Sure. Point straight at GitHub and hack away. And when it goes down, get yourself a coffee.

But everything that's anywhere near production should have already pointed those github requests to a mirror or other tool in your own controlled ecosystem. The status of GitHub should have nothing to do with whether you're customers are getting what they're paying you for. Same goes for Docker containers and every other kind of remotely distributed dependency.

[+] hipadev23|2 years ago|reply
That exists on every dev’s machine.
[+] bogota|2 years ago|reply
Host a backup of your own code? It’s easy and can be done on a rpi. I wrote a go program in 1000 lines that automatically does this for me. And then I actually started using that as the main source and pushing the backup to github.

It also pulls down anything i star into a different folder which get a sync one a day. The rest get a sync every hour.

[+] antoineMoPa|2 years ago|reply
If only there was a decentralized system to allow version management of code.
[+] lmarcos|2 years ago|reply
In the context of companies: if you don't have your own cache of github,npm, etc., you're doing it wrong.
[+] shepherdjerred|2 years ago|reply
I assume that there are some archives of public GitHub repositories somewhere.
[+] rglullis|2 years ago|reply
I need a macro template for those memes with Bart Simpson on the blackboard and make it say "I will always have a backup plan for third-party services".

Seriously people: gitea exists. Gitlab self-hosted exists. Drone/Woodpecker CI exists. It's not that difficult to set up a project that does not depend on Github. I spent less time setting these up than the amount of down time that Github has had this year.

[+] waihtis|2 years ago|reply
Problem is, this is one of 50 things that can go wrong for which you should have a robust backup plan for
[+] verve_rat|2 years ago|reply
Or, just take the rest of the day off? The world will still be there after the weekend.
[+] andrelaszlo|2 years ago|reply

    ⮕ git push
    ERROR: Repository not found.
    fatal: Could not read from remote repository.
    
    Please make sure you have the correct access rights
    and the repository exists.
That scared me for a minute. ':D
[+] misnome|2 years ago|reply
Yeah, retriggered our deploy pipeline a few times, checked downdetector and no reports, logged in to machine and verified external connection.

Sighed, assumed I was on the first wave of an incident on top of their current slack incident, then logged off for the day.

[+] methodical|2 years ago|reply
PR really spun gold when they decided to label everything from every single one of their databases getting deleted and backups nuked to intermittent connectivity issues as "degraded". Who exactly are they making feel better by not calling a spade a spade?
[+] brookst|2 years ago|reply
People will take issue with any wording. Why try to find the exact right words each time when people will have exactly the same complaints?
[+] skeaker|2 years ago|reply
Shareholders, generally.
[+] __turbobrew__|2 years ago|reply
404 on all PRs -> degraded

My definition of degradation is different.

[+] edgyquant|2 years ago|reply
I would say it’s degraded. I still see I’m logged in, can still see the org screen and even the repo (on and off.). Don’t think degraded implies anything good. Just that something’s are working others aren’t
[+] yieldcrv|2 years ago|reply
well this was a wakeup call, we don't use github but apparently a bunch of bash scripts for package managers do

we couldn't onboard a new hire because they couldn't even run a curl command for some basic tooling

I thought it was a url, but the url actually loads a bash script that tries to pull from their github repo

[+] kklisura|2 years ago|reply
Regardless of this: does anyone experience general slowness of Github? I view a file (on web) and it takes time for page to be fully interactive - no buttons work, rest of the file cannot be viewed - just the top part is shown (above the fold, maybe). Honestly, it's so nerve wracking.
[+] rvba|2 years ago|reply
Question.

Recently github is pushing very agressively for two factor authenticartion.

So I installed the authenticator app.

But the authenticator does not work when the clock on my phone is not perfectly synchronized. But my phone's clock is intentionally sped up by +15 minutes?

What to do?

[+] MuffinFlavored|2 years ago|reply
Why can't they just roll it back /s

Who decided to deploy something risky on a Friday /s

Why don't they have lower environments like QA to test it in before going to production /s