This is why I love git (and distributed version control systems in general). For the most part a short downtime isn't the end of the world. When it comes back up I'll push my changes and that'll be that.
While I actually agree with you, as I look at GitHub as "somewhat reasonable git hosting", that is not how many (if not even most) projects actually use the website, so it is an unfair analysis: people seriously often use (and even encourage others to use) GitHub as the only web presence for their project, using a Readme.md to build a landing page; meanwhile, for many people, GitHub is how they handle incoming bug reports and pull requests (as they believe, for right or for wrong, that it is easier to use GitHub to collaborate on these features than a more traditional mailing list).
You thereby can't just say "oh yeah, git is distributed, so I don't care if GitHub is offline"; it makes about as much sense as saying "I don't care that Gmail is offline, as email is distributed: I have all my messages locally in Outlook and can send mail through my ISP" while ignoring that other people now can't send you mail and you have lost access to your calendar entirely.
Exactly. To expand on this point, I love the redundancy built into Git.
I had a clone of my personal fork of my department's github repository at home. I read this and thought "that's interesting, I wonder if I could still work on it from home if I want to." I saw with "git config -l" that I had my github repository and my university's github repository, but not my university's mirror of the repository. I did a "git add remote", logged in to my university's VPN, "git fetch", and was up and running with my latest changes from Friday.
If that didn't make any sense, well, the gist of it is that the outage wouldn't affect development work in the slightest, thanks to the fact that every repository is a) content addressable, so you have consistency between servers and b) self-contained, so you have a complete copy everywhere. Hit me up for more info if you're curious; I recently upgraded us from CVS to Git for this redundancy advantage, among others.
I would point out that both TFS and Mercurial do the same thing (as does ClearCase, but ClearCase is awwwful), but with much less memory consumption on the local PC. Not trying to add snark (I upvoted the above comment), just adding that it's not a GitHub feature.
I'm amazed at how much this has knocked me on my arse.
I first attempted to redo the README for a service I've just open-sourced, before realising Github is down.
Then, I attempted to fix the company CI server (OOM errors because of Carrierwave needing more than 500MB of memory to run 1 spec in, for some unknown reason), which failed because it couldn't check out the code.
After giving up on that, I attempted to install Graphite to a company server, where I hit another roadblock because the downloads are hosted on Github, and so I had to use Launchpad, which I had an allergic reaction to.
Also, when I was shelling into the server, oh-my-zsh failed to update because, you guessed it, Github was down.
Still, shouts to the ops team in the trenches, we're rooting for you.
Your company runs its source revisions through Github without a backup solution? Do you really put all your eggs in a basket you have no control over?
I know that in theory a cloud solution should have a higher uptime than an amateuristic set up private server, but cloud solutions have a certain complexity and coherence that make them very vulnerable to these kinds of 'full failures' where nothing is in your control.
Maybe you should take this time to learn from this, and analyze what you could do to reduce the impact of this failure. For example, you could research what it would take for your company to move to another Git provider, perhaps even on your own server or a VM slice at some cloud provider.
I'm not saying you should drop github, because obviously they have great service, but be realistic about cloud service.
Cloud service is like RAID: it is not a backup.
The way RAID is nice for recovering from errors without downtime, there is a chance something bigger happens and you still lose your data cloud is nice for offering scalability and availability but there's a chance everything goes down and you still can't run your operations.
I did a bunch of work on GitHub today just before it went down. Talk about getting lucky.
I'm sure we will all learn a bunch from the post-mortem. These high-profile and very openly discussed failures are always good for learning all kinds of things.
No issues at all on how GitHub is handling it so far. Eager to learn what happened. Hitting refresh on the status page every so often. Better than watching underwater basket-weaving competition at the olympics.
I created a new org on GitHub today (through all the unicorns) and was about to push an existing repo to them so we could all start pulling it. Talk about unlucky.
Well handled, minus the unicorns. Tangentially relevant: I just wish Github offered some sort of an academic plan for students, no private repo means that I cannot use Github at all not because I'm building closed-source software, but because I (obviously) can't put my assignment work for public viewing before the assignment deadline. So I've been using Bitbucket, which is fine and all, but I would have loved to be a part of Github community.
If you want free Git hosting, you can DIY with Gitlab [1] if you want a web-based GUI like Github's.
If you prefer command-line usage, I'd recommend Gitolite [2]. It allows you to give people access to git and only git (as opposed to git's built-in system, which requires granting ssh shell access); and it only uses one OS-level user/group regardless of how many people it's managing.
Either of the above solutions are for your compsci professors who are clued-in enough to be comfortable with CLI in general and Git in particular.
If you're trying to give files to technically clueless humanities professors, I'd suggest only using Git privately, to develop your paper or whatever, then using a plain old email attachment, or hosting on an HTTP server, to submit the assignment. Or going really old-school by printing out an old-fashioned dead-tree hardcopy.
Of course, all of these solutions (except email attachments and printouts) require running your own server, which is actually a great learning experience. I'd recommend prgmr.com for hosting; their smallest plans should be able to fit even an undergrad's budget, and you have full root access to your (Xen VM) system, so you can do all kinds of fun and exotic experiments. It's not necessary for basic usage, but you can install any version of any Linux distro, use LVM, even use a custom-compiled kernel or FreeBSD (the only requirement is guest Xen patches). It's great because if you have problems, they give you access to an ssh-based out-of-band console, rebooter, and rescue image so you can fix them yourself. (By contrast, many other hosts require you to make changes through some half-baked web UI that lacks half the tools you need, require you to install only approved distros and only do OS upgrades on an approved schedule, and require you to file tickets with lengthy turnaround times and/or fees in order to do the most routine troubleshooting or maintenance tasks.)
Disclaimer: My only relationship with prgmr.com is that I've been their hosting customer for a long time (and very happy with them given the nonsense I've had to put up with from other hosts, in case you couldn't figure that part out from my above rant).
My only relationship with Gitolite is a project user. (I've created and maintained three small-scale Gitolite installations.)
I haven't used Gitlab, but I've heard good things about it.
They do offer a free plan for students. I get 5 free private repos.
I can't get the link now since GitHub is down ha, but when it's back up, just have a quick google and you'll find a quick form to fill in and voila. It's great.
BitBucket. It's essentially the same service as GitHub, but allows private repositories for free. And with an academic .edu email you get unlimited collaborators. Much simpler than running your own git server as others have suggested I think.
Github does have academic plans. When I signed up, it was 2 years of a free Micro plan (5 private repos) if you signed up with a verified school email address. When they're back up, check out: https://github.com/edu
GitHub do provide Edu accounts, I have one! I guess I went to github.com/edu or /education or something but it gave me the form at /contact, you say what your up to, where you go and such and they put a coupon on your account to give you a free micro plan.
It should be pretty far down their list of priorities at this point, but I just noticed the "Exception Percentage" value at https://status.github.com/graphs/past_day is saying 483.704%. The fact that they're measuring this in percentages implies the maximum is 100%, but this isn't so.
I imagine it was a percent measured against successful responses. The exceptions now far exceed those successful responses. You are correct in saying it should be 100% but people have a habit of scaling beyond it.
I'm really curious how much longer github can offer so much free service. It's not just git but effectively free web hosting as well, at least for statically served pages.
It seems like it's only a matter of time before something will have to give. Either they'll have to start throttling web serving or cover the site in ads like sourceforge or something
I guess I'd better sign up and start paying ASAP to help be part of the solution
I don't think GitHub's hosting of free repositories costs them that much. As you said yourself, it is just essentially free hosting so the largest cost will probably be storage (which is pretty cheap anyway now) and then the much minor cost of Push/Pull/Clone etc bandwidth and processing.
Think of it less as a free service and think of it more as marketing. I have a few free repos which I used to open source some of my personal modules; however when it came time to select a source revisioning service for work, I remembered how easy GitHub was and went with their payed service. I am fairly sure us now paying for our repositories offset itself and my and quite a few other free repositories.
What's funny is the extent to which Github is being used as a centralized repository for many projects. I don't just mean for project discovery; the issues and gists and other services aren't replicated as often or easily as the code.
In fact, a lot of services depend on github for various reasons, all of which are probably borked now ...
It's easy enough to set up a backup repo so a team can keep collaborating when Github is down, but does anyone have a way to deploy Rails apps with Capistrano+bundler? It's terrible not being able to deploy; sometimes that can be really urgent.
With Cap I can just repoint config/deploy.rb to the backup git repo, but what about bundler?
Of course if you have private patches of gems with bundler's `:git` option, you'd need to repoint all of those, too (as well as keep them in two places).
It'd be awesome if there was some simpler way, maybe a separate declaration in your Gemfile so that any gems added to your project also get installed on a corporate git server, and then bundler uses that as a fallback if `source` doesn't work.
I feel like such an uber goober! I was installing a package that relied on a github file, which for obvious reasons failed....little did i realize that that was the problem, all I saw was such and such python error callback....doh!
Most of developers in my countries 're using SVN. I don't use SVN at all, and i'm failing to convince my partner that Git is better than SVN, just because i don't believe in hardware stability.
I never forget to pull before going remote, and today I did, and the one time I actually need github to be up and it's not. But I can't be upset, even if this wasn't my fault, it's github.
[+] [-] jtchang|13 years ago|reply
[+] [-] saurik|13 years ago|reply
You thereby can't just say "oh yeah, git is distributed, so I don't care if GitHub is offline"; it makes about as much sense as saying "I don't care that Gmail is offline, as email is distributed: I have all my messages locally in Outlook and can send mail through my ISP" while ignoring that other people now can't send you mail and you have lost access to your calendar entirely.
[+] [-] drunkpotato|13 years ago|reply
I had a clone of my personal fork of my department's github repository at home. I read this and thought "that's interesting, I wonder if I could still work on it from home if I want to." I saw with "git config -l" that I had my github repository and my university's github repository, but not my university's mirror of the repository. I did a "git add remote", logged in to my university's VPN, "git fetch", and was up and running with my latest changes from Friday.
If that didn't make any sense, well, the gist of it is that the outage wouldn't affect development work in the slightest, thanks to the fact that every repository is a) content addressable, so you have consistency between servers and b) self-contained, so you have a complete copy everywhere. Hit me up for more info if you're curious; I recently upgraded us from CVS to Git for this redundancy advantage, among others.
[+] [-] callahad|13 years ago|reply
(Though I do wish GitHub Issues weren't as centralized...)
[+] [-] tkellogg|13 years ago|reply
Unless the Mayans were off by one
[+] [-] shoopy|13 years ago|reply
[+] [-] silverbax88|13 years ago|reply
[+] [-] rubynerd|13 years ago|reply
I first attempted to redo the README for a service I've just open-sourced, before realising Github is down.
Then, I attempted to fix the company CI server (OOM errors because of Carrierwave needing more than 500MB of memory to run 1 spec in, for some unknown reason), which failed because it couldn't check out the code.
After giving up on that, I attempted to install Graphite to a company server, where I hit another roadblock because the downloads are hosted on Github, and so I had to use Launchpad, which I had an allergic reaction to.
Also, when I was shelling into the server, oh-my-zsh failed to update because, you guessed it, Github was down.
Still, shouts to the ops team in the trenches, we're rooting for you.
[+] [-] tinco|13 years ago|reply
I know that in theory a cloud solution should have a higher uptime than an amateuristic set up private server, but cloud solutions have a certain complexity and coherence that make them very vulnerable to these kinds of 'full failures' where nothing is in your control.
Maybe you should take this time to learn from this, and analyze what you could do to reduce the impact of this failure. For example, you could research what it would take for your company to move to another Git provider, perhaps even on your own server or a VM slice at some cloud provider.
I'm not saying you should drop github, because obviously they have great service, but be realistic about cloud service.
Cloud service is like RAID: it is not a backup.
The way RAID is nice for recovering from errors without downtime, there is a chance something bigger happens and you still lose your data cloud is nice for offering scalability and availability but there's a chance everything goes down and you still can't run your operations.
[+] [-] robomartin|13 years ago|reply
I'm sure we will all learn a bunch from the post-mortem. These high-profile and very openly discussed failures are always good for learning all kinds of things.
No issues at all on how GitHub is handling it so far. Eager to learn what happened. Hitting refresh on the status page every so often. Better than watching underwater basket-weaving competition at the olympics.
[+] [-] syncsynchalt|13 years ago|reply
[+] [-] rolleiflex|13 years ago|reply
[+] [-] csense|13 years ago|reply
If you prefer command-line usage, I'd recommend Gitolite [2]. It allows you to give people access to git and only git (as opposed to git's built-in system, which requires granting ssh shell access); and it only uses one OS-level user/group regardless of how many people it's managing.
Either of the above solutions are for your compsci professors who are clued-in enough to be comfortable with CLI in general and Git in particular.
If you're trying to give files to technically clueless humanities professors, I'd suggest only using Git privately, to develop your paper or whatever, then using a plain old email attachment, or hosting on an HTTP server, to submit the assignment. Or going really old-school by printing out an old-fashioned dead-tree hardcopy.
Of course, all of these solutions (except email attachments and printouts) require running your own server, which is actually a great learning experience. I'd recommend prgmr.com for hosting; their smallest plans should be able to fit even an undergrad's budget, and you have full root access to your (Xen VM) system, so you can do all kinds of fun and exotic experiments. It's not necessary for basic usage, but you can install any version of any Linux distro, use LVM, even use a custom-compiled kernel or FreeBSD (the only requirement is guest Xen patches). It's great because if you have problems, they give you access to an ssh-based out-of-band console, rebooter, and rescue image so you can fix them yourself. (By contrast, many other hosts require you to make changes through some half-baked web UI that lacks half the tools you need, require you to install only approved distros and only do OS upgrades on an approved schedule, and require you to file tickets with lengthy turnaround times and/or fees in order to do the most routine troubleshooting or maintenance tasks.)
Disclaimer: My only relationship with prgmr.com is that I've been their hosting customer for a long time (and very happy with them given the nonsense I've had to put up with from other hosts, in case you couldn't figure that part out from my above rant).
My only relationship with Gitolite is a project user. (I've created and maintained three small-scale Gitolite installations.)
I haven't used Gitlab, but I've heard good things about it.
[1] http://news.ycombinator.com/item?id=4957145
[2] https://github.com/sitaramc/gitolite
[+] [-] experiment0|13 years ago|reply
I can't get the link now since GitHub is down ha, but when it's back up, just have a quick google and you'll find a quick form to fill in and voila. It's great.
[+] [-] rtkwe|13 years ago|reply
[+] [-] pcerioli|13 years ago|reply
[+] [-] psobot|13 years ago|reply
[+] [-] gst|13 years ago|reply
[+] [-] bencevans|13 years ago|reply
[+] [-] ricardobeat|13 years ago|reply
You can also commit locally, and only push after the deadline.
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] csense|13 years ago|reply
[+] [-] chuhnk|13 years ago|reply
[+] [-] greggman|13 years ago|reply
It seems like it's only a matter of time before something will have to give. Either they'll have to start throttling web serving or cover the site in ads like sourceforge or something
I guess I'd better sign up and start paying ASAP to help be part of the solution
[+] [-] robin_reala|13 years ago|reply
[+] [-] jamescun|13 years ago|reply
Think of it less as a free service and think of it more as marketing. I have a few free repos which I used to open source some of my personal modules; however when it came time to select a source revisioning service for work, I remembered how easy GitHub was and went with their payed service. I am fairly sure us now paying for our repositories offset itself and my and quite a few other free repositories.
[+] [-] kcbanner|13 years ago|reply
[+] [-] niggler|13 years ago|reply
In fact, a lot of services depend on github for various reasons, all of which are probably borked now ...
[+] [-] experiment0|13 years ago|reply
https://status.github.com/messages
[+] [-] zemanel|13 years ago|reply
Was just getting my hands on Homebrew after a fresh OS install when i hit the Octobummer :-/
[+] [-] zalew|13 years ago|reply
//edit after tinco's comment
+refs/heads/:refs/remotes/origin/ -> +refs/heads/:refs/remotes/bitbucket/
[+] [-] ricardobeat|13 years ago|reply
[+] [-] pjungwir|13 years ago|reply
With Cap I can just repoint config/deploy.rb to the backup git repo, but what about bundler?
[+] [-] pjungwir|13 years ago|reply
It'd be awesome if there was some simpler way, maybe a separate declaration in your Gemfile so that any gems added to your project also get installed on a corporate git server, and then bundler uses that as a fallback if `source` doesn't work.
[+] [-] tronicron|13 years ago|reply
http://www.verbosity.ca/hosting-your-own-git-based-shared-re...
[+] [-] vacipr|13 years ago|reply
[+] [-] timothybone|13 years ago|reply
[+] [-] revskill|13 years ago|reply
[+] [-] kcbanner|13 years ago|reply
[+] [-] johnny22|13 years ago|reply
[+] [-] darkchasma|13 years ago|reply
[+] [-] fomojola|13 years ago|reply
[+] [-] buttscicles|13 years ago|reply
Seems to be fine now though.
[+] [-] tzaman|13 years ago|reply
[+] [-] rossjudson|13 years ago|reply
Looks like services implemented in terms of github are not reliable (but there was no guarantee of that anyway).
[+] [-] cjbprime|13 years ago|reply
[+] [-] pcerioli|13 years ago|reply