GitHub Faces 'Major Service Outage' [Update]

Code repository GitHub is facing a major service outage, it said moments ago. Earlier today, the company said it was facing a minor service outage. The downtime comes less than two weeks after it was facing another "minor service outage," which lasted for several hours. The cause for today's disruption remains unknown. The open source company's Twitter feed suggests it has faced several issues over the past few months.

Update: GitHub reports all the services are now operational.

This is why I use GitLab

[Anonymous Coward]

Seriously, GitLab is a well-run, transparent company. They've had literally one major outage throughout the life of their service, and when it happened, they explained in incredible detail what had gone wrong, how they fixed it, and how they'll prevent in the future. I honestly can't believe they aren't more popular when this is the second big GitHub outage in, what, 8 weeks? Props to GNAA.

Re:This is why I use GitLab

[darkain]

GitLab also doesn't have the growing pains that GitHub faces at scale. When GitLab reaches that scale, either 1) they'll have the same issues, or 2) it'll have already been solved by other projects (like GitHub), and they would have learned from where others implemented fixes and new solutions already.

Re:This is why I use GitLab

[Junta]

For me, GitLab scales great in one respect: I can host a completely isolated instance of gitlab. So if it's an internet wide repository, I pretty much have an obligatory github mirror of it because that's the 'proper' thing to do. However when github goes down as it tends to do, it's not a huge deal because my team can still collaborate on our own instance of gitlab.

Re:

[Gr8Apes]

Yeah, I don't get this. Even if I were to host a public/private github repo, I'd still run and have the authoritative copy in house. It's not like it's a huge effort to run a copy either. (if you're really wanting to do it in house on the cheap, buy a Mac Mini and OSX Server - boom - instant repo in about 10 minutes, otherwise, get a decent Apple/Linux box and setup GitLab or something similar)

Build your own GitLab

[tepples]

GitLab scales great in one respect: I can host a completely isolated instance of gitlab.

Case in point: Build your own GitLab appliance with a Raspberry Pi [gitlab.com]. I wonder if anyone else has built an appliance to run Savane, the SourceForge fork powering the GNU Savannah repository farm.

Re:

[Actually, I do RTFA]

So, what you're saying is that I get to avoid all the growing pains of GitHub by sticking to GitLab, and free ride on GitHub's growing pains? Plus use a more open solution? What's the downside?

really?

[lastman71]

Maybe is down, because everyone is checking if is really down...

Re:

[93 Escort Wagon]

An interesting theory... DDOS a service provider by posting a story on Slashdot saying that service is down. Everyone here starts hammering the service, and - viola!

Re:

[grungeman]

No, that was in the old days when websites got "slashdotted" when the were mentioned on Slashdot. Those days are over, Slashdot is not really relevant anymore, which is a shame.

Re:

[kelemvor4]

Maybe is down, because everyone is checking if is really down...

Are they checking because it's down or is it down because they are checking. The world may never know.

Probably

[ArchieBunker]

because of systemd issues. Nah in all seriousness its probably the feds installing backdoors.

This is why git sucks.

[Anonymous Coward]

People who do real work trust Microsoft's Team Foundation Server to get their work done instead of stupid Linux git.

Re:

[Anonymous Coward]

I thought this was some kind of funny joke, but I looked it up, and Microsoft really has a product named Team Foundation Server--and it even embraces git! (Of course they extended it with their own version control system that offers additional functionality.) I had to clutch my side to keep from falling off of my chair. This must mean that there are still people out there who believe Microsoft has mended its ways! "C'mon baby, I love you so much, I swear I won't beat you up this time, I know I said that eve

Re:

[93 Escort Wagon]

Microsoft Team Foundation Server: It never fails under load.

(Because there is no load)

Re:

[Gr8Apes]

VSS? ROTFLMAO. Seriously. Even CVS/Subversion with their known issues with atomic commits seem like warp drive starships next to the VSS travois.

The good and the bad

[Parker Lewis]

The good: Git is decentralized, so everybody can continue working. The bad: several companies are using it as origin.

Re:

[Junta]

Of course, even if that's the case, it's easy enough to change origin or add a new remote if it becomes intolerable. I tend to have an 'internal' remote and a 'public' remote, and 'upstream' so that github going down is no big deal, but when it's up people new to projects can find it and submit pull requests when it can.

Basically opportunistically taking advantage of 'everyone is on github' without the downsides of 'screwed when it goes down'.

Re:

[Zero__Kelvin]

Somebody needs to invent git commands that allow that to be fixed!

Re:

[hord]

Not to mention all the open source projects that rely on GitHub for distribution. I wonder how much Node.js development gets done when GitHub is down.

Other peoples computers

[OzPeter]

That's what the cloud be.

Excessive Dependency Syndrome

[ErichTheRed]

I've spent a 20+ year career in IT, and have noticed the trend over the last decade has been to rely almost fully on a service you don't control for a key part of operations. Most of my work has been with Azure lately, and Microsoft is shifting from releasing updates in packaged format that they publish and host to just pumping it out onto GitHub. There was a story last year about how tons of web projects were broken when a developer removed a portion of functionality from a JavaScript framework hosted publically -- and it turned out to be a trivial sting-manipulation function.

There is nothing wrong with not reinventing the wheel every time and using other peoples' resources. There is something very wrong with assuming that third parties will keep their systems running 100% of the time and never have bad days. Even Microsoft won't guarantee that Azure regions, of which most are _collections_ of fully redundant data centers, will be fully available all the time and that your applications will never experience downtime. You should never assume a resource will be published in the same place and remain online forever, nor should you rely fully on a third party service to provide your only means of providing the service you're providing.

Re:

[Gr8Apes]

Depending upon how it's done, they have a backup of the part of the repo they're tracking, nothing more.

Re:

[Junta]

I agree with your general sentiment. Software devs cares less and less about good packaging and publishing a manageable low frequency stream of 'releases' for folks to consume, but not uselessly long token release cycles. Part of this is overconfidence in continuous integration, part of it is a misunderstanding of 'Agile', part of it is that on the whole developers are currently a bit spoiled by the way industry worships the skill of software development so they don't have to do things like make lives eas

Re:

[Gr8Apes]

Most "Software devs" today are barely aware of anything approaching real large scale code development. Therefore they have never been exposed to the need for proper packaging and maintaining manifests. It's sad state of affairs that the bulk of the profession is barely able to tie their figurative shoelaces.

Re:

[Junta]

Can in part blame containers and things like ansible/puppet/chef/salt.

They are all useful tools for adequately skilled admins to manage what they put together, but increasingly it's been a crutch of software devs to not bother doing good packaging and release management. So what if a hapless admin is slapped with some inscrutible stack trace or syntax error? So what if you get lazy and you ship a vulnerable system library that got updated by the distro vendor ages ago? The alternative would be spending a

As of 14:00 EDT

[Dracos]

The status page reports "Everything operating normally."

That's why you have backups

[maelkum]

You don't host your code on GitHub *only*, right?

You have mirrors on GitLab, Savannah, and a public repository at project's website (say, git.example.org), a local copy on your machine, and - of course - a copy stored safely at an off-site backup location (Tarsnap, S3, hubiC, etc.). I thought so.
The fact that GitHub went down is just a minor inconvenience to you, right?