Open-source Dropbox alternative powered by Git

[mmahemoff]

Some mention of hosting this on GitHub and BitBucket here. I can't imagine Git-As-A-Service providers would be thrilled with this kind of application, it will completely hammer their services compared to the occasional commits and pulls that occur with a code base.

It will probably update their conditions if they don't already preclude it. Or maybe they will embrace it with a premium pricing plan. I certainly wouldn't count on hosting this against BitBucket's free plan.

[hemancuso]

Ever try committing a 100meg file into git? 100% CPU for 10s of seconds. Game over.

Git is a great idea for this, but in practice it performs terribly.

[Orva]

It gets only worse when you keep adding more binary files. After doing this for while your repo will not work unless you pack it, which will take horrific times. If it will succeed: my git was 'surprisingly' segfaulting when trying to acquire 16Gb of ram at once during pack operation. (I was trying to keep RAW-photos synced, git stopped working when there was about 4Gb of files stored.)

Git is made and optimized for text files, no big surprise that it doesn't cope very well with binaries.

[Animus7]

Sure, if you do this the naive way (map files onto the git repo 1:1).

I don't know how this project is doing it, but adding a buffering/sharding layer for large files would alleviate this problem at the cost of worse delta compression (among other things).

That said, I still don't think git is necessarily the best platform to base a Dropbox alternative upon, but the commit resource hogging is very fixable.

[emillon]

git-annex is a nice extension for this kind of use case. Metadata are stored in git, and file contents are stored separately.

http://git-annex.branchable.com/

[SimonSapin]

bup uses the git repository format (packfiles) but splits big files into chuck to avoid this problem. Also does de-duplication at the chunk level.

https://github.com/apenwarr/bup/#readme https://github.com/apenwarr/bup/blob/master/DESIGN#L92

[paulhauggis]

I have this same problem with mercurial. Is there a better alternative for source control of large files?

[miles_matthias]

Just curious - why are there so many Dropbox alternative posting lately? Is re-doing an already elegant solution really a top priority for people? I haven't looked at this post at all (so I'm not trying to make any judgement on it), I'm just curious as to why so many people are interested in making Dropbox alternatives lately.

[Karunamon]

I'd assume the short version is that Dropbox:

1) Isn't free

2) Is accessible to $stateAuthority if they come knocking

3) Makes your data access reliant on a third party who may or may not be up in an emergency or:

4) Could disappear at any time and take your data with them

So basically, the same problems that affect every cloud solution provider ever.

[jakubw]

One reason may be that it actually has interesting engineering aspects. The other one may have to do with some sort of self-validation: "Hey, Dropbox is a massive success, can I build something like this?". It's what usually happens to popular services like Twitter, GitHub - they all have had numerous "me too" attempts.

[tommi]

The alternatives try to address the biggest problem of Dropbox: the Dropbox company knows the content of my files. That in turn means that various US agencies can browse and analyze my files without my knowledge, approval and a warrant.

[zem]

I think it's part of a general push by the bleeding edge away from hosted-cloud solutions and towards controlling bits of the stack themselves. dropbox is a good candidate for replacement because storing a few personal documents to sync with your various devices sounds like something you should be doing for yourself, unlike, say, email which is manifestly harder to do right.

[nesquena]

Just want to add a quick comment about our goals for gitdocs. Obviously this was never intended to be a 'dropbox-killer', in fact at Miso we actually use both. Dropbox for videos, large binary files, business and legal files.

Instead we use gitdocs for storing our "docs": Task lists, wiki, planning, collaborative design, note taking, code snippets et al. And the gitdocs web front-end (http://imgur.com/eaTTY) is optimized for that since it renders wiki pages (formatted markdown/textile), has full code syntax highlighting, file search, revision history and a rich text editor.

[nillort]

Dropbox wins on UX, which is where FOSS fails almost every time. A quick way to know whether such projects will fail: if they mention anything about the technologies used. Making it obvious to use and "just work" is what is needed.

Too bad that is actually the hard part.

[burgerbrain]

For what definition of "fail"?

I think it's pretty obvious that this isn't intended, at least in it's current incarnation, to be a dropbox-killer. It doesn't need to pander to the same audience.

"dares to mention technologies used" is only a killer for a subset of all possible audiences. For example, knowing that Git is hacked together with C and bash doesn't make it any less useful to me (ie, someone who isn't afraid of computers)

[iFire]

Look at Sparkleshare's website and see what you think?

[teichman]

For synchronization of your personal files, check out Unison.

http://www.cis.upenn.edu/~bcpierce/unison/

[flarg]

Seconded - I've used Unison to sync across PCs and between different OSs for many many years; it just works

[pors]

Not really an alternative till it supports clients for all sorts of platforms and mobile devices IMHO

[vegardx]

So, Windows support then? And most mobile devices will be able to access the files through the browser, no need for a client to do that, like with Dropbox.

I love the idea and the hackyness over it all. Love it!

[Karunamon]

Git clients exist for iOS, Android, Windows, and Linux. There's 98% of the computing world right there.

[jakubw]

It's neat because it can be used on top of git without any additional effort but it'd be more interesting to see something that leverages git's server-side post-push hook to notify the other clients about changes. gitdocs, depending on the polling interval, either has an increased probability of causing merge conflicts or does a crazy amount of git pulls.

[joshbuddy]

Actually, it uses file system events so the pushes occur instantly. However, the pulls are done with polling.

[rcthompson]

I don't see any reason they can't add this later. Get something that works first, then make it better.

[spullara]

The real problem with this is that Git is awful at versioning binary / large files.

[callahad]

Related: http://sparkleshare.org/

[twunt]

The readme suggests to use bitbucket.org which I didn't realise had unlimited storage, even on the free plan for up to 5 users?

So if the CPU issue can be resolved for committing large files, could it be used as a backup for your warez etc?

[peterhajas]

I think Lipsync - https://github.com/philcryer/lipsync - might be better, Git isn't great for lots of large binary files.

[listening]

A comparable alternative is never going to free for everyone. Someone has to pay the Amazon fees for S3 use. Isn't it true that the few Dropbox users who do pay for it support all the ones who do not?

[nubela]

Hah, I've done something similar to this, except it uses SVN. Cross platform too! But you know what sucked? Performance. SVN is slooowwwww when it came to binary files.

You can check it out here.

[rcthompson]

This was posted here a few weeks ago, but at the time it was Mac-only. Now it works on Mac, Linux, and Windows. I assume this is the reason for the re-post.

[dahlia]

It reminds me again that Dropbox is hard to beat.

[umrashrf]

I had a same idea right after dropbox but I lost the thought about the clients for all popular OS.