Using Git with Dropbox(solarianprogrammer.com)
solarianprogrammer.com
Using Git with Dropbox
http://solarianprogrammer.com/2012/03/15/git-repo-dropbox/
60 comments
+1 for bitbucket.
I've used dropbox+git for sometime [1]. It throws "conflicted copy" error [2] when two people push to origin at the same time. Bitbucket much better than dropbox, but not as awesome as github.
[1] Shorter version - http://stackoverflow.com/a/1961515/45582
[2] http://shikii.net/blog/ignoring-funny-ref-error-on-git-dropb...
I've used dropbox+git for sometime [1]. It throws "conflicted copy" error [2] when two people push to origin at the same time. Bitbucket much better than dropbox, but not as awesome as github.
[1] Shorter version - http://stackoverflow.com/a/1961515/45582
[2] http://shikii.net/blog/ignoring-funny-ref-error-on-git-dropb...
+1 for bitbucket.org. The free private repos for up to 5 folks is awesome (for git and hg/mercurial). Very stable, never had any issues.
If you have an e-mail address in .edu you can have free private repos with unlimited users having access to them. Just register your .edu e-mail on the site and the 5 user limit is removed.
I also wonder why so many people forget about bitbucket. To me really it's the matter of taste, and there is certainly less differences between github and bitbucket than there is between git and mercurial.
I managed to convert a few Github paying customers to switch to bitbucket. The only reason they state for using github was that they didn't know bitbucket is available for use with git (as well as with mercurial) besides the private repos thingy.
AFAIK thanks to the hashing used with git there is very little chance of corruption. The filenames are 32bit hashes so the probability of collision between the hashes of two given files is 1 / 2^32. The probability of collisions between any of N given files is (N - 1) / 2^32.
I meant a state that the repo does not like and not really corrupted. say you push your changes and someone else pushes their changes and you both try to move the head and you get a dropbox conflict. You would have to go and fuss with the refs and the refs log to find out where the other head should have gone and manually change things that you shouldn't have to. Not the end of the world but for a guide that is seemingly for beginners why hassle with it when there is a free way that is easier to do what they are trying to accomplish?
Just to correct you: the probability is _much_ bigger than (N-1)/2^32.
See the birthday problem: http://en.wikipedia.org/wiki/Birthday_problem
See the birthday problem: http://en.wikipedia.org/wiki/Birthday_problem
The search space of the object name is a sha1 and is 40 characters long and not 32 bits so it's quite a bit larger than the op was talking about(it's actually 160bits).
see: http://book.git-scm.com/1_the_git_object_model.html
see: http://book.git-scm.com/1_the_git_object_model.html
Repo "corruption" via dropbox does not need a SHA-1 collision.
Git uses other files for tracking things, for example: .git/refs/* and .git/HEAD, which point to particular objects (via SHA-1 hashes). If one of these has a "confilicted change" (via dropbox), one person has the HEAD of the repo (or some other branch) pointing to a commit they do not expect it to point to. Even more interesting is when one person wins HEAD and the other wins it's paired branch in .git/refs/ .
Further, git does occasional packing of the object repository. 2 People attempting to pack the object repository when it contains different items might produce interesting results.
Git uses other files for tracking things, for example: .git/refs/* and .git/HEAD, which point to particular objects (via SHA-1 hashes). If one of these has a "confilicted change" (via dropbox), one person has the HEAD of the repo (or some other branch) pointing to a commit they do not expect it to point to. Even more interesting is when one person wins HEAD and the other wins it's paired branch in .git/refs/ .
Further, git does occasional packing of the object repository. 2 People attempting to pack the object repository when it contains different items might produce interesting results.
Am I wrong, or does pushing to the dropboxed git repo at the same time from two different computers will corrupt the repo ?
Hope to not be wrong, but if you push different commits you should be safe.
It can, yes. It's relatively rare though. The nice thing about git is that, when it happens, you can just nuke the Dropbox repository and rebuild it without losing anything. Git's checksums guarantee that the corruption results in an error and not bad data.
I managed to screw up a git repo in Dropbox. I believe it was by making a commit on a non-connected computer than making another commit before the first one had synced to dropbox.
It was not the end of the world, but I would not recommend storing git repo in dropbox. If it's really something so small and insignificant that it doesn't need a git repo, then just put it in dropbox. If you want to use git, take the time to set up a proper repository.
It was not the end of the world, but I would not recommend storing git repo in dropbox. If it's really something so small and insignificant that it doesn't need a git repo, then just put it in dropbox. If you want to use git, take the time to set up a proper repository.
Even if you do screw it up, the commit should all be there and valid. The only thing that happens is HEAD won't point to the right commit on other boxes, and then when you do sync you'll clobber the file containing the commit for HEAD, thus losing the other commits. Fortunately, they'll still exist until a gc occurs, so you can go find them in reflog.
Correct, I still had my HEAD files, but everything else seemed pretty broken. I'm not a Git expert by a long shot, but I would imagine that Dropbox renamed some files like "Foo.bar (Eli's Conflicted Copy)" and that really confused the history.
Anyway, not something I would recommend.
Anyway, not something I would recommend.
Why bother with dropox? Install jgit and push it on your S3 account for $0.01 per month. Or use Bitbucket's free private git repos.
This is kind of an amazing tip. Thanks. I had no idea jgit even existed, let alone could do this.
I don't remember how I found it, but this it's a killer feature to use S3 to host git repos for archival. I now have 3 copies, it's overkill, but Heroku, bitbucket & S3. Unfuddle also has free private git repos.
The best part is that since it's still just a git repository, once it's local you don't have to bother with jgit at all. I'm going to replace a custom git bundle + tarsnap backup thing I have going with this tonight. Gonna be great.
The Mono project translates JGit to C# for MonoDevelop:
https://github.com/mono/ngit/tree/master/gen
https://github.com/mono/ngit/tree/master/gen
What does jgit have to do with S3? From the description on the project page it's just a Java implementation of git.
JGit has support for the S3 protocol built in which makes it possible to push directly to S3
Amazon allows you to run code on their servers with strict usage-based billing. Usage-based bill for a git server is essentially zero. Also works for irc bouncers and other such services with low computational cost that you want to run somewhere on the cloud.
Not sure that is right. There is not remote code executing to support it. jgit has a feature where you can push and fetch from an s3 bucket as if it were just an http repo. The assumption is that the s3 storage space and bandwidth would be low enough to be negligible.
Honestly, I just keep my checked out copy in Dropbox. If it's something I feel like keeping private, I just don't push it anywhere. Even less effort involved.
If you have a working directory checked out in ~/Dropbox, beware of multi-machine madness. I've run into issues with a git repo in ~/Dropbox + IntelliJ + multiple computers -- if you leave IJ running on a machine you're not using, it'll notice file changes and re-index things in git. This can, in turn, cause Dropbox to create conflicts, effectively corrupting your git repo. As a result, I tend to keep only non-Java (i.e., non-IJ) projects in ~/Dropbox.
Run "find .git -name 'conflict'" in your working directory to see if this has happened to you.
Run "find .git -name 'conflict'" in your working directory to see if this has happened to you.
Oops; formatting fail. Should be '<asterisk>-conflict', not just 'conflict'.
I used to do this, but I learned the hard way that it fails terribly when you're working on different branches on different machines.
Wasting your talent and time, just use bitbucket.
Life is short.
Life is short.
Warning: If you use git+Dropbox on a Mac (or have one joined anywhere), keep in mind that most Macs have case-insensitive filesystems.
What this means is checking out a git repo like the Linux kernel will not work. This drove me a little nuts until I figured out what happened (I'm an old school Unix guy so did not realize that HFS+ is case insensitive).
What this means is checking out a git repo like the Linux kernel will not work. This drove me a little nuts until I figured out what happened (I'm an old school Unix guy so did not realize that HFS+ is case insensitive).
IIRC you can reformat as case-sensitive.
It's always unfortunate that I realize this after there's already a pile of stuff on the machine. It usually goes something like this:
- get burned by case insensitive filesystem
- swear that next time I'll reformat it to be case sensitive
[much time passes]
- get new machine
[much more time passes]
- get burned by case insensitive filesystem
- repeat
- get burned by case insensitive filesystem
- swear that next time I'll reformat it to be case sensitive
[much time passes]
- get new machine
[much more time passes]
- get burned by case insensitive filesystem
- repeat
In the past, I've seen a few mentions of bugs / non-running programs when run on a case-sensitive HFS+. Not sure if that's true any more, but changing a default-case-insensitive system to a case-sensitive one isn't usually a problem-free operation.
Last time I formatted my drive as case sensitive I ran into issues with Adobe's CS5 apps and Steam.
There could be more, but I reformatted it again shortly afterwards.
There could be more, but I reformatted it again shortly afterwards.
Or create a (resizeable) image that is case-sensitive. Not sure what the performance overheads of that will be, though.
That's probably a better option in any case as some apps can be erratic on a case-sensitive file system (or at least they were back when I last tried it around 10.2~10.3).
Old news, been doing this for years. (I hadn't heard of bitbucket back then.)
Also, my use case is extremely simple. I'm just sharing my own personal projects between my desktop and my laptop. I only ever use one of the two at a time, and am not using any tool that automatically checks or updates git repos.
Also, my use case is extremely simple. I'm just sharing my own personal projects between my desktop and my laptop. I only ever use one of the two at a time, and am not using any tool that automatically checks or updates git repos.
I really just do all my work in Dropbox folders anyway. Instant code everywhere. Life's good.
Until you compile your code, end up with 300 MB of object files and have no way to exclude them by file extension.
Haha, true. Although not a problem I've got at the moment. Couldn't you just compile into a directory, and exclude that directory?
You bring up an interesting point though, I wonder if there's any kind of hack you could do to the dropbox config files to alter that?
You bring up an interesting point though, I wonder if there's any kind of hack you could do to the dropbox config files to alter that?
Even if you hacked up the makefile to move the object files, there's still a hell of a lot of junk generated when compiling a typical large C project. Dropbox needs something like git ignore files:
http://help.github.com/ignore-files/
http://help.github.com/ignore-files/
Or you could do an out-of-source build, which is cleaner anyway. CMake and autoconfiscated projects support this by default.
I wrote a script awhile back to simplify this workflow: https://github.com/rpetrich/git-dropbox
It is not managable to have too many binaries changed in each commit that are under control git synced with Dropbox. (I know Git is not the best suitable vcs for binaries!) I'd always prefer Bitbucket for that reason. It's an Atlassian product and it is reliable, has no difference than github for private repos. Your files will be on the cloud, is your Dropbox more secure and permanent than Bitbucket? Hell no.
Would storing a git bundle on dropbox be better to avoid concurrent modifications to your refs? (A git bundle is just a git pack file of commits along with new ref pointers in a single file).
Could you push to one bundle per computer and then git fetch all the bundles from your peers? Once all your peers have your changes, you can start making bundles from a later revision so that the incremental bundles aren't too big.
Could you push to one bundle per computer and then git fetch all the bundles from your peers? Once all your peers have your changes, you can start making bundles from a later revision so that the incremental bundles aren't too big.
I'm sure I'm not the target audience, but I'm surprised that in this day and age there isn't always a shared resource lying around (a linode, a slice, an ec2 instance). Or just allow ssh between developers with accounts limited to git access. The dropbox part seems unnecessary.
Except you don't own the stuff in your dropbox any more than you own you pictures on Facebook.
From https://www.dropbox.com/terms:
"You retain full ownership to your stuff. We don’t claim any ownership to any of it."
Or am I missing something?
"You retain full ownership to your stuff. We don’t claim any ownership to any of it."
Or am I missing something?
I believe they added that somewhat recently after their privacy issue fiasco.
There is also this snippet that was added, which assumes that your key isn't really private as opposed to services like SpiderOak which have no knowledge of your key:
> We may disclose to parties outside Dropbox files stored in your Dropbox and information about you that we collect when we have a good-faith belief that disclosure is reasonably necessary to (a) comply with a law, regulation or compulsory legal request; (b) protect the safety of any person from death or serious bodily injury; (c) prevent fraud or abuse of Dropbox or its users; or (d) protect Dropbox's property rights.
There is also this snippet that was added, which assumes that your key isn't really private as opposed to services like SpiderOak which have no knowledge of your key:
> We may disclose to parties outside Dropbox files stored in your Dropbox and information about you that we collect when we have a good-faith belief that disclosure is reasonably necessary to (a) comply with a law, regulation or compulsory legal request; (b) protect the safety of any person from death or serious bodily injury; (c) prevent fraud or abuse of Dropbox or its users; or (d) protect Dropbox's property rights.
If you want to keep it private what the heck would you push it to Dropbox ?!
Also of note, if you're going to push your private repo to the cloud anyway, Bitbucket has free private Git repos. It's not as featureful as Github, but it has more than just the simple file storage you'd get through Dropbox.
funny :)
[deleted]
So how do I do this in the GUI?
You don't have the possibility of corrupting your repo with an unsynced commit from a differennt machine.