Using git and dropbox together(stackoverflow.com)
stackoverflow.com
Using git and dropbox together
http://stackoverflow.com/questions/1960799/using-gitdropbox-together-effectively/1961515#1961515
12 comments
Why are you pushing shitty "not done but commiting because flight is boarding..." to the server? Surely they are only pushed to your local repo and then squashed in a commit to the main server. Or worked out on a feature branch then merged into the mainline?
You know you can git commit to you local repo without needing an internet connection.
You know you can git commit to you local repo without needing an internet connection.
Yes, I know git can do that, and I know pushing doodoo commits is bad practice; that was my point.
However, I prefer not to try to use a laptop for real work (because even the fastest laptops are much slower than a current Mac Pro). So, since I can't feasibly take my ginormous Mac Pro with me, as soon as I leave the office the repo on that workstation won't be my "local" one anymore.
Before moving my code into Dropbox, to avoid pushing something incomplete, I had to manually deal with transferring my local repo to the next computer(s) I was going to be working on it (typically that would mean copying the repo to a laptop for the plane, and then to a workstation at the destination).
That process was annoying and wasteful, and using Dropbox fixed that for me. Hooray!
However, I prefer not to try to use a laptop for real work (because even the fastest laptops are much slower than a current Mac Pro). So, since I can't feasibly take my ginormous Mac Pro with me, as soon as I leave the office the repo on that workstation won't be my "local" one anymore.
Before moving my code into Dropbox, to avoid pushing something incomplete, I had to manually deal with transferring my local repo to the next computer(s) I was going to be working on it (typically that would mean copying the repo to a laptop for the plane, and then to a workstation at the destination).
That process was annoying and wasteful, and using Dropbox fixed that for me. Hooray!
i am still a svn user so i dont know exactly how git works, but how are you supposed to get your data on your other machines if you dont push it to the git server which is available from anywhere? Thats the point of syncing the working copy with dropbox, so you dont have to get it manually from your work machine.
As i understand it git repos are local so my other machines wont have access to that repo, or am i wrong ?
In git committing and pushing are two different things. With git you have a copy of the repo, as does anyone else who checks it out. Think of each repo like its own little svn server, everyone has one locally. So you can commit to that and yes no one has access (unless you open it up to them, but no need to complicate this). So you can commit your unfinished work without other people seeing it, and still have history to be able to go back if necessary. Then when it's ready you can push that back to the shared repo.
Branching in git is really cheap and easy compared to svn. A good way to work to always branch before starting anything. Work on that branch, commiting whenever you feel like. You don't need to push that branch unless someone else wants to work on the unfinished feature with you, or needs something from it before it is ready. In the meantime if the mainline moves on ahead you can just keep merging it into your feature branch to get the new changes. When it's ready you can then merge it back into the mainline and push that.
Branching in git is really cheap and easy compared to svn. A good way to work to always branch before starting anything. Work on that branch, commiting whenever you feel like. You don't need to push that branch unless someone else wants to work on the unfinished feature with you, or needs something from it before it is ready. In the meantime if the mainline moves on ahead you can just keep merging it into your feature branch to get the new changes. When it's ready you can then merge it back into the mainline and push that.
I'm missing where you answered his question. He's talking about sharing his latest edit to his other machines, which aren't permanently network connected, without creating an explicit git version.
> You don't need to push that branch unless someone else wants to work on the unfinished feature with you, or needs something from it before it is ready.
If you want to share a commit that isn't ready, it really should go on a feature branch and not the mainline. Then you can squash it into a real commit when its ready.
If you want to share a commit that isn't ready, it really should go on a feature branch and not the mainline. Then you can squash it into a real commit when its ready.
Exactly. That approach is the biggest benefit imo. With working copies synced via dropbox you can just switch Machines and continue were you left off without having to push incomplete/not working code. You also get a cloud backup of all your local changes for free.
Just dont work on a working copy that is out of Sync, but since you are only using this yourself, chances of corrupting something are much lower as with the approach described above.
For people with multiple machines this is awesomeness. Everybody else still benefits from the extra backup layer of dropbox.
For people with multiple machines this is awesomeness. Everybody else still benefits from the extra backup layer of dropbox.
It makes no sense to push to the server before they're done, why do you do this? Are you afraid your hard drive is going to go kaput while you're on the plane, or you're not taking your computer with you? That's the only reason I can think of to do this... git is not like svn, where commit and push are the same thing; on git, a push is the same as publishing for others to see, the commit is what's important and it doesn't require a net connection.
This is also what I do. The really annoying thing is that Dropbox doesn't sync permissions. It makes bit mess when a file is 777 in Git, but Dropbox resets it to 644 (I think).
This only happens when working on multiple machines.
Hopefully Dropbox will start syncing permissions soon.
This only happens when working on multiple machines.
Hopefully Dropbox will start syncing permissions soon.
Why don't you just pull the changes onto your laptop from your desktop, rather than pushing to the server then pulling back?
the thing with dropbox is that it does that automatically. No need to manually sync files all the time (which you tend to forget from time to time).
Just leave work and continue working on the same stuff at home. Easy and convenient.
There's a hump where people associate the push as something magical in version control. While that may be true of some systems, in a distributed system the commit is actually what's important.
I find git+dropbox particularly useful for non-public projects I work on on my own. I used to use a private github unfuddle repository and do a remote push, but now I just forget about the push and commit from within the dropbox folder.
The nice thing about Dropbox, and why I'm actually surprised this needs to be asked, is that it's completely transparent..it's just a folder. Create a project, git init and commit like you normally would.
I find git+dropbox particularly useful for non-public projects I work on on my own. I used to use a private github unfuddle repository and do a remote push, but now I just forget about the push and commit from within the dropbox folder.
The nice thing about Dropbox, and why I'm actually surprised this needs to be asked, is that it's completely transparent..it's just a folder. Create a project, git init and commit like you normally would.
It needs to be asked because a Git repository is a database.
What would happen if you had two MySQL servers running off the same data files in a shared Dropbox folder? Is that a good replication strategy?
What would happen if you had two MySQL servers running off the same data files in a shared Dropbox folder? Is that a good replication strategy?
Easy there! MySQL is something completely different. If relational database replication would be as easy as Dropbox, then they would all support it out of the box. Do NOT use MySQL on Dropbox!
Dropbox is for stuff, which needs to be synced but is (usually) not accessed concurrently. The programmer with the Dropbox-git will not code on two computers at once.
Dropbox is for stuff, which needs to be synced but is (usually) not accessed concurrently. The programmer with the Dropbox-git will not code on two computers at once.
No it's not. Git stores all it's internal data as hashes, and the chance of a collision is really small.
It was designed so it could be accessed via ssh for instance with multiple people pushing into at the same time.
It was designed so it could be accessed via ssh for instance with multiple people pushing into at the same time.
[deleted]
You'll probably end up corrupting the shared bare repo. This approach is only suitable for a small team (two in my case) where people can just shout over their cubicle walls: "Hey! Nobody push! I'm pushing now!". – Ates Goral Sep 13 '10 at 20:04
That's generally not as likely as you'd think - since the names of the files are hashes of their content, the worst that can happen is you'll get dummy files. Bob's 8331a5b is just as good as Sam's 8331a5b. You could conflict refs (like where 'master' points to), but that window is generally pretty small.
That's good to hear, especially since I have a number of my personal repos in my dropbox.
What about git repack?
git repack won't delete any files until it finishes creating a new packfile. You might end up with a window in which the deletions are synced but not the upload of the new pack - this means git operations from other hosts may error out, but there'll be no permanent damage once the sync finishes. Additionally, git won't delete orphaned objects in automatic GCs until a certain time period passes - so no problem with concurrent commits. And if all else fails you can always undelete every file under .git/objects through dropbox :)
Yeah, people defending this 'solution' made me go 8( , too. Felt like I was right back in 1997, doing 'version control' with carefully numbered directories because Sourcesafe 'is overkill'.
I've been advocating something like this for ages (albeit I use Mercurial) [1], it is a really really wonderful meeting of technologies. My approach is slightly different; to use a remote repository for version control storage and sync my development environment with dropbox. Some thoughts...
The approach detailed here is totally unsuitable for team work; you will end up running into issues because you are handing the Git "server" off to a technology that is unsuited to handle it. You are much much better off, unless certain no one else will push at the same time, using a proper Git server.
The point of meeting these technologies is to give you the advantage of a personal shared workspace (amongst development machines) with the power of proper version control.
I can be playing around with a personal project at work.. and when I get home simply pick up where I left off if inspiration still grabs me. My productivity has gone up a noticeable amount.
1. http://www.errant.me.uk/blog/2010/04/a-nifty-way-to-manage-c...
The approach detailed here is totally unsuitable for team work; you will end up running into issues because you are handing the Git "server" off to a technology that is unsuited to handle it. You are much much better off, unless certain no one else will push at the same time, using a proper Git server.
The point of meeting these technologies is to give you the advantage of a personal shared workspace (amongst development machines) with the power of proper version control.
I can be playing around with a personal project at work.. and when I get home simply pick up where I left off if inspiration still grabs me. My productivity has gone up a noticeable amount.
1. http://www.errant.me.uk/blog/2010/04/a-nifty-way-to-manage-c...
I don't understand this. It's very easy and cheap to get ssh access to a linux box setup somewhere. git over ssh is easy and has the same basic setup as the instructions in this link and it doesn't have the problems mentioned in the link.
I think the better all round solution (in terms of privacy, functionality and maintenance) is to go with a free private git hosting provider like assembla.com. I have tried both the dropbox solution as well as setting up git+ssh on a vps, and assembla is now my goto for personal and side projects.
how is it better for privacy to use a private git hosting provider than any other linux box you control?
If you don't care about privacy, then you might as well just use github and have access to all of their cool collaboration features.
If you don't care about privacy, then you might as well just use github and have access to all of their cool collaboration features.
Because it's incredibly dangerous to think your machines are more secure just because you own them. How much time per day do you spend making sure your machine and everything on it is secure?
We employ teams of people to keep your code safe on GitHub.
We employ teams of people to keep your code safe on GitHub.
There is the overhead of maintenance. Dropbox makes backups automatically. Also, sharing repos is very easy with Dropbox, even to a new member. You don't have to open an ssh account for another team member.
But you have to resolve dropbox conflicts. That work grows as you add people, whereas adduser is a constant amount of work for each new team member.
What maintenance is there for git over ssh? And, git makes backups automatically :)
What maintenance is there for git over ssh? And, git makes backups automatically :)
What about using git and s3? Any thoughts on this? I've seen it done for both java and python (dulwich I believe) implementations of git (as well as hg)--but not cgit. Would one have to do this using webdav or fuse?
Same issues with s3 with sync updates. Might be easier to work around as you have etags but wont work out of the box.
i just placed my projects folder within the dropbox folder, and use github as normal. perhaps i'm doing it incorrectly...
The state of your projects folder can change pretty rapidly when you check out different branches. For some reason I'm not a fan of my dropbox folder constantly trying to keep up with my branch changes.
I do like the idea of having a viewable copy of my git repo available, but to be honest I don't ever want to touch code if I can't git diff/commit
I do like the idea of having a viewable copy of my git repo available, but to be honest I don't ever want to touch code if I can't git diff/commit
Ditto this - the idea of having the bare repo in Dropbox, but having my frequently-changing stuff (especially, as you point out, with branch changes) locally seems to me to be the sanest alternative.
I hacked together a quick script to facilitate this (http://github.com/agnoster/git-dropbox), since it seems like one of those things where less friction potentially leads to significant changes in behavior.
I hacked together a quick script to facilitate this (http://github.com/agnoster/git-dropbox), since it seems like one of those things where less friction potentially leads to significant changes in behavior.
I agree, I used to have my working copies in Dropbox folder, but I observed some occasional conflicts. After those, I switched to having the bare repo on Dropbox.
Let R be the copy of the repository on your local machine that is syncing with Dropbox. You could do:
git clone R R2
While using git and dropbox might appear to work most of the time, and shouldn't lose data, it's not an ideal pairing -- have a look at my answer to this stack overflow question: http://stackoverflow.com/questions/2199637/is-this-plain-stu...
TL;DR: Git's hashing means that everything's going to be there, packing means that it's going to be a very inefficient sync (and you'll wind up with multiple copies of the same data) and branches may not behave as you'd expect.
TL;DR: Git's hashing means that everything's going to be there, packing means that it's going to be a very inefficient sync (and you'll wind up with multiple copies of the same data) and branches may not behave as you'd expect.
The only completely safe way to do this is to have a detached work tree in your Dropbox folder and your repository outside of it (i.e. set GIT_DIR and GIT_WORK_TREE, or provide --git-dir and --work-tree/core.worktree). That way each machine can automatically sync uncommitted work tree changes, which avoids the "push a crummy commit because I'm leaving work / boarding a flight / on a boat" problem, but you still use the conventional Git mechanisms for transferring objects between repositories, which avoids the "I'm afraid of conflicting refs / repository corruption" problem.
I do this myself... at first as a side effect as I work on any number of machines and sync my entire home directory. but I got used to being able to roll back to a recent version even if I somehow screwed up committing (has saved me a couple of times, e.g. when I accidentally don't add a file to the repo, assume it's there and it ends up deleted at some point). Although I treat git exactly the same (I don't do any fewer or more commits/branches etc) I enjoy the extra safety harness.
I don't see the win on top of github here, they're both "public" services, and github is designed specifically to host git repos. With dropbox you commit and wait for your code to sync; with git push you are synced as soon as the command completes.
ipad developers: Please give me a decent solution for code editing with git integration
ipad developers: Please give me a decent solution for code editing with git integration
I have indirectly heard of someone writing a Javascript client for git using the http transport. It should be doable though a fair amount of work.
The win is the dropbox is free while github charge for private repos.
This is what I keep thinking as well. I think the difference is... when I want to switch to my laptop in a coffee shop, I have to push to github, regardless of what state the code is in. The build may be broken, a feature may be half implemented. I don't really like pushing with "sync to laptop" as the commit message.
So maybe dropbox is just meant to sync across your development area and github is still where the version control takes place?
If someone can elaborate on why they like this dual model, please let me know if I'm on the right track... or if there is a git-only solution that works as well.
So maybe dropbox is just meant to sync across your development area and github is still where the version control takes place?
If someone can elaborate on why they like this dual model, please let me know if I'm on the right track... or if there is a git-only solution that works as well.
When I'm moving stuff from work to home or the other way around, I'm usually committing my topic branch as far as I can using "clean" commits (doing one thing and only one), followed by one "temp: moving to home" commit.
Then I push to our Gitorious (not trusting github with company internal code, so preferring a self-hosted solution) which we use as a sandbox and/or for code reviews and discussion.
Then, at home, I pull and continue working.
Once the topic branch is completed, I rebase -i to edit and or squash to finally clean out that temp commit. Once I,'m happy, I push to our main repository (which is hosted using gitosis and works quite like a central svn repository where the whole team has push access and where we deploy from)
This works for me because I'm totally comfortable with rebase -i, add -i, squash and amend now so I can get rid of that temp commit.
If I weren't (some people refuse to rebase even on their topic branches and want to work exactly like subversion), then the dtopbox solution would be compelling because of that ugly temporary commit that I can't get rid of without rebase.
Then I push to our Gitorious (not trusting github with company internal code, so preferring a self-hosted solution) which we use as a sandbox and/or for code reviews and discussion.
Then, at home, I pull and continue working.
Once the topic branch is completed, I rebase -i to edit and or squash to finally clean out that temp commit. Once I,'m happy, I push to our main repository (which is hosted using gitosis and works quite like a central svn repository where the whole team has push access and where we deploy from)
This works for me because I'm totally comfortable with rebase -i, add -i, squash and amend now so I can get rid of that temp commit.
If I weren't (some people refuse to rebase even on their topic branches and want to work exactly like subversion), then the dtopbox solution would be compelling because of that ugly temporary commit that I can't get rid of without rebase.
Push to your own fork of your project on github (if the project is not yours and you don't want to push incomplete things to the main repo), you can have multiple remote repos configured on your local clone and seamlessly switch and merge between them. Or if you don't want to fork, push to a work branch. That's why git has branching, forking and multiple remote repos features, it's not only a "version control system", it's a "distributed development system". Distributed means not only multiple people all working on one thing, but you also distributing your work.
This should do it, assuming you're on development:
git checkout -b tmp
git commit -a -m "throwaway"
git push wherever tmp
#switch machines
git checkout development
git fetch wherever
git merge wherever/tmp --no-ff
git reset HEAD^
git push wherever :tmp
Within git, branches are super-lightweight and easy to create. If you're coordinating with other developers (or finding yourself needing to shuttle code around between machines frequently) it would be worth creating a remote dev branch in your repo.
With a dev branch, you have the flexibility to just what you describe-- commit broken or half-completed code in order to move to another box-- and still clean up the commit or merge it with others before you move it to your "master" branch for deployment or sharing.
With a dev branch, you have the flexibility to just what you describe-- commit broken or half-completed code in order to move to another box-- and still clean up the commit or merge it with others before you move it to your "master" branch for deployment or sharing.
yeah, but then your logs and diffs get messy because you can't stage in hunks. or maybe I haven't grokked how. maybe `git merge stagingbranch -no-commit`?
Yes, you haven't grokked it. When you work on a development branch like this you can rewrite its history with git rebase --interactive before pushing your commits to the main branch.
You do have to be a bit careful not to introduce broken revisions as a part of your history-rewriting there, of course. Generally if you only use squash (which is sufficient to clean up "sync to laptop"-like changes) it won't be a problem though - it's when you start deleting or reordering commits that interesting things happen.
OF COURSE you can stage in hunks. That's why the index is so awesome in git.
Try git add -p (or -i, I prefer -p though) and you will see (and incidentally, never ever use commit -a any more)
And to alter commits before merging, use rebase -i and make use of the edit and squash options.
Try git add -p (or -i, I prefer -p though) and you will see (and incidentally, never ever use commit -a any more)
And to alter commits before merging, use rebase -i and make use of the edit and squash options.
This workflow always seems backwards to me. "git add -p" lets me stage partial commits in the index, where I cannot build or test them. What I want to do is stage them into my working copy from somewhere else. I wonder how hard "git stash apply -p" would be to add....
Switch to a new branch before commiting, commit only what you want then switch back. Doing a git checkout -b new-branch-for-this-bit-of-code is quick and painless if you want to stage things while you're working, doesn't change anything since you're creating a new branch from your current point, and gives you much easier logging, diffing and merging capabilities later on. This is one of the recommended workflows for git, in fact.
You can also commit parts then stash the rest, test, pop the stash, commit more, stash, test, etc, etc. You can add things to commits with --amend, so you can tweak them if you notice something is missing after you added things with git add -p and then commited. I do this a lot when I want to split a commit and use git add -p.
If you really want to test things from somewhere else, use git new-workdir to create a new working dir based off your local clone. Whatever you stash or commit to one, the other will see, since they're really pointing to the same local git clone (if you look at the .git/ dir of a workdir, you'll notice it's all symlinks to the original clone). Just make sure every local workdir you create like this is on a different branch, otherwise it will get confused when you change things on one side (because, of course, they're all pointing to the same local clone, but suddenly the local files won't match). I use this to have a "test workdir", which is based on a clean and permanently updated branch, where I cherry-pick/merge from the work branch into it, build and test, without having to stash on my work branch, reconfigure things and otherwise stop the coding workflow.
You can also commit parts then stash the rest, test, pop the stash, commit more, stash, test, etc, etc. You can add things to commits with --amend, so you can tweak them if you notice something is missing after you added things with git add -p and then commited. I do this a lot when I want to split a commit and use git add -p.
If you really want to test things from somewhere else, use git new-workdir to create a new working dir based off your local clone. Whatever you stash or commit to one, the other will see, since they're really pointing to the same local git clone (if you look at the .git/ dir of a workdir, you'll notice it's all symlinks to the original clone). Just make sure every local workdir you create like this is on a different branch, otherwise it will get confused when you change things on one side (because, of course, they're all pointing to the same local clone, but suddenly the local files won't match). I use this to have a "test workdir", which is based on a clean and permanently updated branch, where I cherry-pick/merge from the work branch into it, build and test, without having to stash on my work branch, reconfigure things and otherwise stop the coding workflow.
[deleted]
I found this step by step guide useful as well for setting up git & dropbox: http://blog.wekeroad.com/danger-danger/git-and-dropbox-sitti...
This way I can hack at work, hack at home, and hack on the plane... and I am never tempted to push one of those shitty "not done but committing because my flight to china is boarding" commits to the central repo (which is on a server accessed via ssh like normal).
It also saves my time because I don't have to git pull every time I switch computers in order to keep hacking where I left off.
That's a big, big win in my opinion, but it is just the same big win that Dropbox provides for various kinds of work. I don't see anything magical about the git+dropbox combo.