The memory "leak" persists across restarts of my process. That is what beats me. If memory were leaking while the process was alive and was reclaimed by the OS upon killing the process, then I could debug the process itself.
I read that running multiple git commands at the same time on the same repo should be fine (I can't remember where I read this). It is probably better to synchronize regardless!
Thank you! I tried hard to keep the code very simple sacrificing some 20%-scenario requirements (such as automatic conflict resolution).
I am using this to synchronize my Markdown based notes files across machines. There are under a hundred files now so I haven't hit any issues in that department yet.
However, there is a gradual memory leak which persists even when I kill and restart my process. I observed this only on a KVM based Linux guest - not sure if this is because of inotify based listening but I'm going to have to dig deeper to find out.
No apologies required! Thanks for taking the time to post the links. I discovered "syncthing" via this thread and while it sucks that I did not come across it before, I am excited to try it out.
It was a ton of fun on a saturday afternoon when I wrote this code. I was trying to avoid rolling my own syncing solution but I guess my Google-fu failed me :)
My goal was to write a very thin wrapper around the workflow I would follow if I had to sync the changes myself manually. The tricky part was in figuring out how to inform multiple client machines when the backup server noticed a change in the file system. I wanted to avoid writing a server-side component that I had to install on the server and maintain.
When I found that I could use a combination of "ssh" and "inotifywait" (run inotifywait on the server using ssh from the client and listen for changes), I was pleasantly surprised that this even worked! I see my implementation in this aspect as the equivalent of AJAX long-polling that used to be applied for chat like communication in the browser in some implementations. i.e. When some modification happens on the server filesystem, the "inotifywait" command quits thereby unblocking the "ssh" command upon which I do a "git pull".
Because of the above, I was able to keep my implementation really simple - The whole functionality was achieved in under 300 lines of code.
Thanks for the tip. I wanted to keep my implementation extremely bare-bones and simple so I chose to leave conflict resolution (which itself is a rare event in my a use-case as I usually only make modifications on one synced machine at a time).
Very cool! I wish I'd found your repo before writing my implementation. In fact, I was going to rewrite mine in Golang to reduce memory consumption on my back-up server. I will take yours for a spin.
Nice! I wish they support fetching changes from the remote repo upon an file change event. I'd written a dropbox equivalent using git as the storage mechanism that does something similar - https://github.com/prashanthellina/pullbox. It works well, but I like hiding the interaction behind FUSE.
1. Make it work on OSX 2. Use a lock file to prevent multiple instances from running when I put this in crontab
I found watchman[0]. Got to see if I can use that in place of "inotifywait".
[0]: https://facebook.github.io/watchman/docs/install.html