S3concurrent, yet another S3 syncing project(winstonchen.co)
winstonchen.co
S3concurrent, yet another S3 syncing project
http://www.winstonchen.co/2015/09/introducing-s3concurrent-yet-another-s3.html
1 comments
How does its performance compare to aws s3 sync?
Note that you can adjust max_concurrent_requests which defaults to 10.
I'm always surprised by the number of "download from s3" tools there are. AWS-CLI does a really quite nice job of it.
To be fair, here is a tool that has features no other tool I know: https://github.com/rlmcpherson/s3gof3r
It can saturate your bandwidth and you can pipe to/from it.
It's great to deal with hundreds GB's archive on the fly.
"Examples:
$ tar -cf - /foo_dir/ | gof3r put -b my_s3_bucket -k bar_dir/s3_object
$ gof3r get -b my_s3_bucket -k bar_dir/s3_object | tar -x
It's written in golang. So it's a simple binary.
It's written in golang. So it's a simple binary.