Every so often I find myself working on that odd job that requires syncing files with Amazon's S3. In the beginning, I tried some of the various S3 FUSE interfaces—hoping for something that would play nice with rsync—but FUSE's stability always left something to be desired and more often than not I'd be left with that one transfer that never would quite finish correctly.

Eventually I discovered boto and settled in to using a hacked together (but stable) Python/boto solution for these type of tasks—all the while wondering why nobody took the time to write a "real" rsync-like client for S3.

Well, this last time around I finally decided to stop whining and take matters into my own hands. After a couple of late nights fleshing out my original boto solution, I'm happy to announce what I'm calling "boto rsync"—an rsync like wrapper for boto's cloud storage interfaces (both S3 and Google Storage).

Please take a look at the project on github and let me know what you think: http://github.com/seedifferently/boto_rsync

more...