S3cmd – Sync Files Between S3 bucket and Local Directory

1. Syncing Files from Local => S3 Bucket

For example I want to sync my local directory /root/mydir/ to S3 bucket directory s3://tecadmin/mydir/ where tecadmin is bucket name. I have created some new files in /root/mydir/ and sync to s3 bucket using following command.

# s3cmd sync /root/mydir/ s3://tecadmin/mydir/ 

[Sample Output]

/root/mydir/index.php -> s3://tecadmin/mydir/index.php  [1 of 2]
 397 of 397   100% in    0s     4.02 kB/s  done
/root/mydir/readme.html -> s3://tecadmin/mydir/readme.html  [2 of 2]
 9202 of 9202   100% in    0s   103.62 kB/s  done
Done. Uploaded 9599 bytes in 0.3 seconds, 27.92 kB/s

Note: Do not forgot to add trailing slash (/) in local directory path when specifying s3 bucket with full directory path.

To keep preserve file attributes like date/time etc use -p or –preserve parameter like below

# s3cmd sync /root/mydir/ --preserve s3://tecadmin/mydir/

If we want to sync only newly created file on source use –skip-existing parameter. It will skip all files which already exists on destination either its modified on source.

 s3cmd sync /root/mydir/ --skip-existing s3://tecadmin/mydir/

If you want to delete all files from s3 bucket which has removed from local use –delete-removed parameter.

# s3cmd sync /root/mydir/ --delete-removed s3://tecadmin/mydir/

2. Syncing Files from S3 Bucket => Local Directory

For this example I am again using same folder and bucket used above. To test this i have put some extra files in s3 bucket (s3://tecadmin/mydir/) and executed following command to sync all files to local directory.

# s3cmd sync s3://tecadmin/mydir/ /root/mydir/

[Sample Output]
s3://tecadmin/mydir/logo.jpg -> /root/mydir/logo.jpg  [2 of 3]
 7219 of 7219   100% in    0s   125.28 kB/s  done
s3://tecadmin/mydir/user.php -> /root/mydir/user.php  [3 of 3]
 40380 of 40380   100% in    0s   596.33 kB/s  done
Done. Downloaded 47599 bytes in 0.3 seconds, 184.40 kB/s

We can also used –preserve, –skip-existing and –delete-removed parameters during syncing files from S3 bucket to Local directory as followings.

# s3cmd sync s3://tecadmin/mydir/ --preserve  /root/mydir/
# s3cmd sync s3://tecadmin/mydir/ --skip-existing /root/mydir/
# s3cmd sync s3://tecadmin/mydir/ --delete-removed /root/mydir/
Share this Post:
Digg Google Bookmarks reddit Mixx StumbleUpon Technorati Yahoo! Buzz DesignFloat Delicious BlinkList Furl

Comments are closed.