How to Install s3cmd to manage AWS S3 Bucket ~ Appychip

How to Install s3cmd on ubuntu

s3cmd is a command line utility to manage s3 buckets on amazon web services. It can be useful in many cases, for example saving logs to s3 via logrotate, creating s3 buckets.

Uses of s3cmd

Uploading of data to s3
Retrieving of data from s3
Creating s3 bucket
Manage data on s3 bucket for example logrotate

Installation

sudo apt-get install s3cmd

Configure s3cmd environment

To configure s3cmd we need to have AWS ACCESS_KEY and SECRET_KEY. You can get these from AWS security credentials page. After getting the keys, run the following command.

s3cmd --configure

This command will result in several input to be given from terminal itself, which are as follows and will let you know How To Install

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3
Access Key: xxxxxxxxxxxxxxxxxxxxxx
Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: xxxxxxxxxx
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]: Yes

New settings:
  Access Key: xxxxxxxxxxxxxxxxxxxxxx
  Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  Encryption password: xxxxxxxxxx
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

We are done setting s3cmd. Now Let's look on how to use s3cmd command to manage s3 buckets.

Using s3cmd command

1. Creating New Bucket

s3cmd mb s3://mybucket

Bucket 's3://mybucket/' created

2. Uploading file in Bucket

s3cmd put file.txt s3://mycbucket/

file.txt -> s3://mybucket/file.txt  [1 of 1]
 190216 of 190216   100% in    0s  1668.35 kB/s  done

3. Uploading Directory in Bucket

s3cmd put -r backup s3://mybucket/

backup/file1.txt -> s3://mybucket/backup/file1.txt  [1 of 2]
 9984 of 9984   100% in    0s    18.78 kB/s  done
backup/file2.txt -> s3://mybackup/backup/file2.txt  [2 of 2]
 0 of 0     0% in    0s     0.00 B/s  done

4. List Data of S3 Bucket

s3cmd ls s3://mybucket/

                       DIR   s3://mybucket/backup/
2013-09-03 10:58    190216   s3://mybucket/file.txt

5. Download Files from Bucket

s3cmd get s3://mybucket/file.txt

s3://mybucket/file.txt -> ./file.txt  [1 of 1]
 4 of 4   100% in    0s    10.84 B/s  done

6. Remove Data of S3 Bucket

#Removing file from s3 bucket 
s3cmd del s3://mybucket/file.txt

File s3://mybucket/file.txt deleted

#Removing directory from s3 bucket 
s3cmd del s3://mybucket/backup

File s3://mybucket/backup deleted

7. Remove S3 Bucket

s3cmd rb s3://mybucket

Bucket 's3://mybucket/' removed

8. List All S3 Bucket

s3cmd ls

9. Sync log file to S3 Bucket

s3cmd sync /var/log/nginx/*/gz s3://mybucket/

Appychip

Monday, 23 November 2015