USE AWS S3 WITH S3CMD

One of the most popular Amazon S3 command line clients is s3cmd, which is written in python. As a simple AWS S3 command line tool, s3cmd is ideal to use when you want to run scripted cron jobs such as daily backups.

STEP 1:- 

• To install s3cmd on Ubuntu or Debian:

     [  nitin@nitin-ubuntu: ~ ] $   sudo apt-get install s3cmd

STEP 2:-

•  Configure:

      [  nitin@nitin-ubuntu: ~ ] $ s3cmd --configure 
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3
Access Key: (Access Key for the admin S3 User we have created before)
Secret Key: (Seccret Key for the admin S3 User we have created before)

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: (your-password)
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]:

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't conect to S3 directly
HTTP Proxy server name:

New settings:
  Access Key: ***********************
  Secret Key: ***************************
  Encryption password: ***********
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: False
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] y
Please wait...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/home/nitin/.s3cfg'

STEP 3:- 

• Listing Buckets:

   [  nitin@nitin-ubuntu: ~ ] $ s3cmd ls 

• Listing Bucket contents (folders):

   [  nitin@nitin-ubuntu: ~ ] $ s3cmd ls s3://Bucket-Name 

• Listing Bucket contents (files):

   [  nitin@nitin-ubuntu: ~ ] $  s3cmd ls s3://Bucket-Name/Folder-Name 

• Download all folder content:

   [  nitin@nitin-ubuntu: ~ ] $  s3cmd get s3://Bucket-Name/Folder-Name/* 

• Delete all folder content:

   [  nitin@nitin-ubuntu: ~ ] $ s3cmd del s3://Bucket-Name/Folder-Name/* 

AMAZON EC2 WITH COMMAND LINE

                                

Step 1 :-

• Sign Up for Amazon EC2

When you create an Amazon Web Services (AWS) account, AWS automatically signs up the account for all AWS services, including Amazon EC2. With Amazon EC2, you pay only for what you use. New AWS customers can get started with Amazon EC2 for free. For more information, see Amazon EC2 Pricing.

If you have an AWS account already, skip to the next step. If you don't have an AWS account, use the following procedure to create one.

To create an AWS account

1. Go to http://aws.amazon.com and click Sign Up Now.

2. Follow the on-screen instructions.

Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Step 2:

• Launch an Amazon EC2 Instance Command line :-

Download

nitin@nitin-ubuntu:~$ wget "http://www.amazon.com/gp/redirect.html/ref=aws_rc_ec2tools?location=http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip&token=A80325AA4DAB186C80828ED5138633E3F49160D9"

nitin@nitin-ubuntu:~$ sudo unzip ec2-api-tools.zip

nitin@nitin-ubuntu:~$ sudo mv ec2-api-tools-1.6.7.4 ec2

nitin@nitin-ubuntu:~$ sudo mv ec2 /usr/bin/

• Configure enviroment variable

nitin@nitin-ubuntu:~$ sudo vim /etc/bash.bashrc

export JAVA_HOME=/usr/local/java

export EC2_HOME=/usr/bin/ec2/

export AWS_ACCESS_KEY=<Give your Access key>

export AWS_SECRET_KEY=<Give your Secret key>

export PATH=/usr/bin/ec2/bin/:$EC2_HOME:$JAVA_HOME:$PATH

Step 3:-

• Test Your Configration :-

nitin@nitin-ubuntu:~$ sudo ec2-describe-regions

REGION eu-west-1 ec2.eu-west-1.amazonaws.com

REGION sa-east-1 ec2.sa-east-1.amazonaws.com

REGION us-east-1 ec2.us-east-1.amazonaws.com

REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com

REGION us-west-2 ec2.us-west-2.amazonaws.com

REGION us-west-1 ec2.us-west-1.amazonaws.com

REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com

REGION ap-southeast-2 ec2.ap-southeast-2.amazonaws.com

Make sure you will get above output

• Check Avaliable images form AWS

nitin@nitin-ubuntu:~$ ec2-describe-images -a

You will get all images (AMI) Amazon machine Images from aws

Step 4:-

• To Create new instance on EC2

Key Pair additions :-

In order to login securely to a public AMI instance, you will need a public/private keypair.Amazon EC2 public images use this feature to provide secure access without passwords. There is a command in the API tools for this. You only have to choose a name for the keypair as parameter

nitin@nitin-ubuntu:~$ ec2-add-keypair keypair_for_ubuntu > keypair_for_ubuntu.pem

• Instance Creations :-

nitin@nitin-ubuntu:~$ ec2run <AMI Image ID> -k <Path for key pair file> -t <Type of instance>

nitin@nitin-ubuntu:~$ ec2run ami-76057f24-k keypair_for_ubuntu.pem -t t1.micro

Option Explantion :-

ami-76057f24 -> AMI image

keypair_for_ubuntu.pem -> Key pair genrated by ec2-add-keypair

t1.micro -> Type of instance

Instance ID returned by ec2-run-instances command is an identifier and will be used later to manipulate this instance (in this example, i-27494b46)

• Check Instance Status

nitin@nitin-ubuntu:~$ ec2-describe-instance-status i-XXXXXX

Status must show active

Step 5:

• Access the instance with ssh

nitin@nitin-ubuntu:~$ sudo ssh -i <PATH of Key Pair> user@<Hostname address you will get from ec2run command>

nitin@nitin-ubuntu:~$ sudo ssh -i .ec2/keypair_for_ubuntu ubuntu@ec2-<DNS-NAME-IPADDRESS>