Wednesday 21 September 2016

Programmatically upload files to amazon (AWS) S3

Background

This post will show you how to programmatically upload files to your AWS S3 account using AWS S3 SDK. Lets start by creating access key.

This post hopes you already have a S3 account set up on your AWS console.



Credentials setup


Lets start be creating access key. Following screenshots show you how to create your access key using AWS IAM (Identity and access management).






Once you have create a user you need to give it access to S3. Follow next set of steps for that -



 This will give your new user access to S3.Now you can use these credentials in the code. Before we go to code there is one more step. You need to set up your credential file. It will be in
  • ~/.aws/credentials
If its not there create one and add credentials to it as show below from the user you had created from IAM console.


I have used some template here but replace it with your exact access key if and secret key. Lets go on to the code now.

Programmatically upload files to amazon s3

I am using eclipse and ivy dependency management. So your ivy file should look like following -

<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
    <info
        organisation="osfg"
        module="AwsS3Demo"
        status="integration">
    </info>
    
    <dependencies>
        <dependency org="com.amazonaws" name="aws-java-sdk-s3" rev="1.11.36"/>
    </dependencies>
</ivy-module>


Note the dependency we have used. It for aws s3 only.

Now lets head on to the code -

package com.osfg;

import java.io.File;
import java.io.IOException;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;

/**
 * 
 * @author athakur
 *
 */
public class AwsS3Demo {
    
    private static String AWS_BUCKET_NAME = "test-athakur";
    private static String AWS_KEY_NAME = "testData";
    private static String UPLOAD_FILE = "/Users/athakur/Desktop/data.txt";
    
    public static void main(String[] args) throws IOException {
        AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider());
        try {
            System.out.println("Uploading a new object to S3 from a file\n");
            File file = new File(UPLOAD_FILE);
            s3client.putObject(new PutObjectRequest(
                    AWS_BUCKET_NAME, AWS_KEY_NAME, file));

         } catch (AmazonServiceException ase) {
            System.out.println("Caught an AmazonServiceException, which " +
                    "means your request made it " +
                    "to Amazon S3, but was rejected with an error response" +
                    " for some reason.");
            System.out.println("Error Message:    " + ase.getMessage());
            System.out.println("HTTP Status Code: " + ase.getStatusCode());
            System.out.println("AWS Error Code:   " + ase.getErrorCode());
            System.out.println("Error Type:       " + ase.getErrorType());
            System.out.println("Request ID:       " + ase.getRequestId());
        } catch (AmazonClientException ace) {
            System.out.println("Caught an AmazonClientException, which " +
                    "means the client encountered " +
                    "an internal error while trying to " +
                    "communicate with S3, " +
                    "such as not being able to access the network.");
            System.out.println("Error Message: " + ace.getMessage());
        }
    }

}

Just run the above code and it should upload your file to the AWS bucket. Make sure -
  1. Credentials are correct in ~/.aws/credentials file
  2. That credential has access to S3 module
  3. File is present on your machine
  4. bucket name is valid
NOTE : ProfileCredentialsProvider internally uses ~/.aws/credentials file for the credentials to authenticate against AWS S3.

You can also find above code snippet at -
  • http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjSingleOpJava.html

IF all goes correctly file should get uploaded to the S3 -







Related Links

t> UA-39527780-1 back to top