Amazon S3 Setup

This guide walks you through creating AWS IAM credentials with the minimum permissions needed for Pillar to read documents from your S3 bucket.

Prerequisites

• An AWS account
• An S3 bucket containing the documents you want to sync
• Access to the AWS Console

Step 1: Open IAM Console

1. Go to the AWS Console
2. Search for IAM in the search bar and open it
3. In the left sidebar, click Users

Step 2: Create an IAM User

1. Click Create user
2. Enter a descriptive username (e.g., pillar-s3-reader)
3. Click Next

Step 3: Set Permissions

You have two options for setting permissions:

Option A: Attach a Custom Policy (Recommended)

1. Select Attach policies directly
2. Click Create policy (opens in a new tab)
3. Switch to the JSON tab
4. Paste this policy, replacing YOUR-BUCKET-NAME with your actual bucket name:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR-BUCKET-NAME",
        "arn:aws:s3:::YOUR-BUCKET-NAME/*"
      ]
    }
  ]
}

5. Click Next
6. Name the policy (e.g., PillarS3ReadOnly)
7. Click Create policy
8. Go back to the user creation tab, click the refresh button, and search for your new policy
9. Select the policy and click Next

Option B: Use AWS Managed Policy

For simpler setup (but broader access):

1. Select Attach policies directly
2. Search for and select AmazonS3ReadOnlyAccess
3. Click Next

Note: The managed policy grants read access to ALL buckets. Option A is more secure.

Step 4: Create the User

1. Review your settings
2. Click Create user

Step 5: Generate Access Keys

1. Click on the user you just created
2. Go to the Security credentials tab
3. Scroll down to Access keys
4. Click Create access key
5. Select Application running outside AWS
6. Click Next, then Create access key

Important: Copy both the Access Key ID and Secret Access Key now. The secret key is only shown once.

Step 6: Add Credentials to Pillar

1. In Pillar, select Amazon S3 as your cloud provider
2. Enter your Bucket Name
3. Select the correct AWS Region
4. Paste your Access Key ID
5. Paste your Secret Access Key
6. Click Test Connection to verify access

Understanding the Policy

The recommended policy grants only two permissions:

Pillar never writes, deletes, or modifies your data.

Multiple Buckets

To grant access to multiple buckets, add their ARNs to the Resource array:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::bucket-one",
        "arn:aws:s3:::bucket-one/*",
        "arn:aws:s3:::bucket-two",
        "arn:aws:s3:::bucket-two/*"
      ]
    }
  ]
}

Security Best Practices

• Use dedicated IAM users: Create a separate user just for Pillar
• Principle of least privilege: Only grant read access to specific buckets
• Rotate access keys periodically: Delete and recreate keys every 90 days
• Never commit credentials: Keep your access keys secure and out of source control
• Enable MFA on your root account: Protect your AWS account

Troubleshooting

"Access Denied" error

• Verify the IAM user has the correct policy attached
• Check that the bucket name matches exactly (case-sensitive)
• Ensure the policy includes both the bucket ARN and bucket/* for objects
• Verify the AWS region is correct

"Invalid credentials" error

• Double-check the Access Key ID—it should start with AKIA
• Make sure there are no extra spaces in the credentials
• Try generating new access keys if the current ones aren't working

"Bucket not found" error

• Verify the bucket name is spelled correctly
• Ensure the bucket exists in the specified region
• Check that the bucket isn't in a different AWS account

Finding Your Bucket Region

1. Go to S3 in the AWS Console
2. Click on your bucket name
3. Go to the Properties tab
4. Look for AWS Region at the top

Common regions:

• us-east-1 — US East (N. Virginia)
• us-west-2 — US West (Oregon)
• eu-west-1 — Europe (Ireland)
• ap-northeast-1 — Asia Pacific (Tokyo)