Google Cloud Storage Setup

This guide walks you through creating a Google Cloud service account with the minimum permissions needed for Pillar to read documents from your GCS bucket.

Prerequisites

A Google Cloud Platform (GCP) account
A GCS bucket containing the documents you want to sync
Access to the Google Cloud Console

Step 1: Open IAM & Admin

Go to the Google Cloud Console
Select your project from the dropdown at the top
In the left sidebar, navigate to IAM & Admin → Service Accounts

Step 2: Create a Service Account

Click + Create Service Account at the top
Enter a descriptive name (e.g., pillar-bucket-reader)
Optionally add a description: "Read-only access for Pillar document sync"
Click Create and Continue

Step 3: Grant Permissions

Grant the service account the minimum required role:

In the "Grant this service account access to project" section, click Select a role
Search for and select: Storage Object Viewer (roles/storage.objectViewer)
Click Continue, then Done

Note: The Storage Object Viewer role provides read-only access to objects in your buckets. Pillar only needs to list and download files—it never modifies your data.

Step 4: Create a JSON Key

Find your newly created service account in the list and click on it
Go to the Keys tab
Click Add Key → Create new key
Select JSON as the key type
Click Create

A JSON file will be downloaded to your computer. This file contains your credentials.

Step 5: Add Credentials to Pillar

Open the downloaded JSON file in a text editor
Copy the entire contents
In Pillar, paste the JSON into the Service Account JSON field
Click Test Connection to verify access

Restricting Access to a Specific Bucket (Optional)

For tighter security, you can restrict the service account to a single bucket:

Go to Cloud Storage → Buckets
Click on your bucket name
Go to the Permissions tab
Click Grant Access
Enter your service account email (e.g., [email protected])
Select the role: Storage Object Viewer
Click Save

Then remove the project-level permission you granted in Step 3.

Security Best Practices

Use dedicated service accounts: Create a separate service account just for Pillar
Principle of least privilege: Only grant Storage Object Viewer—never Storage Admin or Editor
Rotate keys periodically: Delete and recreate keys every 90 days
Never commit keys to source control: Keep your JSON key file secure

Troubleshooting

"Permission denied" error

Verify the service account has the Storage Object Viewer role
Check that the bucket name is spelled correctly
Ensure the service account has access to the specific bucket

"Invalid credentials" error

Make sure you copied the entire JSON file contents, including the curly braces
Verify the JSON is valid (no missing characters)
Try generating a new key if the current one isn't working

Example JSON Key Structure

Your service account JSON key should look similar to this:

examples/data-sources/gcs-service-account-key.json

{
  "type": "service_account",
  "project_id": "your-project-id",
  "private_key_id": "abc123...",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "123456789",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/..."
}

Paste this entire JSON object into Pillar—do not extract individual fields.

Search documentation