Skip to content

GCS_SETUP_GUIDE.md

Latest commit

 

History

History
228 lines (160 loc) · 6 KB

GCS_SETUP_GUIDE.md

File metadata and controls

228 lines (160 loc) · 6 KB

Google Cloud Storage Setup Guide

This guide will help you set up Google Cloud Storage for your SREF Library images.

Why Google Cloud Storage?

✅ Free Tier: 5GB storage + 1GB egress/month
✅ Global CDN: Fast worldwide delivery
✅ Reliable: Google's enterprise infrastructure
✅ Scalable: Grows with your needs
✅ Cost Effective: Pay only for what you use

Prerequisites

  1. Google Account - Sign up at cloud.google.com
  2. Python 3.7+ - Already installed
  3. Google Cloud SDK (optional) - For easier setup

Step 1: Create Google Cloud Project

  1. Go to Google Cloud Console
  2. Click "Select a project" → "New Project"
  3. Name: sref-library (or your preferred name)
  4. Click "Create"

Step 2: Enable Cloud Storage API

  1. In your project, go to "APIs & Services" → "Library"
  2. Search for "Cloud Storage API"
  3. Click "Enable"

Step 3: Create Storage Bucket

  1. Go to "Cloud Storage" → "Buckets"
  2. Click "Create bucket"
  3. Bucket name: sref-library-images (must be globally unique)
  4. Location: Choose closest to your users (e.g., us-central1)
  5. Storage class: Standard
  6. Access control: Uniform
  7. Click "Create"

Step 4: Set Up Authentication

Option A: Service Account (Recommended for Production)

  1. Go to "IAM & Admin" → "Service Accounts"
  2. Click "Create Service Account"
  3. Name: sref-uploader
  4. Description: Service account for uploading SREF images
  5. Click "Create and Continue"
  6. Role: Storage Admin
  7. Click "Continue" → "Done"
  8. Click on the service account → "Keys" tab
  9. Click "Add Key" → "Create new key"
  10. Choose "JSON" → "Create"
  11. Download the JSON file and save it securely

Option B: User Authentication (Easier for Development)

  1. Install Google Cloud SDK: cloud.google.com/sdk/docs/install
  2. Run: gcloud auth application-default login
  3. This uses your user credentials automatically

Step 5: Install Dependencies

# Install Google Cloud Storage SDK
pip install google-cloud-storage

Step 6: Configure Environment Variables

Create a .env file:

# For Service Account (Option A)
GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/service-account-key.json
GCS_BUCKET_NAME=sref-library-images

# For User Authentication (Option B) - just set bucket name
GCS_BUCKET_NAME=sref-library-images

Step 7: Make Bucket Publicly Readable

Your images need to be publicly accessible. Run the provided script:

# Edit the script first to set your bucket name
nano make_bucket_public.sh

# Then run it
./make_bucket_public.sh

Or manually:

  1. Go to your bucket in Cloud Console
  2. Click "Permissions" tab
  3. Click "Add Principal"
  4. New principal: allUsers
  5. Role: Storage Object Viewer
  6. Click "Save"

Step 8: Upload Images

# Upload all images to Google Cloud Storage
python upload_to_gcs.py

This will:

  • Upload all 2,320 JPG files to your GCS bucket
  • Create gcs_image_mapping.json with URL mappings
  • Show progress for each image

Step 9: Deploy to Vercel

# Deploy your app
vercel --prod

How It Works

Local Development

  • App uses local images from static/images/
  • Fast local development

Production (Vercel)

  • App uses GCS URLs from gcs_image_mapping.json
  • Images served from Google's global CDN
  • Super fast worldwide delivery

Image URL Format

  • Local: static/images/101236067_01.jpg
  • GCS: https://storage.googleapis.com/sref-library-images/sref-images/101236067_01.jpg

File Structure After Setup

sref-library/
├── app.py                          # Updated to use GCS URLs
├── gcs_image_mapping.json         # Generated by upload script
├── upload_to_gcs.py              # Upload script
├── make_bucket_public.sh         # Bucket permissions script
├── .env                          # Your credentials (keep secret!)
├── static/
│   ├── css/
│   ├── js/
│   └── images/                   # Excluded from deployment
└── sref_analysis/               # Search index (included)

Troubleshooting

"Bucket not found"

  • Verify bucket name is correct
  • Check you're in the right Google Cloud project
  • Ensure bucket exists in the console

"Permission denied"

  • Check service account has Storage Admin role
  • Verify JSON key file path is correct
  • For user auth, run gcloud auth application-default login

"Images not loading"

  • Ensure bucket is publicly readable
  • Check the public access policy
  • Verify image URLs in the mapping file

Upload fails

  • Check your internet connection
  • Verify Google Cloud credentials
  • Ensure you have sufficient quota

Cost Estimation

For your 46MB of images:

Free Tier (First 12 months):

  • Storage: 5GB free (your 46MB fits easily)
  • Egress: 1GB/month free
  • Total: $0/month

After Free Tier:

  • Storage: ~$0.01/month (46MB × $0.020/GB)
  • Egress: ~$0.15/month (1GB × $0.12/GB)
  • Total: ~$0.16/month

Benefits

✅ Solves Size Issue - Images no longer count toward 250MB limit
✅ Global CDN - Fast image delivery worldwide
✅ Cost Effective - Stays within free tier for months
✅ Reliable - Google's enterprise infrastructure
✅ Scalable - Grows with your needs
✅ Secure - Fine-grained access control

Next Steps

  1. Run the upload script
  2. Test locally to ensure images load
  3. Deploy to Vercel
  4. Verify images work in production

Your app will now have fast, globally distributed image delivery from Google Cloud Storage! 🚀

Optional: Cloud CDN

For even faster delivery, you can add Google Cloud CDN:

  1. Go to "Network Services" → "Cloud CDN"
  2. Create a load balancer with your GCS bucket as backend
  3. This provides additional caching and optimization

This is optional but recommended for high-traffic applications.