Skip to content

fix(storage): rate limit bucket deletion in cleanup#5612

Open
abhinavgautam01 wants to merge 1 commit intogoogleapis:mainfrom
abhinavgautam01:fix/rate-limit-bucket-deletion
Open

fix(storage): rate limit bucket deletion in cleanup#5612
abhinavgautam01 wants to merge 1 commit intogoogleapis:mainfrom
abhinavgautam01:fix/rate-limit-bucket-deletion

Conversation

@abhinavgautam01
Copy link
Copy Markdown

Fixes #5219

Problem

When there are many stale buckets in integration tests, the cleanup process
was deleting them in parallel and exceeding GCP's Storage API rate limit
(approximately one request every two seconds).

Solution

Serialize bucket deletion by removing parallel spawning (tokio::spawn and
join_all) and instead delete buckets sequentially with a 2-second delay
between each deletion to respect the API rate limit.

Changes

  • Changed cleanup_stale_buckets() to delete buckets sequentially
  • Added 2-second delay between bucket deletions
  • Maintains proper error handling for each deletion attempt
  • Delay is not applied after the last bucket (no unnecessary wait)

Testing

This change should prevent rate limit errors during stale bucket cleanup
in integration tests.

@abhinavgautam01 abhinavgautam01 requested review from a team as code owners May 7, 2026 11:15
@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label May 7, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the cleanup_stale_buckets function in the storage examples to serialize bucket deletions. The implementation replaces concurrent task spawning with a sequential loop that introduces a 2-second delay between deletions to comply with GCP rate limits. Feedback was provided to replace println! calls with the tracing crate to align with the repository's structured logging standards.

Comment thread src/storage/examples/src/lib.rs
   Serialize bucket deletion to respect GCP API rate limit (~1 request per 2 seconds).
   Uses structured logging with tracing crate for consistency with repository standards.
@abhinavgautam01 abhinavgautam01 force-pushed the fix/rate-limit-bucket-deletion branch from 9c33688 to 1294c4a Compare May 7, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: rate limit storage bucket deletion

1 participant