Skip to content
This repository was archived by the owner on Mar 19, 2026. It is now read-only.

Add AWS S3 storage backend#534

Closed
dobesv wants to merge 1 commit intomicrosoft:mainfrom
dobesv:main
Closed

Add AWS S3 storage backend#534
dobesv wants to merge 1 commit intomicrosoft:mainfrom
dobesv:main

Conversation

@dobesv
Copy link
Copy Markdown

@dobesv dobesv commented May 7, 2025

Add support for cache storage to AWS S3.

Copy link
Copy Markdown
Member

@ecraig12345 ecraig12345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR, but most of the current users of lage/backfill won't want a tree of AWS dependencies pulled in by default.

An alternative would be to specify the AWS deps as peerDependencies that are marked as optional in peerDependenciesMeta (and update the provider to throw if the dep isn't present). The peer deps would need to be specified in both the backfill-cache and backfill packages.

Also, due to the potentially-missing dependency and the way lage's types rollup works, it would be best to avoid directly references the types in backfill-config--so clientConfig?: object in S3CacheStorageOptions.

Comment thread packages/cache/src/S3CacheStorage.ts Outdated
@dobesv
Copy link
Copy Markdown
Author

dobesv commented Aug 9, 2025

  • I moved the shared Azure/S3 classes to their own files so they can be shared between those two backends.
  • I changed the types in config so they don't need to reference the aws-sdk package types

@dobesv dobesv requested a review from ecraig12345 August 9, 2025 18:37
@ecraig12345
Copy link
Copy Markdown
Member

ecraig12345 commented Mar 19, 2026

Sorry for the extremely delayed response again. We just merged the backfill repo into the lage repo so it's easier to manage, so I'm about to archive this repo and its PRs.

Regarding this specific PR: I appreciate the contribution and recognize that it would likely be useful for others, but since our team has limited resources to devote to lage/backfill, I'm hesitant to take on maintenance of another backend that we don't use and wouldn't be able to test. As a bit of context, lage, backfill, beachball, and just (and a few others) were made by people from some teams in Office--so as much as we'd like to take a more robust, community-oriented approach to making great open source tools for both our teams and others, it's never been our primary responsibility and makes it hard to find the right balance.

As an alternative, I realized recently that since most users of lage don't use remote caching at all, probably a better approach is to handle all heavy backends including Azure as plugins (microsoft/lage#1037). Basically modifying the existing CustomStorageConfig so it takes a module name or path. At that point you could publish the S3 implementation as a separate npm package, and I'd be happy to add references to that in the relevant docs.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants