Skip to content

feat: support S3-backed context stores #14

@beinan

Description

@beinan

Background

Context URIs are currently assumed to be local filesystem paths. Cloud-hosted agents need to persist shared context into remote object storage.

Proposal

  • Teach to detect URIs and initialize a Lance dataset backed by S3 via object store configuration.
  • Surface optional AWS credentials/configuration through environment variables and/or explicit parameters.
  • Update the Python path so S3 URIs are passed straight through and validated early.

Design Notes

  • Leverage Lance's existing integration (, ) rather than reimplementing S3 clients.
  • Allow per-run temporary credentials by honoring standard AWS env vars and the Lance object store configuration file.
  • Keep the URI format aligned with Lance's convention () so downstream tooling interoperates cleanly.

Acceptance Criteria

  • A context created with an S3 URI can append, checkout, and search records without regressions.
  • Python integration tests cover smoke usage against moto or a stubbed S3 endpoint.
  • Documentation explains required AWS permissions and credential discovery order.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions