Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/go-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,11 @@ jobs:
- name: Run Tests via Justfile
env:
AWS_S3_BUCKET: testbucket
AWS_S3_REGION: us-east-1
AWS_REGION: us-east-1
AWS_DEFAULT_REGION: us-east-1
AWS_ACCESS_KEY_ID: minioadmin
AWS_SECRET_ACCESS_KEY: minioadmin
AWS_S3_ENDPOINT: http://127.0.0.1:9000
AWS_EC2_METADATA_DISABLED: "true"
run: nix develop --command just test-local
run: nix develop --command just test-local
12 changes: 10 additions & 2 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
# Variables
MINIO_CONTAINER_NAME := "simples3-minio"
MINIO_DATA_DIR := ".minio-data"
MINIO_ENDPOINT := "http://127.0.0.1:9000"
MINIO_REGION := "us-east-1"
MINIO_ACCESS_KEY := "minioadmin"
MINIO_SECRET_KEY := "minioadmin"
AWS_S3_BUCKET := "testbucket"

# Default command - lists all available recipes
Expand All @@ -17,7 +21,7 @@ test:
test-local: setup
@echo "🧪 Running tests with local MinIO..."
@sleep 2
@go test -v ./...
@AWS_S3_BUCKET={{AWS_S3_BUCKET}} AWS_S3_REGION={{MINIO_REGION}} AWS_REGION={{MINIO_REGION}} AWS_DEFAULT_REGION={{MINIO_REGION}} AWS_S3_ACCESS_KEY={{MINIO_ACCESS_KEY}} AWS_S3_SECRET_KEY={{MINIO_SECRET_KEY}} AWS_ACCESS_KEY_ID={{MINIO_ACCESS_KEY}} AWS_SECRET_ACCESS_KEY={{MINIO_SECRET_KEY}} AWS_S3_ENDPOINT={{MINIO_ENDPOINT}} AWS_EC2_METADATA_DISABLED=true SIMPLES3_CLI_INTEGRATION=1 go test -v ./...


# --- Go Module Management ---
Expand Down Expand Up @@ -61,7 +65,11 @@ minio-reset: minio-clean minio-up
setup: minio-up
@echo "⚙️ Setting up development environment..."
@sleep 3
@aws --endpoint-url http://127.0.0.1:9000/ s3 mb s3://{{AWS_S3_BUCKET}} || true
@if command -v aws >/dev/null 2>&1; then \
AWS_ACCESS_KEY_ID={{MINIO_ACCESS_KEY}} AWS_SECRET_ACCESS_KEY={{MINIO_SECRET_KEY}} AWS_DEFAULT_REGION={{MINIO_REGION}} aws --endpoint-url {{MINIO_ENDPOINT}} s3 mb s3://{{AWS_S3_BUCKET}} || true; \
else \
echo "ℹ️ aws CLI not installed; Go tests will bootstrap {{AWS_S3_BUCKET}} automatically"; \
fi
@echo "✅ Development environment ready!"

dev-env: setup
Expand Down
178 changes: 176 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,180 @@ using AWS Signature Version 4.
go get github.com/rhnvrm/simples3
```

## CLI

A first-cut `simples3` CLI is available under `cmd/simples3`.

### Build

```sh
go build ./cmd/simples3
```

### Auth and endpoint resolution

The CLI resolves credentials and region from:
1. explicit flags such as `--region` and `--endpoint`
2. standard AWS environment variables like `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`, and `AWS_PROFILE`
3. shared AWS config files (`~/.aws/credentials` and `~/.aws/config`)

Custom S3-compatible endpoints such as MinIO can be passed with `--endpoint`.

### Machine contract

- Default text output is for humans and may change.
- `--json` is the stable automation path. JSON success and error payloads are written to stdout.
- Exit codes:
- `0` success
- `1` runtime failure
- `2` usage or validation failure
- `3` partial failure for bulk commands when `--continue-on-error` is used
- JSON errors use this envelope:

```json
{
"command": "cp",
"ok": false,
"error": {
"type": "usage",
"message": "source and destination are required",
"exitCode": 2
}
}
```

Bulk commands (`cp`, `mv`, `rm`, `sync`) return per-operation JSON with a summary:

```json
{
"command": "sync",
"ok": true,
"summary": {
"total": 3,
"changed": 1,
"deleted": 1,
"unchanged": 1,
"dryRun": true,
"noop": false
},
"operations": [
{
"action": "sync",
"source": "./dist/app.js",
"destination": "s3://my-bucket/site/app.js",
"status": "would-sync"
}
]
}
```

### Commands

```text
ls list buckets or objects
cp copy local files and S3 objects
rm remove S3 objects and prefixes
mb make bucket
rb remove bucket
mv move local files and S3 objects
presign generate a presigned object URL
sync synchronize source to destination
tags get, set, or delete object tags
versioning get or set bucket versioning
versions list object versions and delete markers
lifecycle get, set, or delete bucket lifecycle configuration
acl get or set bucket or object ACLs
```

### Transfer and sync flags

`cp`, `mv`, `rm`, and `sync` share these automation-oriented flags:

- `--dry-run`
- `--continue-on-error`
- `--concurrency <n>`
- `--retries <n>`
- `--include <pattern>` / `--exclude <pattern>` where applicable

`sync --plan` implies `--dry-run` and includes unchanged entries in JSON output.

`sync --delete` is only allowed for recursive syncs.

`rm` has a few safety guards:
- `simples3 rm s3://bucket` is refused; use `rb` for buckets
- prefix-like targets require `--recursive`
- `--version-id` is only valid for single-object deletes

### Metadata and lifecycle inputs

- `tags set` accepts repeatable `--tag key=value` flags or `--tags-file PATH|-` with a JSON object map.
- `lifecycle set` accepts `--file PATH|-` with lower-camel JSON.
- `acl set` accepts either `--acl <canned-acl>` or `--policy-file PATH|-`.

Lifecycle document example:

```json
{
"rules": [
{
"id": "expire-logs",
"status": "Enabled",
"filter": {
"prefix": "logs/"
},
"expiration": {
"days": 30
}
}
]
}
```

ACL policy example:

```json
{
"owner": {
"id": "owner-id",
"displayName": "owner"
},
"grants": [
{
"grantee": {
"type": "CanonicalUser",
"id": "owner-id",
"displayName": "owner"
},
"permission": "FULL_CONTROL"
}
]
}
```

### Examples

```sh
simples3 ls
simples3 ls s3://my-bucket/prefix/
simples3 cp ./notes.txt s3://my-bucket/docs/
simples3 cp --recursive ./public s3://my-bucket/site/
simples3 cp --json --continue-on-error --concurrency 4 ./batch s3://my-bucket/inbox/
simples3 cp s3://my-bucket/archive/report.csv ./report.csv
simples3 rm --recursive s3://my-bucket/tmp/
simples3 rm --json --continue-on-error s3://my-bucket/tmp/
simples3 mv s3://my-bucket/inbox/file.txt s3://my-bucket/archive/file.txt
simples3 presign --expires 15m s3://my-bucket/path/file.txt
simples3 sync --plan --json ./dist s3://my-bucket/site/
simples3 sync --delete ./dist s3://my-bucket/site/
simples3 tags set --tag env=prod --tag team=platform s3://my-bucket/path/file.txt
simples3 tags get --json s3://my-bucket/path/file.txt
simples3 versioning set --status enabled s3://my-bucket
simples3 versions --json s3://my-bucket/path/file.txt
simples3 lifecycle set --file lifecycle.json s3://my-bucket
simples3 acl get --json s3://my-bucket
simples3 acl set --policy-file acl.json s3://my-bucket
```

## Quick Start

```go
Expand Down Expand Up @@ -775,7 +949,7 @@ The library includes comprehensive tests that run against a local MinIO instance
# Run all tests (without MinIO)
just test

# Run tests with local MinIO
# Run tests with local MinIO (includes CLI integration tests)
just test-local

# Run specific test
Expand Down Expand Up @@ -809,7 +983,7 @@ export AWS_S3_BUCKET="testbucket"

## Contributing

Contributions welcome! Check [ROADMAP.md](ROADMAP.md) for planned features. Please add tests and ensure `just test-local` passes before submitting PRs.
Contributions welcome! Check [ROADMAP.md](ROADMAP.md) for planned features. Please add tests and ensure `just test-local` passes before submitting PRs. That MinIO-backed run now includes the `cmd/simples3` CLI integration harness in addition to the library tests.

## Author

Expand Down
148 changes: 148 additions & 0 deletions cmd/simples3/command_acl.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
package main

import (
"fmt"

"github.com/rhnvrm/simples3"
)

type aclResult struct {
Command string `json:"command"`
OK bool `json:"ok"`
Target string `json:"target"`
Status string `json:"status"`
CannedACL string `json:"cannedAcl,omitempty"`
Policy *accessControlPolicyDocument `json:"policy,omitempty"`
}

func (rt *runtime) runACL(args []string) error {
if len(args) == 0 {
return usageErrorf("acl requires a subcommand: get or set")
}
switch args[0] {
case "get":
return rt.runACLGet(args[1:])
case "set":
return rt.runACLSet(args[1:])
default:
return usageErrorf("unknown acl subcommand %q", args[0])
}
}

func (rt *runtime) runACLGet(args []string) error {
var flags awsFlags
var versionID string
fs := newFlagSet("acl get", func() {
fmt.Fprint(rt.stderr, "Usage: simples3 acl get [--profile PROFILE] [--region REGION] [--endpoint URL] [--json] [--version-id ID] s3://bucket[/key]\n")
})
addAWSFlags(fs, &flags)
fs.StringVar(&versionID, "version-id", "", "object version ID")
if err := parseFlagSet(fs, args); err != nil {
return err
}
if fs.NArg() != 1 {
return usageErrorf("acl get requires exactly one bucket or object URI")
}
loc, err := parseLocation(fs.Arg(0))
if err != nil {
return err
}
if !loc.isS3() {
return usageErrorf("acl get target must be an S3 URI like s3://bucket or s3://bucket/key")
}
if versionID != "" && loc.key == "" {
return usageErrorf("--version-id requires an object target")
}
settings, err := rt.resolveAWSSettings(flags)
if err != nil {
return err
}
client := settings.newClient()
var policy simples3.AccessControlPolicy
if loc.key == "" {
policy, err = client.GetBucketAcl(loc.bucket)
} else {
policy, err = client.GetObjectAcl(simples3.GetObjectAclInput{Bucket: loc.bucket, ObjectKey: loc.key, VersionId: versionID})
}
if err != nil {
return err
}
doc := aclDocumentFromPolicy(policy)
result := aclResult{Command: "acl", OK: true, Target: loc.String(), Status: "loaded", Policy: &doc}
if flags.json {
return writeJSON(rt.stdout, result)
}
return writeJSON(rt.stdout, doc)
}

func (rt *runtime) runACLSet(args []string) error {
var flags awsFlags
var cannedACL string
var policyFile string
var versionID string
fs := newFlagSet("acl set", func() {
fmt.Fprint(rt.stderr, "Usage: simples3 acl set [--profile PROFILE] [--region REGION] [--endpoint URL] [--json] [--version-id ID] (--acl VALUE | --policy-file PATH|-) s3://bucket[/key]\n")
})
addAWSFlags(fs, &flags)
fs.StringVar(&cannedACL, "acl", "", "canned ACL value")
fs.StringVar(&policyFile, "policy-file", "", "JSON file or - for stdin with ACL policy")
fs.StringVar(&versionID, "version-id", "", "object version ID")
if err := parseFlagSet(fs, args); err != nil {
return err
}
if fs.NArg() != 1 {
return usageErrorf("acl set requires exactly one bucket or object URI")
}
loc, err := parseLocation(fs.Arg(0))
if err != nil {
return err
}
if !loc.isS3() {
return usageErrorf("acl set target must be an S3 URI like s3://bucket or s3://bucket/key")
}
if versionID != "" && loc.key == "" {
return usageErrorf("--version-id requires an object target")
}
if (cannedACL == "") == (policyFile == "") {
return usageErrorf("acl set requires exactly one of --acl or --policy-file")
}
settings, err := rt.resolveAWSSettings(flags)
if err != nil {
return err
}
client := settings.newClient()
result := aclResult{Command: "acl", OK: true, Target: loc.String(), Status: "updated", CannedACL: cannedACL}
if policyFile != "" {
var doc accessControlPolicyDocument
if err := decodeJSONSource(rt, policyFile, &doc); err != nil {
return err
}
result.Policy = &doc
if loc.key == "" {
err = client.PutBucketAcl(simples3.PutBucketAclInput{Bucket: loc.bucket, AccessControlPolicy: doc.toPolicy()})
} else {
err = client.PutObjectAcl(simples3.PutObjectAclInput{Bucket: loc.bucket, ObjectKey: loc.key, VersionId: versionID, AccessControlPolicy: doc.toPolicy()})
}
if err != nil {
return err
}
if flags.json {
return writeJSON(rt.stdout, result)
}
printLine(rt.stdout, "updated acl %s", loc.String())
return nil
}
if loc.key == "" {
err = client.PutBucketAcl(simples3.PutBucketAclInput{Bucket: loc.bucket, CannedACL: cannedACL})
} else {
err = client.PutObjectAcl(simples3.PutObjectAclInput{Bucket: loc.bucket, ObjectKey: loc.key, VersionId: versionID, CannedACL: cannedACL})
}
if err != nil {
return err
}
if flags.json {
return writeJSON(rt.stdout, result)
}
printLine(rt.stdout, "updated acl %s", loc.String())
return nil
}
Loading
Loading