Conversation
Opens a draft PR for plan + skipped e2e tests + docs review. No implementation yet. Full plan in PR body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
workflow with 1 step💻 Local Development
▲ Production (Vercel)
workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
❌ Some benchmark jobs failed:
Check the workflow run for details.
Check the workflow run for details. |
Adds the docs and skipped e2e tests for review *before* implementation.
No runtime changes — all four function shims (setAttribute, setAttributes,
getAttribute, getAttributes), the runs.list filter, and the keys/values
enumeration endpoints are referenced as if they exist, with the necessary
ambient declarations and `// @setup` blocks to keep the docs typecheck and
the workspace typecheck green.
Docs:
- docs/content/docs/foundations/attributes.mdx — concept guide.
- docs/content/docs/api-reference/workflow/{set-attribute,set-attributes,
get-attribute,get-attributes}.mdx — API reference for each export.
- Wires the new pages into the foundations + api-reference index Cards
and meta.json.
- packages/docs-typecheck/src/docs-globals.d.ts — temporary ambient
declarations marked `TODO(attributes V1)` for the unimplemented APIs
so the docs samples typecheck.
Tests:
- workbench/example/workflows/9_attributes.ts — workflow + step that
drive every code path the e2e suite asserts on.
- workbench/{nextjs-turbopack,nextjs-webpack,nitro-v3}/workflows/
9_attributes.ts — the standard symlink fan-out.
- packages/core/e2e/attributes.test.ts — full describe.skip suite
covering: round-trip, batch single-event, start({attributes}) initial
state, writer attribution (workflow vs step + attempt), retry
attribution, replay determinism, runs.list AND-filter, and
listAttributeKeys / listAttributeValues with prefix.
Each `as any` / `as unknown as ...` and ambient shim carries a
`TODO(attributes)` marker so the cleanup is greppable when the
implementation lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* origin/main: feat: serializable AbortController/AbortSignal (#1301) Auto-remove workflow packages from serverExternalPackages (#1481) Add missing changeset for Zod 4.4.x compatibility fix in @workflow/world (#1939) Push backport branch via GraphQL createCommitOnBranch for signed commits (#1937) Fix backport workflow opencode permission and surface AI failures (#1936) Restructure backport workflow with AI-driven decisions (#1934)
Mirrors the cancellation / AbortSignal pattern landed on main in #1301: attribute docs are v5-only since the underlying API will only ship in the v5 (workflow 5.x) branch. - Add `preRelease: true` to the frontmatter of all 5 attribute docs (foundations/attributes + 4 api-reference/workflow/*-attribute*). The v5 docs setup (docs/lib/geistdocs/version-source.ts) filters these out of the v4 sidebar tree and rewrites their URLs to /v5. - Drop the corresponding Cards from foundations/index.mdx and api-reference/workflow/index.mdx so v4 readers don't get broken links. v5 readers discover the pages via the sidebar. - Drop the inline "V1 preview" Callouts now that the v5 layout renders its own pre-release banner for preRelease pages. foundations/meta.json already lists `attributes` (alongside the freshly-merged `cancellation`); keep it there for the v5 sidebar. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workflow Attributes — V1 Plan
Context
We want a first-class attributes feature for workflow runs: small, plaintext, indexed, mutable key/value metadata that user code (workflow body or step body) can set at runtime and that observability (web, CLI, list APIs) can search and filter on.
This is an MVP toward the Attributes RFC (#132), but deliberately scoped down: no idempotency (
unique:true), no reserved$-namespace tags, no nested types — justRecord<string, string>. The shape of the event log, the schema, and the world API are designed so the omitted features can be layered on later without breaking changes.The user's goals, in priority order:
Scope (decided)
@workflow/coreSDK + world-local + world-vercel client wiring + tests.workflow-server, Tinybird, Web, and CLI come in follow-up phases. The plan still calls out the contract those later phases need to honor.attr_setevent records whether it came from the workflow body or a specific(stepId, attempt).start()accepts initial attributes in V1. Carried on therun_createdevent and materialized onto the run at birth.Design
Storage shape
Add an optional
attributes: Record<string, string>field directly on theWorkflowRunentity. Stored unencrypted, alongsideworkflowName/status/errorCode(other plaintext metadata fields). NOT a ref. Bounded by validated limits so it fits cheaply on the run row.V1 limits (validated in
@workflow/core, defended again in the world):$(reserved for future system keys)The run row holds only the current snapshot. The full history (who wrote what, when) lives in the event log. UIs that want a "who set this" annotation fetch events on demand.
Mutation model — event-sourced, with writer context
A single new event type, designed up-front to carry one-or-many changes per event:
The
changesarray meanssetAttributeandsetAttributes(the batch helper) share one event type:setAttribute('phase', 'done')→ emitsattr_set { changes: [{key:'phase', value:'done'}], writer }.setAttribute('phase', undefined)→ emitsattr_set { changes: [{key:'phase', value:null}], writer }.setAttributes({phase:'done', orderId:'ord_123'})→ emitsattr_set { changes: [{key:'phase',value:'done'}, {key:'orderId',value:'ord_123'}], writer }.No separate
attr_set_manyevent needed; the single-key case is just a one-element array. If the user passes an empty record the runtime short-circuits and emits nothing.Both workflow code and step code emit the same
attr_setevent; only thewriterdiscriminator differs.writerlets observability render "set by stepprocessOrder(attempt 2)" vs. "set by workflow body" in the timeline. It is never read by application code viagetAttribute()— that just returns the current snapshot string. Step retries that re-callsetAttributere-emit freshattr_setevents (no client-side coalescing); the event log faithfully records which attempt produced which write.The naming
attr_setfollows the existing{entity}_{action}event-naming convention (run_created,step_started,hook_disposed, etc.), abbreviated to keep the prefix in line with the others.The world atomically applies all
changesin a single event to the run'sattributesmap. Last-write-wins ordered by event arrival (the existing event-write transaction wraps the entity update, so the order is well-defined; in the rare case where ULID order and arrival order diverge the per-key snapshot follows arrival order). This shape gives us:attributesview from the event log, replaying each event'schangesarray in order).eventData.unique,eventData.expectedValue, etc. (per-change or per-event), and anattr_conflictevent (parallel tohook_conflict), without breaking the shape.Storage materialization in DDB-backed worlds
This applies to
workflow-server(Phase 2), but the world API is shaped to be cheap to implement against it.attributesis stored as a native DDB Map type on the run row, not a JSON-serialized string. Concretely, inworkflow-server/lib/data/electrodb.ts:This is the prerequisite for the rest:
Single-event, multi-change apply: each
attr_setevent becomes oneUpdateItemwhoseUpdateExpressionis built from the event'schangesarray. Sets and unsets fold into one expression:Atomic on the run row. The existing event-write transaction (events table + run row) wraps it; no extra
TransactWriteItemsbeyond what every other event type already pays. DDB's per-request limits (~256 expression names / values) are well above our V1 cap of 64 attributes per run, so even a full-replacesetAttributesof every key fits in a single request.No read-modify-write: the merge happens inside DDB's expression engine. No optimistic locking needed for the simple LWW-by-arrival semantic V1 ships with.
Server gets the full post-merge map for free:
ReturnValues: 'ALL_NEW'on theUpdateItemreturns the entire run row, including the freshly-mergedattributes. The server uses that to emit Tinybird usage facts (Phase 3) without an extra read.Migration: existing rows have no
attributesfield. Newrun_createdevents seedattributes: {}(orattributes: <init>if the start call provided one). On read, missing field → empty record.A future tighter ordering guarantee (per-key ULID stamps + conditional update) can be added later without changing the event log shape.
Tinybird/ClickHouse path (Phase 3, contracted in this plan)
The world API doesn't dictate the wire format to Tinybird, but we record the intended shape so the server team has a target:
attr_setevent (the usage fact may carry multiplechangesif the event was a batch):{ teamId, projectId, runId, eventId, ts, changes: Array<{key, value: string|null}>, writer_type, writer_step_id?, writer_attempt?, attributes_full: Record<string,string> }.attributes_fullis the post-merge snapshot from DDBALL_NEW. Storing it per-event makes ClickHouse queries trivial: "latest row perrunId, filter onattributes_full[key] = value" with no joins.{ runId, key, value, isDeleted, version=eventId }into aReplacingMergeTree(version)keyed on(runId, key)); the trade is harder multi-key AND-filter queries. We'd default to the denormalized full-snapshot table.This means the event log stays delta-only even though Tinybird stores full snapshots — the server is the only place that needs the merge logic, and DDB does it for it.
Client-side write semantics — always delta, server emits async
Final shape:
attr_setevent whoseeventData.changesis one or more{ key, value }entries. There is no two-mode protocol; workflow and step writers are uniform on the wire and only differ via thewriterdiscriminator.UpdateItem(SET #attrs.#k = :vorREMOVE #attrs.#k) withReturnValues: 'ALL_NEW', which returns the freshly-merged full attributes map in the same RPC.attributes_fullis fired off-thread viawaitUntil/ the existingworkflowUsageTrackerqueue (Phase 3) — it does not block the response back to the runtime/SDK.This means the runtime sees the same latency as any other lightweight event (e.g.
step_started), and the analytics path is decoupled. If usage-fact emission fails or lags, the run state in DDB is still authoritative.The client never has to compute or ship a full snapshot; whether the user calls
setAttributeonce or 100 times in a row, every event is bounded in size. High-volume use is still rate-bounded by per-event DDB cost, but the wire / replay shape stays simple.Initial attributes at
start()start(workflow, input, { attributes })accepts anattributes?: Record<string,string>option, validated client-side with the same rules assetAttribute. They are carried onrun_createdevent:The world materializes them directly onto the new run — no
attr_setevents are emitted for them (they are the run's birth state, not a mutation). This keeps the event log minimal for the common case where attributes are only set at start.Runtime SDK
Four new exports from
@workflow/core(re-exported byworkflow):setAttribute(k, v)is implemented assetAttributes({[k]: v})so they share the wire path; the convenience name is just sugar for the common case.Available in both workflow context and step context. Implementation paths:
Step context (Node.js):
setAttributescallsworld.events.create(runId, { eventType: 'attr_set', eventData: { changes, writer: { type:'step', stepId, attempt } } })directly. The step'scontextStoragecarries a local mutableattributesmap seeded from the run snapshot at step start, so reads within the step body see local writes (read-your-writes).Workflow context (sandboxed VM): bridged through the controller the same way
sleep/createHookare. The workflow VM holds a localattributesmap rebuilt from the event log on replay;setAttributesenqueues an event request (withwriter: { type: 'workflow' }) through the host bridge, then optimistically updates the local map.Validation (length,
$prefix, total count after merge) lives in a shared helper in@workflow/coreso both contexts produce identical errors. Violations throwFatalErrorand are checked before emitting the event so an invalid batch never lands on the wire.World interface additions
Extend
@workflow/worldStorage.runs:Filter is folded into the existing
runs.listrather than a separatelistByAttribute, so it composes withworkflowName/status/ pagination.World implementations (Phase 1)
@workflow/world-local: full implementation. The event-application pipeline gains anattr_sethandler (iteratechanges, set or delete onrun.attributes);run_createdhonors initialattributes.runs.listfilter is an in-memoryObject.entriesmatch;listAttributeKeys/listAttributeValuesscan all run files and aggregate.@workflow/world-vercel: client-side HTTP wrappers only. Sendsattributes[key]=valuequery params onruns.list; calls new endpoints for keys/values. The actual server impl lands in Phase 2 (workflow-server).The wire contract for Phase 2 / Phase 3 is documented in this PR's
world-vercelREADME so the server team has a clear handoff:GET /v3/runs?attributes[key]=value&...— extends existing list endpoint.GET /v3/attributes/keys?prefix=...&limit=...&cursor=...GET /v3/attributes/values?key=K&prefix=...&limit=...&cursor=...POST /v3/runs/:id/eventsalready accepts new event types —attr_setslots in.POST /v3/runsrun_createdbody extended withattributes.What V1 does NOT include
unique: true/ atomic conflict semantics.$-prefixed system keys (we just block them in V1 to keep the namespace clean).packages/web/packages/cli.Critical files
@workflow/worldpackages/world/src/runs.ts— addattributes?: Record<string,string>toWorkflowRunBaseSchema; addattributesfilter toListWorkflowRunsParams; addListAttributeKeysParams,ListAttributeValuesParams, response types.packages/world/src/events.ts— addattr_settoEventTypeSchema+ newAttrSetEventSchema(withchanges: Array<{key,value:string|null}>and thewriterdiscriminator) in the event union. ExtendRunCreatedEventSchemawith optionalattributes. Note: NOT added toEVENT_DATA_REF_FIELDSsince the data is plaintext.packages/world/src/interfaces.ts— extendStorage.runswithlistAttributeKeys/listAttributeValues.@workflow/corepackages/core/src/attributes.ts—setAttribute,setAttributes,getAttribute,getAttributes, plusvalidateAttributeKey/Valueand the shared "buildattr_seteventData from a record + writer" helper. Dispatches to a context-aware backend (step vs workflow).packages/core/src/index.ts— export the four functions.packages/core/src/step/context-storage.ts— extend the step context with a mutableattributesmap (seeded from the run snapshot at step start) and an event-emittingsetAttributesimpl that tags writer ={ type:'step', stepId, attempt }.packages/core/src/workflow/attributes.ts— VM-side bridge to the controller, mirroringworkflow/sleep.ts. Tags writer ={ type:'workflow' }.packages/core/src/runtime.ts/runtime/start.ts/runtime/resume-hook.ts— handleattr_setevents when replaying the event log to reconstruct the workflow VM's attributes view (apply the fullchangesarray); threadstart(){ attributes }through to therun_createdevent payload.@workflow/world-localpackages/world-local/src/storage.ts— handleattr_setevent in the event-application pipeline (apply each change inchangesarray, set or remove onrun.attributes); honorattributesfromrun_created; extendlistfilter to honorattributes.@workflow/world-vercelpackages/world-vercel/src/runs.ts— extendlistWorkflowRunsto sendattributes[key]=valuequery params; addlistAttributeKeys/listAttributeValuesHTTP wrappers.packages/world-vercel/src/index.ts— wire new methods into the World object.Reused utilities (do not re-implement)
world.events.createis already the universal event-emitting path —attr_setslots in directly.contextStorage(packages/core/src/step/context-storage.ts) is the right home for step-side per-call state, including thestepId/attemptneeded for writer attribution.WORKFLOW_CONTEXT_SYMBOL/ VM-bridge pattern inpackages/core/src/workflow/sleep.tsandworkflow/create-hook.tsis the template for the workflow-VM side.FatalErrorfrom@workflow/errorsfor validation failures.EVENT_DATA_REF_FIELDSinevents.ts— leave unchanged; attributes are plaintext.Test plan
packages/corefor:$prefix, max count) producingFatalError. Total-count validation must consider the post-merge state, not just the incoming batch.setAttributeandsetAttributes.setAttributes({})is a no-op (no event emitted).run_created.attributes+ multipleattr_setevents, including events with multi-change arrays).writer:{type:'step',...}; from workflow body carrywriter:{type:'workflow'}.attr_setevent on the second attempt withwriter.attempt = 2.setAttribute → events.list → run.attributesround-trip.setAttributes({a, b}) → events.listproduces a singleattr_setevent with twochangesentries.start({ attributes })materializes onto the run with noattr_setevents.runs.list({ attributes })filter (AND semantics, multiple matches, no match).listAttributeKeys/listAttributeValuesenumeration with prefix.workbench/nextjs-turbopack: a workflow thatstart(input, { attributes: { tenant: 't1' } }).setAttribute('phase', 'init'), awaits a step that callssetAttributes({ orderId: 'ord_123', stepKind: 'process' }), thensetAttribute('phase', 'done').world.runs.get(runId)thatattributes === { tenant: 't1', phase: 'done', orderId: 'ord_123', stepKind: 'process' }.attr_setevent with twochangesentries (world.events.list).world.runs.list({ attributes: { orderId: 'ord_123' } })returns the run.world.runs.listAttributeKeys()includestenant,phase,orderId,stepKind.Verification
pnpm typecheck && pnpm lint && pnpm testfrom the repo root.workbench/nextjs-turbopackper CLAUDE.md instructions, hitting the test outline above.pnpm changeset addwith minor bumps for@workflow/core,@workflow/world,@workflow/world-local,@workflow/world-vercel, andworkflow(fixed with core).Completing the unfinished spec message
The user's draft fragment, completed: