From 0a088584daa03ed217e6e68979dd8c92d369a008 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Oct 2025 23:58:24 +0000 Subject: [PATCH 1/3] docs: add unified logger epic planning and task breakdown MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Create comprehensive task epic for unified event-driven logging framework covering client/server implementation, security, schema validation, and testing infrastructure. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- plan.md | 13 +++- tasks/unified-logger-epic.md | 147 +++++++++++++++++++++++++++++++++++ 2 files changed, 157 insertions(+), 3 deletions(-) create mode 100644 tasks/unified-logger-epic.md diff --git a/plan.md b/plan.md index 7f53e45..3856748 100644 --- a/plan.md +++ b/plan.md @@ -11,7 +11,14 @@ ### 📋 Context7 GitHub Action Epic -**Status**: 📋 PLANNED -**File**: [`tasks/context7-github-action-epic.md`](./tasks/context7-github-action-epic.md) -**Goal**: Integrate Context7 GitHub Action to automatically maintain up-to-date code documentation for LLMs and AI code editors +**Status**: 📋 PLANNED +**File**: [`tasks/context7-github-action-epic.md`](./tasks/context7-github-action-epic.md) +**Goal**: Integrate Context7 GitHub Action to automatically maintain up-to-date code documentation for LLMs and AI code editors **Tasks**: 6 tasks (configuration, workflow creation, API integration, release integration, testing, documentation) + +### 📋 Unified Logger Epic + +**Status**: 📋 PLANNED +**File**: [`tasks/unified-logger-epic.md`](./tasks/unified-logger-epic.md) +**Goal**: Create event-driven logging framework for unified telemetry across client and server +**Tasks**: 9 tasks (core infrastructure, client implementation, server implementation, event configuration, security/privacy, transport, schema validation, testing, documentation) diff --git a/tasks/unified-logger-epic.md b/tasks/unified-logger-epic.md new file mode 100644 index 0000000..afdefc7 --- /dev/null +++ b/tasks/unified-logger-epic.md @@ -0,0 +1,147 @@ +# Unified Logger Epic + +**Status**: 📋 PLANNED +**Goal**: Create event-driven logging framework for unified telemetry across client and server + +## Overview + +Developers need a unified logging solution that works consistently across client and server environments to enable reliable telemetry, debugging, and monitoring without coupling to specific dispatch implementations. This logger subscribes to framework events and handles sampling, sanitization, batching, and transport with built-in offline resilience and GDPR compliance. + +--- + +## Core Logger Infrastructure + +Create base logger types, interfaces, and event subscription mechanism. + +**Requirements**: +- Given framework emits events via Observable, should subscribe and process matching events +- Given logger receives event, should apply global and per-event configuration +- Given serialization or sanitization fails, should log error to console and continue processing other events +- Given logger is created with invalid configuration, should throw descriptive error + +--- + +## Client Logger Implementation + +Implement browser-based logger with localStorage buffering and network resilience. + +**Requirements**: +- Given browser is online, should flush batched events immediately in background +- Given browser is offline, should buffer events to localStorage without data loss +- Given browser reconnects after offline period, should auto-flush pooled buffers +- Given localStorage quota exceeded, should evict oldest events FIFO +- Given navigator.sendBeacon is available, should prefer it over fetch for reliability +- Given transport fails with retryable error, should retry with exponential backoff and jitter +- Given event batches reach size or time threshold, should flush automatically + +--- + +## Server Logger Implementation + +Implement Node.js logger with middleware integration. + +**Requirements**: +- Given createLogger is called on server, should return logger with attach function +- Given attach is called with request and response, should set response.locals.logger +- Given server logger receives event, should process without localStorage buffering +- Given server environment, should support console and custom endpoint destinations + +--- + +## Event Configuration System + +Implement per-event sampling, sanitization, and serialization overrides. + +**Requirements**: +- Given event type has custom sampler, should apply RxJS operator to event stream +- Given event type has custom sanitizer, should scrub payload before serialization +- Given event type has custom serializer, should use it instead of global serializer +- Given event type has no custom config, should fall back to global defaults +- Given event configuration includes invalid RxJS operator, should fail fast with clear error + +--- + +## Security and Privacy Layer + +Implement sanitizers, consent checking, and PII scrubbing utilities. + +**Requirements**: +- Given payload contains headers, should apply headerSanitizer before logging +- Given payload contains request/response data, should apply payloadSanitizer before logging +- Given event is non-essential and consent denied, should skip logging +- Given event is essential, should log regardless of consent status +- Given PII detection patterns match, should redact before storage or transport +- Given developer uses logger, should have clear documentation on PII scrubbing requirements + +--- + +## Transport Layer + +Implement batching, retry logic, and idempotent delivery. + +**Requirements**: +- Given batch reaches batchSizeMax events OR 64KB bytes, should flush immediately +- Given batch hasn't flushed in flushIntervalMs, should flush on timer +- Given POST to endpoint returns 4xx client error, should not retry +- Given POST to endpoint returns 5xx server error, should retry with backoff +- Given POST to endpoint returns network error, should retry up to max attempts +- Given server receives duplicate eventId, should respond 204 without processing +- Given client sends events to /api/events/[id], should validate origin, content-type, and schema +- Given request body exceeds maxByteLength, should reject with 413 +- Given event timestamp is outside skewWindow, should reject with 400 + +--- + +## Schema Validation + +Implement JSON schema validation for event types using Ajv. + +**Requirements**: +- Given event is posted to server, should validate against registered schema +- Given event fails schema validation, should reject with detailed error +- Given event type has no schema, should apply default event schema +- Given schema uses Ajv validator, should compile schemas once at initialization +- Given developer registers event schema, should validate schema definition itself + +--- + +## Testing Infrastructure + +Create test utilities, mocks, and adapters for logger testing. + +**Requirements**: +- Given tests need storage isolation, should provide mock storage adapter +- Given tests need dispatch isolation, should provide spy/mock dispatch +- Given tests need deterministic timing, should inject clock function +- Given tests run in parallel, should not share localStorage state +- Given logger is used in tests, should expose dispose/cleanup method + +--- + +## Documentation and Examples + +Document integration patterns, PII guidelines, and usage examples. + +**Requirements**: +- Given developer integrates logger, should have clear client and server setup examples +- Given developer needs to scrub PII, should have comprehensive PII detection guidelines +- Given developer configures events, should have RxJS operator examples for common patterns +- Given developer handles retention, should have documentation referencing GDPR requirements +- Given developer needs JSON schemas, should have examples for Redux-style action objects + +--- + +## Questions + +### Framework Interfaces +- Should we create TypeScript definition files (.d.ts) for the framework interfaces (events$, dispatch, useDispatch) even though implementation is out of scope? + +### JSON Schema Examples +- Would you like me to include Ajv schema examples for common event types in the task epic, or should that be part of the implementation documentation? + +### Essential vs Non-Essential Events +- Should we add a marker field like `event.metadata.essential: boolean` to distinguish event types, or handle this purely through configuration? + +### Error Handling +- For serialization/sanitization failures, should we provide a configurable `onError` callback, or always default to console.error? + From 1a121028d14667cfaec965fd897f8ff4ba58b08f Mon Sep 17 00:00:00 2001 From: Eric Elliott Date: Thu, 23 Oct 2025 17:57:11 -0700 Subject: [PATCH 2/3] Revise unified logger requirements and behavior Updated requirements and refined logging behavior for both client and server. Modified event handling and transport logic, including consent checks and error handling. --- tasks/unified-logger-epic.md | 387 +++++++++++++++++++++++++++++++---- 1 file changed, 351 insertions(+), 36 deletions(-) diff --git a/tasks/unified-logger-epic.md b/tasks/unified-logger-epic.md index afdefc7..a0218d7 100644 --- a/tasks/unified-logger-epic.md +++ b/tasks/unified-logger-epic.md @@ -31,7 +31,7 @@ Implement browser-based logger with localStorage buffering and network resilienc - Given browser reconnects after offline period, should auto-flush pooled buffers - Given localStorage quota exceeded, should evict oldest events FIFO - Given navigator.sendBeacon is available, should prefer it over fetch for reliability -- Given transport fails with retryable error, should retry with exponential backoff and jitter +- Given transport fails with retryable error, should ignore (client telemetry is fire-and-forget) - Given event batches reach size or time threshold, should flush automatically --- @@ -41,10 +41,10 @@ Implement browser-based logger with localStorage buffering and network resilienc Implement Node.js logger with middleware integration. **Requirements**: -- Given createLogger is called on server, should return logger with attach function -- Given attach is called with request and response, should set response.locals.logger -- Given server logger receives event, should process without localStorage buffering -- Given server environment, should support console and custom endpoint destinations +- Given createLogger is called on server, should return logger +- Given withLogger is called with request and response, should set response.locals.logger +- Given server logger receives event after subscribing to `events$` observable, should call logger.log() or logger.error() which should internally dispatch to the transport logger (default: console.log/console.error) +- Given server environment, should support console and custom transport --- @@ -66,29 +66,34 @@ Implement per-event sampling, sanitization, and serialization overrides. Implement sanitizers, consent checking, and PII scrubbing utilities. **Requirements**: +Server: - Given payload contains headers, should apply headerSanitizer before logging - Given payload contains request/response data, should apply payloadSanitizer before logging -- Given event is non-essential and consent denied, should skip logging -- Given event is essential, should log regardless of consent status -- Given PII detection patterns match, should redact before storage or transport -- Given developer uses logger, should have clear documentation on PII scrubbing requirements + +Client: +- Given telemetry consent denied, should skip logging + +Server: +- Given PII detection patterns match, should redact (scrub) before storage or transport +- Given developer wants to use logger, should have clear documentation on PII GDPR-compliant scrubbing --- -## Transport Layer +## Client Transport Layer -Implement batching, retry logic, and idempotent delivery. +Implement batching and idempotent delivery. **Requirements**: -- Given batch reaches batchSizeMax events OR 64KB bytes, should flush immediately -- Given batch hasn't flushed in flushIntervalMs, should flush on timer -- Given POST to endpoint returns 4xx client error, should not retry -- Given POST to endpoint returns 5xx server error, should retry with backoff -- Given POST to endpoint returns network error, should retry up to max attempts +- Given we are online and queued events reaches batchSizeMax events, should flush immediately +- Given we are online and batch hasn't flushed in flushIntervalMs, should flush on timer +- Given POST fails, log error to client console and ignore (fire and forget client) + +Server: - Given server receives duplicate eventId, should respond 204 without processing - Given client sends events to /api/events/[id], should validate origin, content-type, and schema - Given request body exceeds maxByteLength, should reject with 413 -- Given event timestamp is outside skewWindow, should reject with 400 +- Given event timestamp is outside skewWindow, should reject with 400 (default skewWindow to 24 hours) +- Given event timestamp is outside the futureSkewWindow, should reject with 400 (default futureSkewWindow: 1h:5m) --- @@ -105,16 +110,13 @@ Implement JSON schema validation for event types using Ajv. --- -## Testing Infrastructure +## Testing Tips -Create test utilities, mocks, and adapters for logger testing. - -**Requirements**: -- Given tests need storage isolation, should provide mock storage adapter -- Given tests need dispatch isolation, should provide spy/mock dispatch -- Given tests need deterministic timing, should inject clock function -- Given tests run in parallel, should not share localStorage state -- Given logger is used in tests, should expose dispose/cleanup method +- Spies and stubs: vi.fn and vi.spyOn + Vitest ships tinyspy under the hood. Simple, fast, and no extra deps. +- Module mocking: vi.mock with vi.importActual for partial mocks + Works cleanly with ESM. Avoid require. +- Timers: vi.useFakeTimers and vi.setSystemTime --- @@ -123,7 +125,7 @@ Create test utilities, mocks, and adapters for logger testing. Document integration patterns, PII guidelines, and usage examples. **Requirements**: -- Given developer integrates logger, should have clear client and server setup examples +- Given developer wants to use the logger, should have clear client and server setup examples - Given developer needs to scrub PII, should have comprehensive PII detection guidelines - Given developer configures events, should have RxJS operator examples for common patterns - Given developer handles retention, should have documentation referencing GDPR requirements @@ -131,17 +133,330 @@ Document integration patterns, PII guidelines, and usage examples. --- -## Questions - ### Framework Interfaces -- Should we create TypeScript definition files (.d.ts) for the framework interfaces (events$, dispatch, useDispatch) even though implementation is out of scope? +- Out of scope. + +--- + +Original issue: + +# Feature Request: Unified Event Driven Logger for AIDD (`aidd/logger`) + +## Summary +Create a unified event driven logging and telemetry framework for client and server in AIDD. + +The framework provides dispatch. Its implementation is out-of-scope for this issue. + +On client use `aidd/client/useDispatch`. On server use `response.locals.dispatch`. +The logger subscribes to the framework event stream and applies per event rules for sampling, sanitization, serialization, batching, and transport. + +Client uses localStorage for write through buffering. When online it flushes immediately. When offline it pools buffers and auto flushes on reconnection. + +--- + +## Core design + +```sudo +// Provided by the broader aidd framework (implementation is out-of-scope) +events$ // rxjs Observable of all dispatched events +useDispatch() // client hook +response.locals.dispatch // server per request + +// Provided by the logger +createLogger(options) +``` + +All logging originates from framework `dispatch(event)`. +The logger subscribes to `events$` and routes matching events to storage and transport. + +--- + +## Client API + +```js +import { createLogger } from 'aidd/logger/client.js'; +import { useDispatch } from 'aidd/client/use-dispatch.js'; + +const { log, withLogger } = createLogger(options); + +// dev code uses the framework dispatch +const dispatch = useDispatch(); + +dispatch(reportPageview({ page: 'home' }); +``` + +Behavior +- Monitor online and offline state +- Write through to localStorage on every log +- If online then flush immediately in background +- If offline then append to pooled buffers and auto flush on reconnection +- Batch events for network efficiency +- Prefer navigator.sendBeacon with fetch POST fallback +- Retry with backoff and jitter +- Evict oldest when storage cap reached + +--- + +## Server API + +```sudo +import { createLogger } from 'aidd/logger/server.js' + +const withLogger = createLogger(options) + +// withLogger attaches `logger` to +response.locals.dispatch +``` + +Behavior +- No client buffering logic on the server side. + +--- + +## Configuration + +```sudo +createLogger(options) + +options + endpoint // POST target or internal queue + payloadSanitizer // (any) => any + headerSanitizer // (headers) => headers + serializer // (any) => string + batchSizeMax = 50 + flushIntervalMs // background flush tick when online + consentProvider // () => { analytics: bool } + getIds // () => { userPseudoId requestId } + level // default info + sampler = takeEveryglobal default sampler # Feature Request: Unified Event Driven Logger for AIDD (`aidd/logger`) + +## Summary +Create a unified event driven logging and telemetry framework for client and server in AIDD. +The logger does not create `dispatch`. The framework provides dispatch. +On client use `aidd/client/useDispatch`. On server use `response.locals.dispatch`. +The logger subscribes to the framework event stream and applies per event rules for sampling, sanitization, serialization, batching, and transport. +Client uses localStorage for write through buffering. When online it flushes immediately. When offline it pools buffers and auto flushes on reconnection. + +--- + +## Core design + +```sudo +// Provided by the framework +events$ // rxjs Observable of all dispatched events +useDispatch() // client hook +response.locals.dispatch // server per request + +// Provided by the logger +createLogger(options) +``` + +All logging originates from framework `dispatch(event)`. +The logger subscribes to `events$` and routes matching events to storage and transport. + +--- + +## Client API + +```sudo +import { createLogger } from 'aidd/logger/client.js' +import { useDispatch } from 'aidd/client/useDispatch.js' + +const { log, withLogger } = createLogger(options) + +// dev code uses the framework dispatch +const dispatch = useDispatch() + +dispatch({ type: 'page_view', payload: { message: 'home', timeStamp: Date.now() } }) +log('client ready') +``` + +Behavior +- Monitor online and offline state +- Write through to localStorage on every log +- If online then flush immediately in background +- If offline then append to pooled buffers and auto flush on reconnection +- Batch events for network efficiency +- Prefer navigator.sendBeacon with fetch POST fallback +- Retry with backoff and jitter +- Evict oldest when storage cap reached + +--- + +## Server API + +```sudo +import { createLogger } from 'aidd/logger/server.js' + +const { log, attach } = createLogger(options) + +// attach adds logger to response.locals.logger +// per request dispatch is at response.locals.dispatch +``` + +Behavior +- Mirror client API for parity +- `attach({ request, response })` sets `response.locals.logger` + +--- + +## Configuration + +```sudo +createLogger(options) + +options + endpoint // POST target or internal queue + payloadSanitizer // (any) => any + headerSanitizer // (headers) => headers + serializer // (any) => string + batchSizeMin // default 10 + batchSizeMax // default 50 or 64k bytes + flushIntervalMs // background flush tick when online + maxLocalBytes // cap for localStorage pool + consentProvider // () => { analytics: bool } + getIds // () => { sessionId userPseudoId requestId? } + clock // () => Date.now() + level // default info + sampler: observablePipableOperator = takeEvery + events? // per event overrides, see below +``` + +--- + +## Per event configuration + +```sudo +createLoggerMiddleware({ + events: { + [type]: { + shouldLog = true + sampler = takeEvery // rxjs pipeable operator + sanitizer = standardSanitizer // (payload) => payload + serializer = standardSerializer // (payload) => string + level = info + } + } +}) +``` + +Notes +- `sampler` can be any rxjs pipeable operator such as takeEvery sampleTime throttleTime bufferTime +- `sanitizer` runs before serialization +- `serializer` outputs a compact string +- Missing entries use global defaults + +--- + +## Event envelope and log payload + +```sudo +LogPayload + timeStamp // Date.now() at creation + message // string + logLevel // debug info warn error fatal + sanitizer // optional override + serializer // optional override + context // key map of contextual fields + props // additional structured dat + createdAt? // Time of server injestion + +Event + type // string + payload // LogPayload | any +``` + +Client and server enrich with + +```sudo +Enrichment + schemaVersion = 1 + eventId = cuid2() + userPseudoId + requestId + appVersion + route + locale +``` +--- + +## Security and privacy + +``` +Privacy { + collect minimum required + no passwords tokens credit cards or sensitive pii + apply payloadSanitizer and headerSanitizer + check consentProvider before logging any "non-essential" tracking events, which implies we need a way to mark events essential + opt out disables non-essential logging + +Security (Server side) + +POST $eventsRoute[enevtId] + require method is POST + require contentType is application/json + require origin in AllowedOrigins + require referer origin matches origin + parse body + require byteLength <= maxByteLength + for each event in body.events + require eventId + require client timeStamp within skewWindow + require schema validation + idempotent by eventID + enqueue + respond 204 +``` +--- + +## Example usage + +``` +// client +const logger = createLogger({ + endpoint: '/api/events', // versioning is built into the metadata + events: { + pageViewed: { sampler: takeEvery, level: info }, + mouseMoved: { sampler: sampleTime(500), level: info }, + error: { sampler: takeEvery, level: error, sanitizer: scrubError } + } +}); + +const dispatch = useDispatch(); + +dispatch({ + type: 'pageViewed', + payload: { route: '/' +}); + +// server +const serverLogger = createLogger({ + endpoint: 'console', // default + events: { + httpRequest: { sampler: takeEvery, level: info, sanitizer: scrubRequest }, + httpResponse: { sampler: takeEvery, level: info, sanitizer: scrubResponse }, + error: { sampler: takeEvery, level: error, sanitizer: scrubError } + } +}) + +// in route handler +request.locals.dispatch(action creator(action)) +``` + +--- + +## Best practices + +Structured logs: stable JSON keys for aggregation + +Correlation: requestId userPseudoId + + +# Agent instructions -### JSON Schema Examples -- Would you like me to include Ajv schema examples for common event types in the task epic, or should that be part of the implementation documentation? +Before implementing, /review this issue description for adherence to best practices (including repo, logging, gdpr compliance, security) -### Essential vs Non-Essential Events -- Should we add a marker field like `event.metadata.essential: boolean` to distinguish event types, or handle this purely through configuration? +Create a /task epic and save it -### Error Handling -- For serialization/sanitization failures, should we provide a configurable `onError` callback, or always default to console.error? +Make sure there's a questions section in the task epic if you have any questions From 35bf10b2786a21c62a102d50505394806f9a449c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 24 Oct 2025 01:13:44 +0000 Subject: [PATCH 3/3] feat: add SudoLang interfaces section to task epic template MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Update task-creator.mdc to require Interfaces section in epics - Add comprehensive SudoLang interfaces to unified logger epic - Define all types: Event, LogPayload, EnrichedEvent, LoggerOptions, etc. - Align all requirements with defined interfaces - Include framework interfaces (Dispatch, Events$) marked as out-of-scope Interfaces take precedence - requirements verified for consistency. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- ai/rules/task-creator.mdc | 39 ++- tasks/unified-logger-epic.md | 576 ++++++++++++----------------------- 2 files changed, 228 insertions(+), 387 deletions(-) diff --git a/ai/rules/task-creator.mdc b/ai/rules/task-creator.mdc index b45f4a4..746e761 100644 --- a/ai/rules/task-creator.mdc +++ b/ai/rules/task-creator.mdc @@ -77,7 +77,7 @@ epicTemplate() { """ # ${EpicName} Epic - **Status**: 📋 PLANNED + **Status**: 📋 PLANNED **Goal**: ${briefGoal} ## Overview @@ -86,6 +86,34 @@ epicTemplate() { --- + ## Interfaces + + Define key types and interfaces in SudoLang format: + + ```sudo + $InterfaceName { + ...otherInterfaceToMixIn + [$keyName]: $type + } + ``` + + Example: + ```sudo + type ID = string(cuid2) + type Timestamp = number(epoch ms) + + User { + id: ID + name: string + createdAt: Timestamp + meta { + lastSignedIn: Timestamp + } + } + ``` + + --- + ## ${TaskName} ${briefTaskDescription} @@ -104,11 +132,18 @@ epicConstraints { Explain what gaps are being addressed Keep it terse + // Interfaces: + Define all key types and interfaces in SudoLang format + Interfaces take precedence - requirements must align with defined interfaces + Include types for configuration, events, payloads, and return values + Use clear, descriptive names + // Tasks: No task numbering (use task names only) Brief description (1 sentence max) Requirements section with bullet points ONLY using "Given X, should Y" format Include ONLY novel, meaningful, insightful requirements + Requirements must be consistent with defined interfaces NO extra sections, explanations or text } @@ -116,6 +151,8 @@ reviewEpic() { After creating the epic file, verify: 1. Single paragraph overview starting with WHY + 1. Interfaces section includes all key types in SudoLang format + 1. Requirements align with defined interfaces 1. No task numbering 1. All requirements follow "Given X, should Y" format 1. Only novel/insightful requirements remain (eliminate obvious boilerplate) diff --git a/tasks/unified-logger-epic.md b/tasks/unified-logger-epic.md index a0218d7..5af779e 100644 --- a/tasks/unified-logger-epic.md +++ b/tasks/unified-logger-epic.md @@ -9,15 +9,129 @@ Developers need a unified logging solution that works consistently across client --- +## Interfaces + +```sudo +// Base types +type ID = string(cuid2) +type Timestamp = number(epoch ms) +type LogLevel = "debug" | "info" | "warn" | "error" | "fatal" +type Sanitizer = (payload: any) => any +type Serializer = (payload: any) => string +type RxJSOperator = function(Observable) => Observable + +// Framework interfaces (out of scope - mocked for testing) +type Dispatch = (event: Event) => void // synchronous, returns nothing +type Events$ = Observable // RxJS Observable + +// Event structures (Redux-compatible action objects) +Event { + type: string + payload: any // typically LogPayload but can be any +} + +LogPayload { + timestamp: Timestamp // Date.now() at creation + message: string + logLevel: LogLevel + sanitizer?: Sanitizer // optional override + serializer?: Serializer // optional override + context?: Record // contextual fields + props?: Record // additional structured data +} + +EnrichedEvent { + ...Event + schemaVersion: number // = 1 + eventId: ID // cuid2() + userPseudoId: string + requestId?: ID + sessionId?: ID + appVersion?: string + route?: string + locale?: string + createdAt?: Timestamp // server ingestion time +} + +// Configuration +LoggerOptions { + endpoint?: string // POST target (client: '/api/events', server: 'console') + payloadSanitizer?: Sanitizer // (any) => any + headerSanitizer?: (headers: Record) => Record + serializer?: Serializer // (any) => string (default: JSON.stringify) + batchSizeMin?: number // default 10 + batchSizeMax?: number // default 50 events OR 64KB bytes, whichever first + flushIntervalMs?: number // timer for auto-flush when online + maxLocalBytes?: number // cap for localStorage pool + maxByteLength?: number // max request body size (server) + skewWindow?: number // acceptable timestamp skew (default: 24h in ms) + futureSkewWindow?: number // acceptable future timestamp skew (default: 1h5m in ms) + consentProvider?: () => { analytics: boolean } + getIds?: () => { + sessionId?: ID + userPseudoId?: string + requestId?: ID + } + clock?: () => Timestamp // default: Date.now + level?: LogLevel // default: 'info' + sampler?: RxJSOperator // default: takeEvery (pass-through) + events?: Record // per-event overrides +} + +EventConfig { + shouldLog?: boolean // default: true + sampler?: RxJSOperator // rxjs pipeable operator + sanitizer?: Sanitizer // (payload) => payload + serializer?: Serializer // (payload) => string + level?: LogLevel // default: 'info' +} + +// Return types +Logger { + log: (message: string, context?: Record) => void + info: (message: string, context?: Record) => void + warn: (message: string, context?: Record) => void + error: (message: string | Error, context?: Record) => void + debug: (message: string, context?: Record) => void + fatal: (message: string | Error, context?: Record) => void + dispose?: () => void // cleanup subscriptions +} + +ClientLogger { + ...Logger + withLogger: (Component: any) => any // HOC for React components +} + +ServerLogger { + ...Logger + withLogger: ({ request, response }: { + request: any + response: any + }) => Promise<{ request: any, response: any }> +} + +// Storage adapter (for testing) +StorageAdapter { + getItem: (key: string) => Promise + setItem: (key: string, value: string) => Promise + removeItem: (key: string) => Promise + keys: () => Promise +} +``` + +--- + ## Core Logger Infrastructure -Create base logger types, interfaces, and event subscription mechanism. +Create base logger factory, event subscription mechanism, and error handling. **Requirements**: -- Given framework emits events via Observable, should subscribe and process matching events -- Given logger receives event, should apply global and per-event configuration -- Given serialization or sanitization fails, should log error to console and continue processing other events -- Given logger is created with invalid configuration, should throw descriptive error +- Given createLogger is called with LoggerOptions, should return Logger with all methods +- Given framework emits Event via events$ Observable, should subscribe and process matching events +- Given logger receives Event, should apply global LoggerOptions and per-event EventConfig +- Given serializer or sanitizer throws error, should log to console.error and continue processing +- Given logger is created with invalid LoggerOptions, should throw descriptive TypeError +- Given logger.dispose is called, should unsubscribe from events$ and clean up resources --- @@ -26,13 +140,18 @@ Create base logger types, interfaces, and event subscription mechanism. Implement browser-based logger with localStorage buffering and network resilience. **Requirements**: -- Given browser is online, should flush batched events immediately in background -- Given browser is offline, should buffer events to localStorage without data loss -- Given browser reconnects after offline period, should auto-flush pooled buffers -- Given localStorage quota exceeded, should evict oldest events FIFO -- Given navigator.sendBeacon is available, should prefer it over fetch for reliability -- Given transport fails with retryable error, should ignore (client telemetry is fire-and-forget) -- Given event batches reach size or time threshold, should flush automatically +- Given createLogger is called in browser, should return ClientLogger +- Given browser is online and Event received, should batch to localStorage and flush in background +- Given browser is offline and Event received, should append to localStorage without flushing +- Given browser reconnects (online event), should auto-flush all pooled buffers from localStorage +- Given localStorage exceeds maxLocalBytes quota, should evict oldest EnrichedEvent FIFO +- Given navigator.sendBeacon is available and batch ready, should prefer sendBeacon over fetch +- Given navigator.sendBeacon unavailable, should fallback to fetch POST +- Given batch reaches batchSizeMax events OR 64KB bytes, should flush immediately +- Given batch not flushed within flushIntervalMs, should flush on timer +- Given POST to endpoint fails, should log error to console and ignore (fire-and-forget) +- Given consentProvider returns analytics false, should skip logging non-essential events +- Given Event payload, should enrich with schemaVersion, eventId, userPseudoId, sessionId from getIds --- @@ -41,10 +160,13 @@ Implement browser-based logger with localStorage buffering and network resilienc Implement Node.js logger with middleware integration. **Requirements**: -- Given createLogger is called on server, should return logger -- Given withLogger is called with request and response, should set response.locals.logger -- Given server logger receives event after subscribing to `events$` observable, should call logger.log() or logger.error() which should internally dispatch to the transport logger (default: console.log/console.error) -- Given server environment, should support console and custom transport +- Given createLogger is called on server, should return ServerLogger +- Given withLogger called with request and response, should set response.locals.logger to Logger +- Given server logger receives Event via events$, should call appropriate logger method (log/error/etc) +- Given logger method called, should dispatch to transport (default: console.log/console.error) +- Given endpoint is 'console', should log to console with appropriate level +- Given endpoint is custom function, should call function with EnrichedEvent +- Given Event payload, should enrich with schemaVersion, eventId, requestId from response.locals --- @@ -53,11 +175,11 @@ Implement Node.js logger with middleware integration. Implement per-event sampling, sanitization, and serialization overrides. **Requirements**: -- Given event type has custom sampler, should apply RxJS operator to event stream -- Given event type has custom sanitizer, should scrub payload before serialization -- Given event type has custom serializer, should use it instead of global serializer -- Given event type has no custom config, should fall back to global defaults -- Given event configuration includes invalid RxJS operator, should fail fast with clear error +- Given EventConfig has custom sampler RxJSOperator, should pipe events$ through operator +- Given EventConfig has custom sanitizer, should apply before global payloadSanitizer +- Given EventConfig has custom serializer, should use instead of global serializer +- Given Event type not in events map, should use global LoggerOptions defaults +- Given EventConfig includes invalid RxJSOperator, should fail fast with TypeError --- @@ -66,397 +188,79 @@ Implement per-event sampling, sanitization, and serialization overrides. Implement sanitizers, consent checking, and PII scrubbing utilities. **Requirements**: -Server: -- Given payload contains headers, should apply headerSanitizer before logging -- Given payload contains request/response data, should apply payloadSanitizer before logging - -Client: -- Given telemetry consent denied, should skip logging - -Server: -- Given PII detection patterns match, should redact (scrub) before storage or transport -- Given developer wants to use logger, should have clear documentation on PII GDPR-compliant scrubbing +- Given server receives Event with headers, should apply headerSanitizer before logging +- Given server receives Event with request/response data, should apply payloadSanitizer before logging +- Given client consentProvider returns analytics false, should skip all telemetry events +- Given payload matches PII detection patterns, should redact before storage or transport +- Given developer integrates logger, should have documentation on GDPR-compliant PII scrubbing --- ## Client Transport Layer -Implement batching and idempotent delivery. +Implement batching and idempotent delivery to /api/events/[id] endpoint. **Requirements**: -- Given we are online and queued events reaches batchSizeMax events, should flush immediately -- Given we are online and batch hasn't flushed in flushIntervalMs, should flush on timer -- Given POST fails, log error to client console and ignore (fire and forget client) - -Server: -- Given server receives duplicate eventId, should respond 204 without processing -- Given client sends events to /api/events/[id], should validate origin, content-type, and schema -- Given request body exceeds maxByteLength, should reject with 413 -- Given event timestamp is outside skewWindow, should reject with 400 (default skewWindow to 24 hours) -- Given event timestamp is outside the futureSkewWindow, should reject with 400 (default futureSkewWindow: 1h:5m) +- Given client online and queued events reach batchSizeMax, should flush immediately +- Given client online and flushIntervalMs elapsed, should flush on timer +- Given POST to /api/events/[id] fails, should log error to console and ignore (fire-and-forget) +- Given batch to flush, should POST to /api/events/[eventId] with Content-Type application/json +- Given multiple events to flush, should batch into single POST body with events array --- -## Schema Validation +## Server Event Endpoint -Implement JSON schema validation for event types using Ajv. +Implement POST /api/events/[id] handler with validation and idempotency. **Requirements**: -- Given event is posted to server, should validate against registered schema -- Given event fails schema validation, should reject with detailed error -- Given event type has no schema, should apply default event schema -- Given schema uses Ajv validator, should compile schemas once at initialization -- Given developer registers event schema, should validate schema definition itself +- Given request method is not POST, should reject with 405 Method Not Allowed +- Given Content-Type is not application/json, should reject with 415 Unsupported Media Type +- Given request origin not in allowedOrigins, should reject with 403 Forbidden +- Given request referer origin does not match origin header, should reject with 400 Bad Request +- Given request body exceeds maxByteLength, should reject with 413 Payload Too Large +- Given Event timestamp outside skewWindow (past), should reject with 400 Bad Request +- Given Event timestamp outside futureSkewWindow (future), should reject with 400 Bad Request +- Given duplicate eventId received, should respond 204 No Content without processing +- Given Event passes all validations, should enqueue and respond 204 No Content --- -## Testing Tips - -- Spies and stubs: vi.fn and vi.spyOn - Vitest ships tinyspy under the hood. Simple, fast, and no extra deps. -- Module mocking: vi.mock with vi.importActual for partial mocks - Works cleanly with ESM. Avoid require. -- Timers: vi.useFakeTimers and vi.setSystemTime - ---- +## Schema Validation -## Documentation and Examples - -Document integration patterns, PII guidelines, and usage examples. +Implement JSON schema validation for Event types using Ajv. **Requirements**: -- Given developer wants to use the logger, should have clear client and server setup examples -- Given developer needs to scrub PII, should have comprehensive PII detection guidelines -- Given developer configures events, should have RxJS operator examples for common patterns -- Given developer handles retention, should have documentation referencing GDPR requirements -- Given developer needs JSON schemas, should have examples for Redux-style action objects - ---- - -### Framework Interfaces -- Out of scope. - ---- - -Original issue: - -# Feature Request: Unified Event Driven Logger for AIDD (`aidd/logger`) - -## Summary -Create a unified event driven logging and telemetry framework for client and server in AIDD. - -The framework provides dispatch. Its implementation is out-of-scope for this issue. - -On client use `aidd/client/useDispatch`. On server use `response.locals.dispatch`. -The logger subscribes to the framework event stream and applies per event rules for sampling, sanitization, serialization, batching, and transport. - -Client uses localStorage for write through buffering. When online it flushes immediately. When offline it pools buffers and auto flushes on reconnection. +- Given Event posted to server, should validate Event.payload against registered JSON schema +- Given Event fails schema validation, should reject with 400 Bad Request and detailed error +- Given Event type has no registered schema, should validate against default Event schema +- Given schemas defined at initialization, should compile with Ajv once on startup +- Given developer registers EventConfig with schema, should validate schema is valid JSON Schema Draft 7 --- -## Core design - -```sudo -// Provided by the broader aidd framework (implementation is out-of-scope) -events$ // rxjs Observable of all dispatched events -useDispatch() // client hook -response.locals.dispatch // server per request - -// Provided by the logger -createLogger(options) -``` +## Testing Infrastructure -All logging originates from framework `dispatch(event)`. -The logger subscribes to `events$` and routes matching events to storage and transport. +Create test utilities, mocks, and adapters for isolated testing. ---- - -## Client API - -```js -import { createLogger } from 'aidd/logger/client.js'; -import { useDispatch } from 'aidd/client/use-dispatch.js'; - -const { log, withLogger } = createLogger(options); - -// dev code uses the framework dispatch -const dispatch = useDispatch(); - -dispatch(reportPageview({ page: 'home' }); -``` - -Behavior -- Monitor online and offline state -- Write through to localStorage on every log -- If online then flush immediately in background -- If offline then append to pooled buffers and auto flush on reconnection -- Batch events for network efficiency -- Prefer navigator.sendBeacon with fetch POST fallback -- Retry with backoff and jitter -- Evict oldest when storage cap reached - ---- - -## Server API - -```sudo -import { createLogger } from 'aidd/logger/server.js' - -const withLogger = createLogger(options) - -// withLogger attaches `logger` to -response.locals.dispatch -``` - -Behavior -- No client buffering logic on the server side. - ---- - -## Configuration - -```sudo -createLogger(options) - -options - endpoint // POST target or internal queue - payloadSanitizer // (any) => any - headerSanitizer // (headers) => headers - serializer // (any) => string - batchSizeMax = 50 - flushIntervalMs // background flush tick when online - consentProvider // () => { analytics: bool } - getIds // () => { userPseudoId requestId } - level // default info - sampler = takeEveryglobal default sampler # Feature Request: Unified Event Driven Logger for AIDD (`aidd/logger`) - -## Summary -Create a unified event driven logging and telemetry framework for client and server in AIDD. -The logger does not create `dispatch`. The framework provides dispatch. -On client use `aidd/client/useDispatch`. On server use `response.locals.dispatch`. -The logger subscribes to the framework event stream and applies per event rules for sampling, sanitization, serialization, batching, and transport. -Client uses localStorage for write through buffering. When online it flushes immediately. When offline it pools buffers and auto flushes on reconnection. - ---- - -## Core design - -```sudo -// Provided by the framework -events$ // rxjs Observable of all dispatched events -useDispatch() // client hook -response.locals.dispatch // server per request - -// Provided by the logger -createLogger(options) -``` - -All logging originates from framework `dispatch(event)`. -The logger subscribes to `events$` and routes matching events to storage and transport. - ---- - -## Client API - -```sudo -import { createLogger } from 'aidd/logger/client.js' -import { useDispatch } from 'aidd/client/useDispatch.js' - -const { log, withLogger } = createLogger(options) - -// dev code uses the framework dispatch -const dispatch = useDispatch() - -dispatch({ type: 'page_view', payload: { message: 'home', timeStamp: Date.now() } }) -log('client ready') -``` - -Behavior -- Monitor online and offline state -- Write through to localStorage on every log -- If online then flush immediately in background -- If offline then append to pooled buffers and auto flush on reconnection -- Batch events for network efficiency -- Prefer navigator.sendBeacon with fetch POST fallback -- Retry with backoff and jitter -- Evict oldest when storage cap reached - ---- - -## Server API - -```sudo -import { createLogger } from 'aidd/logger/server.js' - -const { log, attach } = createLogger(options) - -// attach adds logger to response.locals.logger -// per request dispatch is at response.locals.dispatch -``` - -Behavior -- Mirror client API for parity -- `attach({ request, response })` sets `response.locals.logger` - ---- - -## Configuration - -```sudo -createLogger(options) - -options - endpoint // POST target or internal queue - payloadSanitizer // (any) => any - headerSanitizer // (headers) => headers - serializer // (any) => string - batchSizeMin // default 10 - batchSizeMax // default 50 or 64k bytes - flushIntervalMs // background flush tick when online - maxLocalBytes // cap for localStorage pool - consentProvider // () => { analytics: bool } - getIds // () => { sessionId userPseudoId requestId? } - clock // () => Date.now() - level // default info - sampler: observablePipableOperator = takeEvery - events? // per event overrides, see below -``` - ---- - -## Per event configuration - -```sudo -createLoggerMiddleware({ - events: { - [type]: { - shouldLog = true - sampler = takeEvery // rxjs pipeable operator - sanitizer = standardSanitizer // (payload) => payload - serializer = standardSerializer // (payload) => string - level = info - } - } -}) -``` - -Notes -- `sampler` can be any rxjs pipeable operator such as takeEvery sampleTime throttleTime bufferTime -- `sanitizer` runs before serialization -- `serializer` outputs a compact string -- Missing entries use global defaults - ---- - -## Event envelope and log payload - -```sudo -LogPayload - timeStamp // Date.now() at creation - message // string - logLevel // debug info warn error fatal - sanitizer // optional override - serializer // optional override - context // key map of contextual fields - props // additional structured dat - createdAt? // Time of server injestion - -Event - type // string - payload // LogPayload | any -``` - -Client and server enrich with - -```sudo -Enrichment - schemaVersion = 1 - eventId = cuid2() - userPseudoId - requestId - appVersion - route - locale -``` ---- - -## Security and privacy - -``` -Privacy { - collect minimum required - no passwords tokens credit cards or sensitive pii - apply payloadSanitizer and headerSanitizer - check consentProvider before logging any "non-essential" tracking events, which implies we need a way to mark events essential - opt out disables non-essential logging - -Security (Server side) - -POST $eventsRoute[enevtId] - require method is POST - require contentType is application/json - require origin in AllowedOrigins - require referer origin matches origin - parse body - require byteLength <= maxByteLength - for each event in body.events - require eventId - require client timeStamp within skewWindow - require schema validation - idempotent by eventID - enqueue - respond 204 -``` ---- - -## Example usage - -``` -// client -const logger = createLogger({ - endpoint: '/api/events', // versioning is built into the metadata - events: { - pageViewed: { sampler: takeEvery, level: info }, - mouseMoved: { sampler: sampleTime(500), level: info }, - error: { sampler: takeEvery, level: error, sanitizer: scrubError } - } -}); - -const dispatch = useDispatch(); - -dispatch({ - type: 'pageViewed', - payload: { route: '/' -}); - -// server -const serverLogger = createLogger({ - endpoint: 'console', // default - events: { - httpRequest: { sampler: takeEvery, level: info, sanitizer: scrubRequest }, - httpResponse: { sampler: takeEvery, level: info, sanitizer: scrubResponse }, - error: { sampler: takeEvery, level: error, sanitizer: scrubError } - } -}) - -// in route handler -request.locals.dispatch(action creator(action)) -``` +**Requirements**: +- Given tests need storage isolation, should provide mock StorageAdapter +- Given tests need dispatch spy, should provide vi.fn mock for Dispatch +- Given tests need events$ stub, should provide Subject from rxjs for controllable event emission +- Given tests need deterministic time, should inject clock function returning fixed Timestamp +- Given tests run in parallel, should not share localStorage state between test suites +- Given logger created in test, should expose dispose method for cleanup --- -## Best practices - -Structured logs: stable JSON keys for aggregation - -Correlation: requestId userPseudoId - - -# Agent instructions - -Before implementing, /review this issue description for adherence to best practices (including repo, logging, gdpr compliance, security) - -Create a /task epic and save it +## Documentation and Examples -Make sure there's a questions section in the task epic if you have any questions +Document integration patterns, PII guidelines, and usage examples. +**Requirements**: +- Given developer integrates logger, should have clear client setup example with createLogger +- Given developer integrates logger, should have clear server middleware example with withLogger +- Given developer needs PII scrubbing, should have comprehensive guidelines with examples +- Given developer configures events, should have RxJS operator examples (takeEvery, sampleTime, throttleTime) +- Given developer handles retention, should have documentation referencing GDPR Article 5 and Article 17 +- Given developer creates Event schemas, should have Ajv JSON Schema examples for Redux action objects