Conversation
Add workspace Cargo.toml, MIT LICENSE, CONTRIBUTING.md, and .gitignore covering Rust, Go, Node, environment files, and IDE artifacts.
Define soroban_events with all columns, computed topic columns, composite and GIN indexes. Add system_state for cursor tracking, indexed_contracts for the contract registry, and ledger_metadata for gap detection. Migration 0001_init.sql mirrors schema.sql at this stage.
TridentError covers RpcError, ParseError, StorageError, and ConfigError via thiserror. SorobanEvent is the canonical normalised event record shared across the indexer, gRPC server, and any future crates.
Streamer owns the RPC polling loop, cursor management, and retry logic. Parser owns XDR decoding, ScVal normalisation, and type coercion to the canonical SorobanEvent shape. Both modules have detailed comment blocks describing their full responsibilities for the next developer.
Minimal Tonic server entry point listening on GRPC_ADDR (default 0.0.0.0:50051). Proto definitions and service implementations are deferred — the commented layout shows exactly where they slot in once .proto files are defined.
Minimal stdlib HTTP server with graceful shutdown. Comment markers show exactly where the REST router, GraphQL handler, WebSocket handler, and Redis Streams consumer are wired in. Listens on PORT env var, default 3000.
TridentClient accepts apiUrl, apiKey, and network. Three public methods with full type signatures: queryEvents (cursor-paginated), getEventById, and subscribeToContract (returns a Subscription handle). Zod schemas mirror the server-side SorobanEvent shape for runtime validation.
Dev compose runs postgres:15 and redis:7 only, with a postgres health check and schema auto-init. Production compose adds indexer and api services with depends_on health check conditions; all secrets are env var references.
Documents every required environment variable with a one-line explanation of its purpose. Covers DATABASE_URL, REDIS_URL, STELLAR_RPC_URL, NETWORK, POLL_INTERVAL_MS, INDEX_DIAGNOSTIC, LOG_LEVEL, PORT, API_KEY_SALT, and PostgreSQL credentials for the production compose file.
SPECIFICATION.md reserves the location for the full technical spec covering the data model, RPC protocol, XDR decoding, cursor semantics, API contracts, and SDK design decisions. Content will be added before Phase 1 development begins.
Three required jobs on push/PR to dev and main: cargo fmt + clippy -D warnings + cargo test for Rust; go vet + golangci-lint for Go; tsc --noEmit for TypeScript. Rust build cache via Swatinem/rust-cache keeps CI times reasonable. Closes #1
Document the exact commands contributors must run locally before opening a PR — cargo fmt/clippy/test, go vet/golangci-lint, and tsc --noEmit — so they know what the pipeline enforces before they push. Update project status to reflect that scaffolding and CI are now complete.
Full indexer pipeline: - Config::from_env() reads all required environment variables - RpcClient calls getEvents via JSON-RPC 2.0 with cursor pagination - Parser decodes base64 XDR ScVal topics/data, normalises to SorobanEvent; all common ScVal variants handled with JSON-safe i128/u128 overflow to string - db module: insert_event (ON CONFLICT DO NOTHING), get/set_cursor, insert_ledger_metadata via sqlx - redis_stream module: XADD to trident:events stream - Streamer loop: cursor recovery on restart, exponential backoff retry (5 attempts, max 30s), full-page pagination, per-event error isolation so one bad event never stalls the stream - SorobanEvent gains ledger_timestamp field in trident-common Closes #2
proto/trident.proto defines EventsService with three RPCs: ListEvents (cursor-paginated with contract_id/topic/ledger filters), GetEvent (by UUID), and StreamEvents (server-side streaming for real-time delivery). All message types are fully documented. build.rs compiles the proto via tonic-build. EventsServiceImpl in src/services/events.rs is wired into the Tonic server with TODO stubs for each RPC showing exactly what query/stream logic goes in each handler. Closes #3
Rust CI fixes: - stellar-xdr 0.0.x no longer published; update to 26.0.1. Fix ScAddress::Contract pattern (ContractId wrapper, not Hash) and add exhaustive match arms for new MuxedAccount/ClaimableBalance/LiquidityPool variants added in 26.x - stellar-strkey 0.0.8 → 0.0.16: to_string() now returns heapless::String; convert via .as_str().to_owned() at both call sites - sqlx::query! requires DATABASE_URL or a prepared cache at compile time; replace all four usages with sqlx::query().bind() chains which compile without a live database - Streamer: fix identical-branch clippy lint — start_ledger was always None; correct logic now uses Some(1) on first run and cursor-based pagination on all subsequent calls - Wire insert_ledger_metadata into poll_once (called once per page after cursor advance) to resolve dead_code lint - Use in_successful_contract_call in Parser to skip events from failed calls (resolves field dead_code lint and is the correct semantic behaviour) - Use page.latest_ledger in streamer debug log (resolves field dead_code lint) - Add protoc-bin-vendored as a build dependency so protoc is bundled and neither CI nor local dev need a system installation
Depo-dev
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the foundational pull request for Trident. It takes the repository from an empty README to a fully scaffolded, CI-gated monorepo with a working Phase 1 indexer pipeline.
Changes
Monorepo structure
Full directory and workspace layout for all four layers:
crates/(Rust),services/api/(Go),sdk/typescript/,database/,docker/, anddocs/. RootCargo.tomlworkspace, MIT licence,.gitignore, and.env.examplewith documented variables.Database
Production-ready PostgreSQL schema —
soroban_eventswith computedtopic_0/topic_1columns, composite and GIN indexes,system_statefor cursor tracking,indexed_contractsfor the contract registry, andledger_metadatafor gap detection. First migration mirrors the schema.Rust — common crate
TridentError(4 variants viathiserror) and the canonicalSorobanEventstruct shared across all Rust crates.Rust — indexer crate (Phase 1 core)
Full event indexing pipeline:
getEventswith cursor paginationScValtopics and data intoSorobanEvent; all primitive variants handled,Map/Vecrecurse into JSON, addresses encode to strkey; events from failed contract calls are filtered outsqlxinserts intosoroban_events,ledger_metadata, andsystem_statecursor managementXADDtotrident:eventsstream for real-time WebSocket deliveryRust — gRPC API crate
proto/trident.protodefinesEventsServicewithListEvents(cursor-paginated, filterable by contract/topic/ledger),GetEvent(by UUID), andStreamEvents(server-side streaming).build.rscompiles protos viatonic-buildwith a vendoredprotoc— no system install required.EventsServiceImplis wired into the Tonic server with clear stubs for each handler.Go — front office API
Minimal stdlib HTTP server with graceful shutdown. Comment markers show exactly where the REST router, GraphQL handler, WebSocket handler, and Redis consumer wire in. Listens on
PORT(default3000).TypeScript SDK
TridentClientwith full type signatures forqueryEvents(cursor-paginated),getEventById, andsubscribeToContract(returns aSubscriptionhandle). Zod schemas mirrorSorobanEventfor runtime validation.Docker
Dev compose runs
postgres:15andredis:7only with health checks and schema auto-init. Production compose addsindexerandapiservices withdepends_onhealth conditions and env-var-only secrets.CI
Three required jobs on every push and PR to
dev/main:cargo fmt --check,cargo clippy -- -D warnings,cargo testgo vet,golangci-lintnpm ci,tsc --noEmitCloses
Closes #1, #2, #3