Skip to content

Latest commit

 

History

History
25 lines (18 loc) · 1.23 KB

File metadata and controls

25 lines (18 loc) · 1.23 KB

Data Ingestions Workspace

A collection of high-performance blockchain data ingestion services (producers) that fetch data from diversas sources and publish it to Kafka using shared Protocol Buffer definitions.

Services

Directory Language Source Target Chain(s)
eth-go-producers Go JSON-RPC Ethereum
evm-producers TypeScript JSON-RPC EVM-Compatible (Base, HyperEVM, etc.)
hyperliquid-producers TypeScript WebSocket Hyperliquid L1
solana-producers TypeScript gRPC (Yellowstone) Solana

Overview

Each service in this workspace is responsible for:

  1. Connecting to a specific blockchain data source (RPC, WebSocket, or gRPC).
  2. Mapping raw blockchain data to shared Protobuf structures.
  3. Publishing the structured data to Kafka topics for downstream consumption.

Shared Dependencies

Producers rely on the following shared packages (located in sibling repositories or managed via workspace links):

  • @ethereum-kafka-indexer/protos: Common Protocol Buffer definitions.
  • @ethereum-kafka-indexer/shared: Shared configuration and utility logic.