Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
94 lines (67 loc) · 3.18 KB

ARCHITECTURE.md

File metadata and controls

94 lines (67 loc) · 3.18 KB

SagaIndexer Architecture

This document details the architectural choices, security model, and data flow of the SagaIndexer system.

1. System Overview

The system follows a microservices topology where security and performance are first-class citizens. All internal communication is strictly protected by mutual TLS (mTLS).

graph TD
    User((User/Admin)) -->|HTTPS/JSON| Gateway[API Gateway]
    Gateway -->|gRPC + mTLS| Auth[Auth Service]
    Gateway -->|gRPC + mTLS| Tracker[Tracker Service]
    Gateway -->|gRPC + mTLS| Analytics[Analytics Service]
    
    subgraph "Security Layer (Zero Trust)"
        Auth
        Tracker
        Analytics
    end
    
    Tracker -->|RPC| EthNode((Ethereum Node))
    Tracker -->|Local| SQLite[(SQLite: Events)]
    Analytics -->|Cache| Redis[(Redis: Metrics)]
    Auth -->|Persistence| Postgres[(Postgres: Users)]
    
    Saga[Saga Orchestrator] -.->|Coordinates| Tracker
    Saga -.->|Coordinates| Analytics
Loading

2. Distributed Transactions: The Saga Pattern

To ensure eventual consistency across multiple microservices without the overhead of heavy distributed transactions (like 2PC), the system implements the Saga Pattern.

sequenceDiagram
    participant T as Tracker
    participant S as Saga Orchestrator
    participant A as Analytics
    
    Note over T,A: Indexing Flow (Success)
    T->>S: New Block Detected
    S->>T: Index Events (Local)
    T-->>S: OK (Events Saved)
    S->>A: Update Metrics (TWAP/Volume)
    A-->>S: OK (Cache Updated)
    
    Note over T,A: Failure Scenario (Compensation)
    A--xS: Calculation Error
    S->>T: Rollback / Mark Block as Invalid
    T-->>S: Compensation Completed
Loading

3. Mutual TLS (mTLS) Security Model

The system operates on a Zero-Trust Network. Unlike traditional TLS, mTLS requires both the client and the server to present valid digital certificates signed by the internal Certificate Authority (CA).

sequenceDiagram
    participant A as Service A (Client)
    participant B as Service B (Server)
    
    Note over A,B: Mutual Handshake
    A->>B: 1. Client Hello + Client Certificate
    B->>A: 2. Server Hello + Server Certificate
    
    Note over A,B: Both validate identity via Internal CA
    
    A->>B: 3. Encrypted gRPC Channel Established
Loading

4. Performance: gRPC vs. JSON

The choice of gRPC and Protocol Buffers over REST/JSON was driven by empirical performance requirements. In the SagaIndexer environment, Protobuf is 53.2% more bandwidth-efficient and up to 11x faster in deserialization.

Why gRPC?

  • Binary Serialization: Reduces payload size dramatically.
  • Multiplexing: HTTP/2 allows multiple concurrent requests over a single connection.
  • Code Generation: Zero boilerplate for client-server integration.

5. Persistence Strategy

The system uses a hybrid persistence model to optimize the CAP Theorem tradeoffs:

  1. PostgreSQL (Auth): Focuses on Consistency and relational data for identity management.
  2. SQLite (Tracker): Focuses on Locality and high-speed indexing for blockchain events.
  3. Redis (Analytics): Focuses on Performance and low-latency access to calculated DeFi metrics.