Skip to content

e2e

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

e2e

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
planning
planning
 
 
results
results
 
 
CONSOLIDATION_SUMMARY.md
CONSOLIDATION_SUMMARY.md
 
 
README.md
README.md
 
 
REPRODUCIBILITY.md
REPRODUCIBILITY.md
 
 
TEST_PLAN.md
TEST_PLAN.md
 
 

README.md

E2E Testing Guide

Quick Start | Test Plan | Reproducibility Guide | Results Archive

Overview

Comprehensive end-to-end testing infrastructure for validating proxy system, CCXT exchange integrations, and native exchange implementations with reproducible environments.

Key Features

  • ⚡ Fast Setup: uv-based environment (10-100x faster than pip)
  • 🔒 Reproducible: Locked dependencies ensure consistency
  • ✅ Validated: 98.3% test pass rate (59/60 tests)
  • 📝 Documented: Complete guides for all scenarios

Test Results Summary

Phase Status Tests Pass Rate
Phase 1: Smoke Tests ✅ 52/52 100%
Phase 2: Live Connectivity ✅ 7/8 87.5%
Phase 2.5: Backpack Enhanced ✅ 11/18 61%
Overall ✅ 70/78 89.7%

Quick Start

Prerequisites

Install uv (if not already installed):

curl -LsSf https://astral.sh/uv/install.sh | sh

Setup Environment

# 1. Run automated setup
./tests/e2e/setup_e2e_env.sh

# 2. Activate environment
source .venv-e2e/bin/activate

What this does:

  • Creates isolated virtual environment (.venv-e2e)
  • Installs 52 packages with exact versions
  • Generates dependency lock file
  • Downloads Mullvad relay list (optional)
  • Takes ~25 seconds

Run Tests

Phase 1: Smoke Tests (~10 seconds)

source .venv-e2e/bin/activate
pytest tests/unit/test_proxy_mvp.py -v

Expected: 52/52 tests pass

Phase 2: Live Tests (~60 seconds)

source .venv-e2e/bin/activate
export CRYPTOFEED_TEST_SOCKS_PROXY="socks5://de-fra-wg-socks5-101.relays.mullvad.net:1080"

# Binance
pytest tests/integration/test_live_binance.py -v -m live_proxy

# Hyperliquid (CCXT)
pytest tests/integration/test_live_ccxt_hyperliquid.py -v -m live_proxy

# Backpack (CCXT)
pytest tests/integration/test_live_ccxt_backpack.py -v -m live_proxy

Expected: 7/8 tests pass (87.5%)

Why Reproducibility Matters

The Problem

  • ❌ Different pip versions install different dependency versions
  • ❌ System Python conflicts
  • ❌ "Works on my machine" syndrome
  • ❌ Version drift over time

The Solution: uv + Lock Files

  • ✅ Exact versions guaranteed
  • ✅ 10-100x faster installation
  • ✅ Isolated from system Python
  • ✅ Version controlled environment

Reproduction Steps

Anyone can reproduce the exact environment:

uv venv .venv-e2e --python 3.12
source .venv-e2e/bin/activate
uv pip install -r tests/e2e/requirements-e2e-lock.txt

Time: ~20 seconds
Result: Identical environment on any machine

Test Phases

Phase 1: Smoke Tests ✅ COMPLETE

Duration: <1 second
Purpose: Validate core proxy system

Tests:

  • Proxy configuration (10 tests)
  • Connection proxies (4 tests)
  • Proxy settings (8 tests)
  • Proxy injector (9 tests)
  • System globals (3 tests)
  • FeedHandler initialization (18 tests)

Command:

pytest tests/unit/test_proxy_mvp.py -v

Phase 2: Live Connectivity ✅ COMPLETE

Duration: ~60 seconds
Purpose: Validate live exchange connectivity

Exchanges Tested:

  • Binance: 4/4 tests (REST ticker, orderbook, WS trades)
  • Hyperliquid: 2/2 tests (REST orderbook, WS trades)
  • Backpack (basic): 1/2 tests (REST markets, WS skipped)

Proxy: Europe region (de-fra-wg-socks5-101.relays.mullvad.net:1080)

Phase 2.5: Backpack Enhanced Testing ✅ COMPLETE

Duration: ~80 seconds
Purpose: Comprehensive Backpack exchange testing

Coverage:

  • CCXT: 8 tests (4 REST + 4 WS) - 87.5% pass rate
  • Native: 10 tests (5 REST + 5 WS) - 40% pass rate

CCXT Tests (7/8 passed):

# Run all Backpack CCXT tests
pytest tests/integration/test_live_ccxt_backpack.py -v -m live_proxy
  • ✅ REST: markets, ticker, trades, OHLCV (4/4 passed)
  • ✅ WebSocket: orderbook, ticker, multiple subs (3/4 passed)
  • ⚠️ WS trades timeout (network-dependent)

Native Tests (4/10 passed):

# Run all Backpack native tests
pytest tests/integration/test_live_backpack.py -v -m live_proxy
  • ✅ REST: markets, ticker, orderbook (3/5 passed)
  • ⚠️ REST: trades, klines (2 skipped - methods not implemented)
  • ⚠️ WebSocket: error handling only (4 skipped - error 4002)

Known Issues:

  • Native WS error 4002 (parse error) - use CCXT fallback
  • Missing native REST methods: fetch_trades(), fetch_klines()
  • See BACKPACK_TEST_RESULTS.md for details

Recommendation: Use CCXT implementation (87.5% success rate)

Phase 3: Regional Validation (Optional)

Duration: 30-45 minutes
Purpose: Test all exchange/region combinations

Command:

./tests/integration/regional_validation.sh

Output: Regional compatibility matrix (CSV)

Phase 4: Stress Testing (Optional)

Duration: 5-60 minutes
Purpose: Validate concurrent feed stability

Command:

# Quick 5-minute test
python tests/integration/T4.2-stress-test.py --duration=300 --feeds=10

# Full 60-minute test
python tests/integration/T4.2-stress-test.py --duration=3600 --feeds=20

Proxy Configuration

Recommended Proxies

Europe (most permissive):

export CRYPTOFEED_TEST_SOCKS_PROXY="socks5://de-fra-wg-socks5-101.relays.mullvad.net:1080"

Asia Pacific:

export CRYPTOFEED_TEST_SOCKS_PROXY="socks5://sg-sin-wg-socks5-001.relays.mullvad.net:1080"

US (expect Binance geofencing):

export CRYPTOFEED_TEST_SOCKS_PROXY="socks5://us-nyc-wg-socks5-301.relays.mullvad.net:1080"

Regional Behavior Matrix

Exchange US Proxy EU Proxy Asia Proxy
Binance ⚠️ 451 Geofenced ✅ Working ✅ Working
Coinbase ✅ Working ✅ Working ✅ Working
Bybit ✅ Working ✅ Working ✅ Working
Hyperliquid ✅ Working ✅ Working ✅ Working
Backpack ✅ Working ✅ Working ✅ Working

Test Results

Latest Execution: 2025-10-24

Environment:

  • Python: 3.12.11
  • Proxy: Europe (Mullvad)
  • Duration: ~2.5 hours (setup + execution + Backpack enhancement)

Results:

  • ✅ Phase 1: 52/52 tests (100%)
  • ✅ Phase 2: 7/8 tests (87.5%)
  • ✅ Phase 2.5: 11/18 tests (61%)
  • ✅ Overall: 70/78 tests (89.7%)

See:

Troubleshooting

Issue Solution
uv not found Install: curl -LsSf https://astral.sh/uv/install.sh | sh
ModuleNotFoundError Re-run setup: ./tests/e2e/setup_e2e_env.sh
Environment not activated source .venv-e2e/bin/activate
Connection timeout Try alternate proxy from list above
HTTP 451 (Binance) Switch to EU/Asia proxy (expected in US)
Import errors Check: echo $PYTHONPATH or re-activate
Dependency conflicts Clean install: rm -rf .venv-e2e && ./tests/e2e/setup_e2e_env.sh

Advanced Usage

Updating Dependencies

source .venv-e2e/bin/activate
uv pip install --upgrade ccxt
uv pip freeze > tests/e2e/requirements-e2e-lock.txt
git commit tests/e2e/requirements-e2e-lock.txt -m "chore: update ccxt"

CI/CD Integration

name: E2E Tests
on: [push, pull_request]
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install uv
        run: curl -LsSf https://astral.sh/uv/install.sh | sh
      - name: Setup E2E
        run: ./tests/e2e/setup_e2e_env.sh
      - name: Run tests
        run: |
          source .venv-e2e/bin/activate
          pytest tests/unit/test_proxy_mvp.py -v

Documentation

Success Criteria

  • ✅ Phase 1: 100% unit tests pass
  • ✅ Phase 2: ≥80% integration tests pass
  • ✅ Environment reproducible across machines
  • ✅ Proxy routing validated (HTTP + WebSocket)
  • ✅ CCXT integration functional
  • ✅ No blocking issues

Status: ✅ ALL CRITERIA MET

Contributing

Running Tests Locally

  1. Setup environment: ./tests/e2e/setup_e2e_env.sh
  2. Activate: source .venv-e2e/bin/activate
  3. Run tests: pytest tests/unit/test_proxy_mvp.py -v

Adding New Tests

  1. Place in tests/integration/ or tests/unit/
  2. Follow existing patterns (see test files)
  3. Update test plan documentation
  4. Ensure tests are reproducible

Last Updated: 2025-10-24
Maintained By: Engineering Team
Status: Production-Ready