Skip to content

Conversation

@Nero0225
Copy link

This PR refactors the validator's reward mechanism to address performance bottlenecks caused by sequential, blocking GitHub API calls. By moving to aiohttp and parallel processing, we significantly reduce the time required to evaluate miners, especially as the subnet scales.

Key Changes

  • Async/Parallel Miner Evaluation:

    • Rewrote the validator's main reward loop in gittensor/validator/evaluation/reward.py to evaluate all miners concurrently using asyncio.gather.
    • Replaced the sequential for loop with parallel task execution, drastically reducing total epoch time.
  • Non-Blocking GitHub API Client:

    • Updated gittensor/utils/github_api_tools.py to use aiohttp instead of requests.
    • Implemented a shared ClientSession approach in score_pull_requests to reuse TCP connections for multiple file checks within a single miner evaluation, reducing handshake overhead.
  • Robust Error Handling & Retry Logic:

    • Added sophisticated retry logic with exponential backoff to get_pull_request_file_changes and get_user_merged_prs_graphql.
    • The client now gracefully handles transient network errors (timeouts, connection resets) and GitHub API secondary rate limits.
  • Testing:

    • Updated tests/utils/test_github_api_tools.py to use unittest.IsolatedAsyncioTestCase for rigorous async testing.
    • Added new tests for session reuse, retry behavior, and error handling.
    • Verified all tests pass in the gittensor-venv environment.

Performance Impact

  • Sequential (Old): $O(N \times M)$ time, where $N$ is miners and $M$ is PRs/files.
  • Parallel (New): Approaches $O(M)$ time (limited by API rate limits and concurrency), as multiple miners are queried simultaneously.

Dependencies

  • Added aiohttp==3.11.11 to requirements.txt.

@LandynDev LandynDev closed this Dec 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants