Skip to content

Conversation

@dcroote
Copy link
Contributor

@dcroote dcroote commented Dec 2, 2025

e2e tests that succeed locally can intermittently fail in CI. This is happening now in #2033, for example. Often, it is specifically airnode-admin with an error like that shown below. Note this error has actually been a nuisance for some time. This PR aims to reduce the error rate originating from an overwhelmed network of requests (there are tons due to the beforeEach of some tests) by running the airnode-admin e2e tests in two ways:

  • By establishing a new provider for each test rather than one for all tests
  • By using StaticJsonRpcProvider instead of JsonRpcProvider since we don't need to confirm the network in tests; relevant part of the docs:
An ethers Provider will execute frequent getNetwork calls to ensure the network calls and network being communicated with are consistent.

Example CI error:

Run yarn run test:e2e-admin
yarn run v1.22.22
$ cd packages/airnode-admin && yarn run test:e2e
$ jest --selectProjects e2e
Running one project: e2e
PASS e2e test/e2e/sdk.feature.ts (8.888 s)
FAIL e2e test/e2e/cli.feature.ts (144.563 s)
  ● CLI › templates › can create data for inline template

    missing response (requestBody="{\"method\":\"eth_accounts\",\"params\":[],\"id\":247,\"jsonrpc\":\"2.0\"}", requestMethod="POST", serverError={"code":"ECONNRESET"}, url="http://127.0.0.1:8545/", code=SERVER_ERROR, version=web/5.7.1)

      at Logger.Object.<anonymous>.Logger.makeError (../../node_modules/@ethersproject/logger/src.ts/index.ts:269:28)
      at Logger.Object.<anonymous>.Logger.throwError (../../node_modules/@ethersproject/logger/src.ts/index.ts:281:20)
      at ../../node_modules/@ethersproject/web/src.ts/index.ts:292:28
      at step (../../node_modules/@ethersproject/web/lib/index.js:33:23)
      at Object.throw (../../node_modules/@ethersproject/web/lib/index.js:14:53)
      at rejected (../../node_modules/@ethersproject/web/lib/index.js:6:65)

Test Suites: 1 failed, 1 passed, 2 total
Tests:       1 failed, 39 passed, 40 total
Snapshots:   1 passed, 1 total
Time:        144.913 s
Ran all test suites.
error Command failed with exit code 1.

Also, closes #2038 as the docs CI takes a long time to run and will still intermittently fail. It probably needs a general overhaul, including the ignoring of CHANGELOG.md files.

@dcroote dcroote marked this pull request as draft December 2, 2025 06:57
@dcroote dcroote marked this pull request as ready for review December 2, 2025 16:36
@dcroote dcroote requested a review from Siegrift December 2, 2025 16:36
Copy link
Contributor

@Siegrift Siegrift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

@dcroote dcroote merged commit d74857c into master Dec 4, 2025
25 checks passed
@dcroote dcroote deleted the e2e-intermittent-failures branch December 4, 2025 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Don't check all links on every PR

3 participants