Skip to content

Stabilize live integration tests#3556

Merged
anth-volk merged 3 commits intomasterfrom
codex/stabilize-live-integration-tests
May 5, 2026
Merged

Stabilize live integration tests#3556
anth-volk merged 3 commits intomasterfrom
codex/stabilize-live-integration-tests

Conversation

@anth-volk
Copy link
Copy Markdown
Collaborator

@anth-volk anth-volk commented May 5, 2026

Fixes #3555

Summary

  • add a per-run suffix to integration_probe_id so workflow reruns do not reuse stale staging cache keys
  • centralize live endpoint polling and retry transient request errors plus HTTP 5xx responses until the integration deadline
  • reduce live calculate smoke payload axis count from 401 to 101
  • keep both national and Utah economy live tests while avoiding exact-value assertions that drift with model/data updates
  • remove the live budget-window in-flight dedupe assertion and rely on the non-live integration test for that behavior

Related Failure

Testing

  • ruff check tests/integration/conftest.py tests/integration/test_live_economy.py tests/integration/test_live_budget_window_cache.py tests/integration/test_budget_window_in_flight_dedupe.py
  • ruff format --check tests/integration/conftest.py tests/integration/test_live_economy.py tests/integration/test_live_budget_window_cache.py tests/integration/test_budget_window_in_flight_dedupe.py
  • .venv/bin/python -m pytest tests/integration/test_budget_window_in_flight_dedupe.py -q
  • .venv/bin/python -m pytest tests/integration/test_live_calculate.py tests/integration/test_live_economy.py tests/integration/test_live_budget_window_cache.py --collect-only -q

@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.94%. Comparing base (0d99785) to head (5f71127).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3556   +/-   ##
=======================================
  Coverage   78.94%   78.94%           
=======================================
  Files          65       65           
  Lines        3762     3762           
  Branches      662      662           
=======================================
  Hits         2970     2970           
  Misses        614      614           
  Partials      178      178           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@anth-volk anth-volk marked this pull request as ready for review May 5, 2026 20:49
@anth-volk anth-volk merged commit a61d113 into master May 5, 2026
7 checks passed
@anth-volk anth-volk deleted the codex/stabilize-live-integration-tests branch May 5, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stabilize live integration tests

1 participant