Encounter stats: store map_point_history on Mongo + backfill script#337
Merged
Conversation
… script The /api/runs/encounter-stats endpoint shipped in #336 returned zero rows because the aggregation walks $map_point_history but that field was never stored on the run doc — only denormalized fields (killed_by, deck, relics, card_choices) were. The raw history lives on disk at data/runs/<hash>.json (which the share-run page already reads). Two changes: 1. submit_run now includes map_point_history in the inserted doc. Going forward, every new submission populates the field so the aggregation has data to walk. 2. tools/backfill_run_encounters_mongo.py reads every disk JSON whose corresponding Mongo doc is missing the field and $sets it. Idempotent — safe to re-run, processes ~5K docs/sec. Operator run after deploy: ssh prod 'cd /var/www/spire-codex && docker compose -f docker-compose.prod.yml exec backend python3 -m tools.backfill_run_encounters_mongo' (Beta box runs the same script against the beta compose file.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the empty
/api/runs/encounter-statsresponse that shipped in #336. Submitted runs now carrymap_point_historyon their Mongo doc, and a single-pass backfill populates it for the existing ~9.9K docs by reading their on-disk JSON.Why
The aggregation walks
$map_point_historyto compute per-encounter sample counts, fatal counts, average damage, and average turns. The field was never stored on the run doc; only denormalized fields (killed_by, deck, relics, card_choices) were. Disk JSONs are the source of truth (the share-run page already reads from there).Changes
submit_runincludesmap_point_historyin the inserted doc going forward. All new submissions populate the field automatically.tools/backfill_run_encounters_mongo.pyfinds every Mongo doc missing the field and$sets it from the corresponding disk JSON. Idempotent. Supports--dry-runand--limit Nfor spot-checks.After-merge ops
Same against
docker-compose.beta.ymlfor the beta site.The hourly autodeploy cron will pull the new backend image but won't run the backfill itself. Operator runs the command above once after the first deploy carrying this change.