Skip to content

examples/us_budgetary_impact.py consumes excessive memory (peak ~7.5 GB RSS / 82 GB VMS) #328

@vahid-ahmadi

Description

@vahid-ahmadi

Reported in the discussion on #264 (JOSS paper PR).

When run on the joss-paper branch (4.2.1), examples/us_budgetary_impact.py exhibits very high memory pressure during the baseline + reform simulations on the enhanced CPS 2024 dataset (2026, ~101k people / 41k households):

  • Vahid: OOM-killed at peak ~30 GB RSS on a Mac (rc=137).
  • Anthony (instrumented run): peak 7.48 GiB RSS, 82.35 GiB VMS; baseline_sim.run() 546.6s wall, reform_sim.run() 596.5s wall; VSCode crashed mid-run from memory pressure.

This makes the canonical US population example impractical for normal local development and for JOSS reviewers attempting to reproduce results.

Suggested next steps:

  • Profile baseline_sim.run() and reform_sim.run() to find the dominant allocations (likely in policyengine-us core, but worth confirming).
  • Determine whether intermediate arrays can be released between runs or whether the two simulations can share underlying data.
  • Document peak memory requirement in the example header until the footprint is reduced.

Related: #264

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions