Address #15; Lay Validation Foundation; Light Cleanup Work

[olivia-banks]

Address #15 and lay foundation for validation; light cleanup work. Running list of changes:

• Move code into the structure recommended by various pieces of tooling (e.g., ./app.py → ./src/epicc/).
• Move file access to using importlib.resources for increased flexibility under differing deployment conditions (necessary for when we package with micropip for WASM/stlite).
• Fixing typing errors, make sure we cleanly lint under standard type checking.
• Implement format read/write (FRW) interface for Define YAML schema and write a validation tool validate_yaml #15 for dealing with arbitrary opaque dictionaries in various formats (YAML/XLSX) while preserving non-semantic (trivia) elements.
• Create a standard validation layer for unstructured data through Pydantic (for both config and model parameters).
• Wire FRW interface into the configuration layer (remove one-off logic).
• Wire FRW interface into the model parameter layer (remove one-off logic).
• Allow generation of "template" files for each Pydantic data model (BaseModel).
• Write unit tests for the FRW interface with pytest.
• Rename from epicc (Epi Cost Calculator) to the new project name, once decided upon.

We're introducing some dependencies here, but they are already baked into pyodide, which is what stlite uses under the hood, so there's no extra overhead!

Discussion for the FRW may be found here.

[olivia-banks]

There's still some more cleanup I have to do on the YAML side.

[Copilot]

Pull request overview

This PR restructures the Streamlit app into a src/epicc/ package layout and introduces a format read/write + validation foundation (Pydantic-backed) for loading parameters and model/config YAML/XLSX in a uniform way, with accompanying unit tests.

Changes:

• Migrates the app from top-level app.py + utils/ + models/ into src/epicc/ (new __main__.py, config/, formats/, model/, models/, utils/).
• Adds a new FRW (format read/write) layer for YAML/XLSX plus Pydantic validation and template generation.
• Adds pytest-based tests for the formats and model loader, and updates dependencies (PyYAML → ruamel-yaml, adds Pydantic).

Reviewed changes

Copilot reviewed 35 out of 43 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
uv.lock	Updates locked dependencies (adds Pydantic + ruamel-yaml; removes PyYAML).
pyproject.toml	Updates runtime deps (adds pydantic[email], ruamel-yaml; removes PyYAML).
.gitignore	Adds ignores for common dev/build artifacts.
app.py	Removes old top-level Streamlit entrypoint.
config/app.yaml	Removes old app config file (migrated).
config/global_defaults.yaml	Removes old global defaults file (migrated).
config/paths.yaml	Removes old model path config (no longer used).
models/tb_isolation.py	Removes old dynamic python-file model module.
models/measles_outbreak.py	Removes old dynamic python-file model module.
utils/model_loader.py	Removes old model discovery/dynamic import loader.
utils/parameter_loader.py	Removes old YAML/XLSX parameter loader utilities.
utils/parameter_ui.py	Removes old Streamlit parameter UI renderer.
utils/section_renderer.py	Removes old section renderer.
src/epicc/main.py	New Streamlit app entrypoint using the new package structure.
src/epicc/web/sidebar.css	Adds CSS for sidebar styling.
src/epicc/config/init.py	Adds config loader (resource-based) and exports CONFIG.
src/epicc/config/schema.py	Adds Pydantic config schema models.
src/epicc/config/default.yaml	Adds default app config YAML.
src/epicc/formats/init.py	Adds format registry, validation wrapper, and read_from_format.
src/epicc/formats/base.py	Introduces the BaseFormat FRW interface.
src/epicc/formats/yaml.py	Adds YAML FRW backend (ruamel-yaml).
src/epicc/formats/xlsx.py	Adds XLSX FRW backend (openpyxl).
src/epicc/formats/template.py	Adds template generation from Pydantic models.
src/epicc/model/init.py	Adds model loader for schema-based YAML models.
src/epicc/model/base.py	Adds BaseSimulationModel ABC for Python-based models.
src/epicc/model/schema.py	Adds Pydantic schema for YAML-defined models.
src/epicc/models/tb_isolation.py	Ports TB Isolation model to BaseSimulationModel.
src/epicc/models/tb_isolation.yaml	Adds default parameters for TB Isolation model.
src/epicc/models/measles_outbreak.py	Ports Measles Outbreak model to BaseSimulationModel.
src/epicc/models/measles_outbreak.yaml	Adds default parameters for Measles Outbreak model.
src/epicc/utils/model_loader.py	Defines built-in model registry (returns model instances).
src/epicc/utils/parameter_loader.py	Adds typed parameter loading + flattening via FRW.
src/epicc/utils/parameter_ui.py	Adds updated Streamlit parameter UI rendering helpers.
src/epicc/utils/section_renderer.py	Adds updated section rendering utilities.
src/epicc/utils/excel_model_runner.py	Refactors existing Excel runner (style cleanup; reuses shared flatten_dict).
tests/epicc/test_formats.py	Adds tests for format selection + read_from_format.
tests/epicc/test_formats_yaml.py	Adds YAMLFormat tests (read/write, error cases).
tests/epicc/test_formats_xlsx.py	Adds XLSXFormat tests (read/write, error cases).
tests/epicc/test_formats_template.py	Adds template generation integration tests (YAML + XLSX).
tests/epicc/test_model_loader.py	Adds tests for schema-based model loader.
sample/TB Isolation.xlsx	Adds sample Excel file for TB Isolation.
sample/Measles Outbreak.xlsx	Adds sample Excel file for Measles Outbreak.
sample/~$TB Isolation.xlsx	Adds an Excel temp/lock file (artifact).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[olivia-banks]

I'm sure #21 will be ready before this one, but I want to merge this one after so that I can resolve conflicts instead of the other way around since that seems like the least complicated path.

[gvegayon]

I'm sure #21 will be ready before this one, but I want to merge this one after so that I can resolve conflicts instead of the other way around since that seems like the least complicated path.

So, merge 21 and then this?

[olivia-banks]

We're moving over from one bundled HTML to a bunch of hashed fragments so that we can fetch in parallel and load quite a bit faster (we need to download megabytes of data, so I think this is worth it). Bundle structure is now like this:

dist/stlite-config.json
dist/files/app.3a27ef3bf08d272bc561acbdada7e1e6dbbcb6ecdccae111849ed83c3818be87.py
dist/files/src/epicc/formats/template.f8f5402a7b225d02918b496dc9e99da8c09362b8b042b50bc871c5b254605a40.py
dist/files/src/epicc/formats/xlsx.c5190d7ff97e3af541dd80c8302e593bbbbfcc6384c53833fbedc3a8390aa7e4.py
dist/files/src/epicc/formats/__init__.f323c290fb4f6057b22c029468b5376dc67072ad4b055e008ef65d3ee4dec6f6.py
...
dist/files/src/epicc/model/__init__.803fe5bec9f1676cd225fdac305a1272747078678fb7c15511f2a7c4ad0c62a2.py
dist/files/src/epicc/model/base.17109af77291668bfb329481e686b12886af46374b5db074c14a92c6b3cd1638.py
dist/files/src/epicc/model/schema.168aace2a4e179636fdf24d9c5fc36c74ec6c89fb1ae197af6297f9a4c2d8ecc.py
dist/files/src/epicc/model/schema.ce66d74e55a043a094e7880e1f16a33d8a9095302fed17b1df5476f6987fc19e.py
dist/files/src/epicc/__main__.e1dd12b31c82fb7b582e6568d577a5cfd25bed568f34fbce3153bd027e68add2.py

This is for all resource files and whatnot. The mapping is in stlite-config.json, and is handled by Stlite natively (this has no overhead on standard Streamlit either).

---

Previously, we were also duplicating the needed packages between the build script and pyproject.toml, which we now derive dynamically:

{
    "entrypoint": "app.py",
    "requirements": [
        "streamlit", "pandas", "numpy", "openpyxl", "pydantic", "ruamel-yaml"
    ],
    "files": { ... }
}

---

This should be ready for another review, then merging into edward_branch, then into main.

[gvegayon]

@EddW1219, you really wanted to do a forced-push?

[olivia-banks]

I just started resolving, I'm not sure why there are vastly more conflicts this time around. Maybe because of the force-push? Regardless, looking at the patches of the commits, it looks like I should just be able to merge in the changes piecewise.

[olivia-banks]

@gvegayon Resolved, moved over the requisite changes and did a -s ours. Can I merge this into edward_branch, and then into main?