diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..44ca33c46 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,151 @@ +# Flow360 Python Client — Repository Guidelines + +These guidelines reflect the conventions in the Flow360 Python client. Follow existing patterns in nearby code and these rules when contributing. + +## Project Structure + +| Directory | Contents | +|---|---| +| `flow360/component/simulation/` | V2 simulation framework (active development) | +| `flow360/component/v1/` | Legacy V1 API (maintenance only) | +| `flow360/cli/` | Click-based CLI (`flow360 configure`, etc.) | +| `flow360/plugins/` | Plugin system (e.g., `report`) | +| `flow360/examples/` | Internal example scripts | +| `examples/` | User-facing example scripts | +| `tests/simulation/` | V2 simulation tests | +| `tests/v1/` | V1 legacy tests | +| `docs/` | Sphinx documentation | + +New features go in `flow360/component/simulation/`. Do not add to `flow360/component/v1/` unless fixing a bug. + +## Workflow & Tooling + +- **Package manager:** Poetry. Prefix every command with `poetry run` to match CI. +- **Setup:** `pip install poetry && poetry install` +- **Pre-commit hooks:** autohooks (configured in `pyproject.toml`). Runs black, isort, and pylint automatically. Install with `autohooks activate`. + +### Check-in Procedure + +Run these before opening a PR: + +```sh +poetry run black . # auto-format +poetry run isort . # sort imports +poetry run pylint $(git ls-files "flow360/*.py") --rcfile .pylintrc # lint +poetry run pytest -rA tests/simulation -vv # V2 tests +poetry run pytest -rA --ignore tests/simulation -vv # V1 tests +``` + +V1 and V2 tests must be run separately (not together). + +## Coding Style + +### Formatting + +- **Black** with line-length 100, target Python 3.10 (`pyproject.toml [tool.black]`). +- **isort** with `profile = "black"` (`pyproject.toml [tool.isort]`). +- **pylint** with `.pylintrc` (max line-length 120, but black enforces 100). +- Do not reformat or re-indent lines you did not modify. + +### Naming + +| Element | Convention | Example | +|---|---|---| +| Functions/methods | `snake_case` | `compute_residual`, `get_solver_json` | +| Classes | `PascalCase` | `SimulationParams`, `Flow360BaseModel` | +| Variables/params | `snake_case` | `moment_center`, `mesh_unit` | +| Constants | `UPPER_CASE` | `CASE`, `SURFACE_MESH` | +| Private | leading underscore | `_preprocess`, `_update_param_dict` | +| Modules | `snake_case` | `simulation_params.py`, `surface_models.py` | + +### Imports + +Standard ordering enforced by isort: + +1. Standard library +2. Third-party (pydantic, numpy, etc.) +3. Local/package imports (absolute paths) + +Convention aliases: +- `import pydantic as pd` +- `import numpy as np` +- `import flow360.component.simulation.units as u` +- `from flow360.log import log` + +Prefer top-level imports. Use lazy imports only to break circular dependencies, and scope them as narrowly as possible. + +### Type Annotations + +- Use modern typing from Python 3.10+: `Optional[X]`, `Union[X, Y]`, `Literal["value"]`. +- Use `Annotated[Type, pd.Field(...)]` for discriminated unions. +- Use `typing.final` decorator on concrete entity classes. +- Use `typing_extensions.Self` for return types of model validators. + +### Docstrings + +Numpy-style docstrings with Sphinx `:class:` cross-references. User-facing classes include an `Example` section with `>>>` code and a trailing `====` marker for doc rendering. + +```python +class ReferenceGeometry(Flow360BaseModel): + """ + :class:`ReferenceGeometry` class contains all geometrical related reference values. + + Example + ------- + >>> ReferenceGeometry( + ... moment_center=(1, 2, 1) * u.m, + ... moment_length=(1, 1, 1) * u.m, + ... area=1.5 * u.m**2 + ... ) + + ==== + """ +``` + +Method docstrings use `Parameters`, `Returns`, `Raises`, and `Examples` sections. + +## Pydantic Model Conventions + +- **Pydantic V2** (`>= 2.8`). Always import as `import pydantic as pd`. +- All models inherit from `Flow360BaseModel` (`flow360.component.simulation.framework.base_model`). +- Model config: `extra="forbid"`, `validate_assignment=True`, camelCase serialization aliases via `alias_generator`. +- Private-by-convention attributes use `private_attribute_` prefix (these are regular pydantic fields, not `pd.PrivateAttr`). +- Discriminated unions use `pd.Field(discriminator="type")` or `pd.Field(discriminator="private_attribute_entity_type_name")`. +- Use `pd.Field()` with `frozen=True` for immutable fields, `description` for documentation, and `alias` for serialization overrides. +- Validator patterns: + - `@pd.field_validator("field", mode="after")` with `@classmethod` + - `@pd.model_validator(mode="after")` + - `@contextual_field_validator(...)` / `@contextual_model_validator(...)` for pipeline-stage-aware validation + +## Error Handling & Logging + +- Use the custom exception hierarchy rooted at `Flow360Error` (`flow360/exceptions.py`). Every exception auto-logs on `__init__`. +- Common exceptions: `Flow360ValueError`, `Flow360TypeError`, `Flow360RuntimeError`, `Flow360ConfigurationError`, `Flow360ValidationError`. +- Use `Flow360DeprecationError` for deprecated features. +- **Logging:** Use `from flow360.log import log` — a custom `Logger` class backed by `rich.Console`. Do not use Python's stdlib `logging` module. + +## Testing + +See `tests/AGENTS.md` for detailed testing conventions. + +Quick reference: +- **Framework:** pytest (no `unittest.TestCase`) +- **V2 tests:** `poetry run pytest -rA tests/simulation -vv` +- **V1 tests:** `poetry run pytest -rA --ignore tests/simulation -vv` +- **Coverage:** `pytest -rA tests/simulation --cov-report=html --cov=flow360/component/simulation` +- Warnings are treated as errors via `tests/pytest.ini`. + +## CI/CD + +- **Code style** (`codestyle.yml`): black → isort → pylint (Python 3.10, ubuntu) +- **Tests** (`test.yml`): runs after code style passes; matrix of Python 3.10–3.13 × ubuntu/macOS/Windows; V2 and V1 tests run separately +- **Docs** (`deploy-doc.yml`): Sphinx documentation build and deployment +- **Publishing** (`pypi-publish.yml`): PyPI release workflow + +## Documentation + +- Sphinx-based docs in `docs/`. Build with `poetry install -E docs` then `make html` in `docs/`. +- Update existing docs before creating new ones. +- Example scripts live in `examples/` (user-facing) and `flow360/examples/` (internal). + +_Update this AGENTS.md whenever workflow, tooling, or conventions change._ diff --git a/flow360/component/simulation/AGENTS.md b/flow360/component/simulation/AGENTS.md new file mode 100644 index 000000000..ca9d301e1 --- /dev/null +++ b/flow360/component/simulation/AGENTS.md @@ -0,0 +1,171 @@ +# Simulation Module Guidelines + +This module (`flow360/component/simulation/`) is the V2 simulation framework. It defines the user-facing configuration API, validation pipeline, and translation to solver-native JSON. + +## Architecture + +| Directory | Purpose | +|---|---| +| `framework/` | Base classes, config, entity system, updater | +| `models/` | Physical models (materials, surface/volume models, solver numerics) | +| `meshing_param/` | Meshing parameter definitions (edge, face, volume, snappy) | +| `operating_condition/` | Operating condition definitions | +| `outputs/` | Output/monitor definitions | +| `time_stepping/` | Time stepping configuration | +| `run_control/` | Run control settings | +| `translator/` | SimulationParams → solver JSON conversion | +| `validation/` | Context-aware validation pipeline | +| `blueprint/` | Safe function/expression serialization and dependency resolution | +| `user_code/` | User-defined expressions, variables, and code | +| `user_defined_dynamics/` | User-defined dynamics definitions | +| `migration/` | V1 → V2 migration utilities | +| `web/` | Web/cloud integration utilities | +| `services.py` | Service facade (validate, translate, convert) | +| `simulation_params.py` | Top-level `SimulationParams` model | +| `primitives.py` | Entity types (Surface, Volume, Edge, Box, Cylinder, etc.) | +| `units.py` | Unit system definitions | + +## Base Class Hierarchy + +``` +pd.BaseModel + └── Flow360BaseModel # framework/base_model.py + ├── EntityBase (ABCMeta) # framework/entity_base.py + │ ├── _VolumeEntityBase + │ │ ├── GenericVolume, Box, Cylinder, Sphere, ... + │ └── _SurfaceEntityBase + │ ├── Surface, GhostSurface, MirroredSurface, ... + ├── _ParamModelBase + │ └── SimulationParams + ├── BoundaryBase (ABCMeta) # models/surface_models.py + │ └── Wall, Freestream, Inflow, Outflow, ... + └── SingleAttributeModel # framework/base_model.py +``` + +### Flow360BaseModel + +All models inherit from `Flow360BaseModel`. It provides: +- JSON/YAML file I/O (`from_file`, `to_file`) +- SHA-256 integrity hashing +- Recursive `preprocess()` for non-dimensionalization +- `require_one_of` / `conflicting_fields` mutual exclusion constraints +- `validate_conditionally_required_field` for pipeline-stage-aware required fields + +### EntityBase + +Abstract base for all simulation entities. Key attributes: +- `name` (frozen, immutable identifier) +- `private_attribute_entity_type_name` (discriminator for serialization) +- `private_attribute_id` (UUID for tracking) + +Concrete entities are decorated with `@final` to prevent subclassing. + +### EntityList and EntitySelector + +- `EntityList[Surface, GhostSurface]` — generic container supporting direct entities and rule-based selectors +- `EntitySelector` — rule-based entity selection with glob/regex predicates +- Selectors are lazily expanded via `ParamsValidationInfo.expand_entity_list()` + +### EntityRegistry + +Central registry holding all entity instances. Populated from `EntityInfo` metadata. Supports type-filtered views and glob pattern access. + +## Key Patterns + +### Unit System Context + +All dimensioned construction must happen inside a `UnitSystem` context manager: + +```python +with SI_unit_system: + params = SimulationParams( + reference_geometry=ReferenceGeometry( + moment_center=(1, 2, 1) * u.m, + area=1.5 * u.m**2, + ), + ... + ) +``` + +When loading from file/dict, the unit system is auto-detected from the serialized data. + +### Private Attribute Convention + +Attributes prefixed with `private_attribute_` are pydantic fields (not `pd.PrivateAttr`). They participate in serialization but are considered internal to the framework. Common examples: +- `private_attribute_entity_type_name` — type discriminator +- `private_attribute_id` — generated UUID +- `private_attribute_zone_boundary_names` — mesh zone names + +### Discriminated Unions + +Polymorphic types use Pydantic discriminated unions: + +```python +ModelTypes = Annotated[ + Union[VolumeModelTypes, SurfaceModelTypes], + pd.Field(discriminator="type"), +] +``` + +Each variant has a `type` literal field (e.g., `type: Literal["Wall"] = pd.Field("Wall", frozen=True)`). + +### Field Constructors + +`SimulationParams` uses specialized field constructors from `validation_context.py`: +- `pd.Field()` — always present +- `CaseField()` — relevant only for solver case configuration +- `ConditionalField(context=[SURFACE_MESH, VOLUME_MESH])` — required only during specific pipeline stages + +### Validators + +Two categories of validators: + +1. **Standard Pydantic** — always run: + - `@pd.field_validator("field", mode="after")` with `@classmethod` + - `@pd.model_validator(mode="after")` + +2. **Contextual** — run only when a `ValidationContext` is active: + - `@contextual_field_validator(...)` — wraps field validators, auto-skips outside context + - `@contextual_model_validator(...)` — wraps model validators, can inject `param_info` + +Extract complex validation logic into standalone functions in `validation/validation_simulation_params.py` or `validation/validation_output.py`. Keep the model class as a thin declarative shell. + +## Validation Pipeline + +Validation is context-aware, using `contextvars` to track the pipeline stage: + +| Level | Constant | When | +|---|---|---| +| Surface meshing | `SURFACE_MESH` | Generating surface mesh params | +| Volume meshing | `VOLUME_MESH` | Generating volume mesh params | +| Case solve | `CASE` | Generating solver params | +| All stages | `ALL` | Full validation | + +The `services.validate_model()` function orchestrates: updater → sanitize → materialize entities → initialize variables → contextual validation. + +## Translation + +Translators in `translator/` convert `SimulationParams` to solver-native JSON. The pipeline: + +1. `SimulationParams._preprocess()` — non-dimensionalize physical units +2. Translator function (`get_solver_json()`, `get_surface_meshing_json()`, etc.) — map to flat JSON +3. Output JSON dict suitable for solver consumption + +Translators are pure functions. Do not add `to_solver()` methods on model classes. + +## Version Migration + +The `framework/updater.py` system applies incremental dict transforms to migrate older `simulation.json` files to the current schema. Per-version functions follow the naming pattern `_to_()` (e.g., `_to_25_2_0()`). The `_ParamModelBase._update_param_dict()` method orchestrates migration automatically on load. + +When making breaking schema changes, add a new migration function to the updater. + +## Adding New Features + +1. **New model field:** Add to the appropriate model class with `pd.Field()`. Add validators if needed. +2. **New entity type:** Inherit from `_VolumeEntityBase` or `_SurfaceEntityBase`. Decorate with `@final`. Add `private_attribute_entity_type_name` literal discriminator. +3. **New boundary condition:** Inherit from `BoundaryBase`. Add to the `SurfaceModelTypes` union. +4. **New validator:** Prefer `@contextual_field_validator` if it depends on pipeline stage. Extract logic to `validation/` files. +5. **Schema migration:** Add a versioned migration function to `framework/updater.py`. +6. **Translator update:** Add translation logic to the appropriate `translator/*.py` file. + +_Update this AGENTS.md when architectural patterns or conventions change._ diff --git a/tests/AGENTS.md b/tests/AGENTS.md new file mode 100644 index 000000000..7ae738f15 --- /dev/null +++ b/tests/AGENTS.md @@ -0,0 +1,151 @@ +# Testing Guidelines + +## Framework & Configuration + +- **Framework:** pytest. No `unittest.TestCase`. +- **Config:** `tests/pytest.ini` — all warnings treated as errors except specific known deprecations. +- **Fixtures:** root `conftest.py` registers plugins via `pytest_plugins = ["tests.utils", "tests.mock_server"]`. +- V2 and V1 tests **must be run separately** (different conftest setups conflict): + - V2: `poetry run pytest -rA tests/simulation -vv` + - V1: `poetry run pytest -rA --ignore tests/simulation -vv` + +## File & Function Naming + +- Test files: `test_.py` +- Test functions: `test_` in `snake_case` +- Test classes (rare, for logical grouping only): `TestDescriptiveName` +- Updater tests use version-stamped names: `test_updater_to__` + +## Fixtures + +### Autouse Patterns + +Most test files define a `change_test_dir` fixture to enable relative paths to `data/` folders: + +```python +@pytest.fixture(autouse=True) +def change_test_dir(request, monkeypatch): + monkeypatch.chdir(request.fspath.dirname) +``` + +### Shared Fixtures + +Root `conftest.py` provides: +- `mock_validation_context` / `mock_case_validation_context` — validation context stubs +- `mock_geometry` / `mock_surface_mesh` / `mock_volume_mesh` — local-storage-backed resource mocks + +`tests/simulation/conftest.py` provides: +- `array_equality_override` — patches unyt array equality for test comparisons + +### Inline Fixtures + +Define test-specific fixtures inline in the test file. Keep them near the tests that use them: + +```python +@pytest.fixture() +def constant_variable(): + return UserVariable(name="constant_variable", value=10) +``` + +## Mocking + +### Primary: monkeypatch + +Use pytest `monkeypatch` for most mocking needs: + +```python +monkeypatch.setattr(http_util, "api_key_auth", lambda: {...}) +monkeypatch.setattr(http, "session", MockRequests()) +``` + +### API Mocking: MockResponse + mock_server + +`tests/mock_server.py` provides the `mock_response` fixture: +- `MockResponse` classes load JSON from `tests/data/mock_webapi/` +- URL-to-response routing via `GET_RESPONSE_MAP` / `POST_RESPONSE_MAP` +- The `mock_response` fixture patches `http.session` and `http_util.api_key_auth` + +### S3 Mocking + +`tests/utils.py` provides `s3_download_override` fixture that redirects S3 downloads to local test data. + +### unittest.mock.patch + +Use sparingly, only when `monkeypatch` cannot handle the scenario (e.g., capturing call arguments with `patch`): + +```python +from unittest.mock import patch + +with patch("flow360.component.project.set_up_params_for_uploading", mock_fn): + project.run_case(...) +``` + +## Assertions + +- Plain `assert` statements (no `self.assertEqual`) +- `pytest.raises(ExceptionType, match=...)` for exception testing +- `capsys.readouterr()` for stdout/stderr capture +- `pytest.mark.parametrize` for data-driven test variants +- `pytest.mark.usefixtures("fixture_name")` to apply fixtures without injection + +```python +with pytest.raises(Flow360ValueError, match=error_msg): + project.get_case(asset_id=query_id) +``` + +## Test Utilities + +`tests/utils.py` provides reusable helpers: + +| Function | Purpose | +|---|---| +| `to_file_from_file_test(obj)` | Serialize → deserialize → assert equality | +| `compare_to_ref(obj, ref_path)` | Serialize and compare against reference file | +| `compare_dict_to_ref(data, ref_path)` | Compare dict against JSON reference | +| `file_compare(file1, file2)` | Unified diff comparison | + +## Test Data Organization + +``` +tests/data/ +├── mock_webapi/ # JSON responses for API mocking +├── / # Per-resource local storage data +└── simulation/ # Simulation JSON fixtures for updater tests + +tests/simulation/data/ # Simulation-test-specific input data +tests/simulation/ref/ # Reference output files for comparison +tests/simulation/service/data/ # Service-layer test data +``` + +Reference files in `ref/` directories are used with `compare_to_ref()` for regression testing. When adding new test cases, create corresponding reference files. + +## Import Conventions in Tests + +```python +import flow360 as fl +from flow360 import SimulationParams, u, math, SI_unit_system +from flow360.component.simulation.services import validate_model +from flow360.exceptions import Flow360ValueError +from tests.utils import compare_to_ref, to_file_from_file_test +``` + +## Quick Commands + +```sh +# Run all V2 tests +poetry run pytest -rA tests/simulation -vv + +# Run a specific test file +poetry run pytest -rA tests/simulation/params/test_expressions.py -vv + +# Run a specific test by keyword +poetry run pytest -rA -k "test_expression_operators" -vv + +# Fast run (no coverage, mute warnings, fail fast) +poetry run pytest -rA tests/simulation --no-cov -W ignore --maxfail=1 + +# Coverage report +poetry run pytest -rA tests/simulation --cov-report=html --cov=flow360/component/simulation +``` + +_Update this AGENTS.md when testing conventions or infrastructure change._