Skip to content

v1 MVP: GB + NL#306

Open
braddf wants to merge 120 commits into
mainfrom
v1/mvp
Open

v1 MVP: GB + NL#306
braddf wants to merge 120 commits into
mainfrom
v1/mvp

Conversation

@braddf
Copy link
Copy Markdown
Contributor

@braddf braddf commented May 6, 2026

Pull Request

Description

Closes #199

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

braddf added 30 commits May 6, 2026 16:50
Updates import path from dp_sdk.ocf to ocf across all call sites.
Adds /forecasts/timeseries and /generation/timeseries returning ForecastMatrix
and GenerationMatrix respectively. Cache is pre-warmed per-region on startup
via _warm_all_v1_caches (config-driven from COUNTRIES). Cold cache returns 503
+ Retry-After: 60. Admin refresh endpoints POST /forecasts/refresh and
POST /generation/refresh trigger background re-warm.

Also renames /forecasts -> /forecasts/snapshot and /generation ->
/generation/snapshot, and makes region_type required on both snapshot routes.
23 tests covering discovery, region browsing, per-region forecast and
generation, snapshot endpoints, timeseries cold/warm cache behaviour,
and admin refresh auth.
… and 400 paths

- parent_id filter on region list
- invalid region_type → 400 on regions, snapshot, and timeseries
- national snapshot fallback code path (uses get_predicted_generation)
- region_ids filter reduces timeseries results to matching regions only
- FixedUUIDStorageClient subclass enables per-region cache key injection
- Two window tests verify start_utc/end_utc filtering includes/excludes values
- Extract _set_forecast_meta and _set_fixed_region_values helpers
- Fix fixture return type annotations to AsyncGenerator
- Encode cache values as bytes to match backend type signature
All v1 route paths changed from /{source}/{country}/… to /{country}/{source}/…
to align with Auth0 permission naming (gb_solar_regional etc.) where country is
the commercial subscription boundary. Top-level nations discovery renamed from
GET /{source}/regions to GET /countries. Cache key order updated to match.
- v1_app gets its own OpenAPI schema, Swagger UI at /v1/docs with
  auto-authorize JS, and isolated middleware scope
- Root middleware (CORS, SlowAPI, Sentry, Apitally, Tracer) wraps the
  whole ASGI tree so v1_app carries only app.state.limiter
- _create_v1_app factory wires OAuth2 init_oauth when auth0 is configured
- _lifespan sets v1_app dependency overrides and passes v1_app to warm task
- Router loop skips v1; sub-app mounted after auth config is resolved
StrEnum derived from COUNTRIES at import time so Swagger renders a
dropdown of valid values; adding a new country to country_config.py
updates the enum automatically.

Unknown country codes now return 422 (FastAPI path validation) rather
than 404 from inside the handler — more accurate and caught earlier.
…mapping

- nation_name set to "nl_national" to match DP location name
- National and netbeheerder region types populated with NL PVNet model variants
- NED NL generation source added (observer: nednl)
- Nation name comparisons made case-insensitive throughout router
- GET /countries now returns CountryDetail: full capability manifest
  including region_types with forecast_models and generation_sources,
  so clients need only one preflight call instead of three
- ValidForecastModel enum derived from all models in COUNTRIES config;
  model query param on forecast endpoints now shows a Swagger dropdown
- ValidObserver already used this pattern; model param now consistent
Raises 400 with a clear "Available: [...]" list before the DP call,
instead of letting an invalid model name through to gRPC.
…rs list in DP client

JsonFormatter now includes exc_info as an "exc" field so uvicorn ASGI
exceptions are visible in structured logs. DP client raises 404 instead
of IndexError when list_forecasters returns no results for the named model.
…ch DP

Adds FORECASTER_LABELS dict mapping DP forecaster_name -> display label,
and a _model() helper so country configs reference DP names only once.
GB national models updated to actual DP names:
  pvnet_intraday        -> pvnet_intra_allbells0
  pvnet_intraday_ecmwf_only -> pvnet_ecmwf
  pvnet_intraday_met_office_only -> pvnet_ukv_only
  pvnet_intraday_sat_only -> pvnet_sat_only
blend_adjust added to GB national.
…ndlers

Each region type now declares a default_model. When no model param is
passed, the handler falls back to the config default rather than letting
None propagate to the DP (which picks arbitrarily). Defaults:
  GB national → blend_adjust, GB gsp/dno → blend
  NL national/netbeheerder → nl_regional_pv_ecmwf_mo_sat_adjust
router.py → thin assembler (12 lines)
helpers.py → shared utilities (CountryCode enum, _energy_type_for, _resolve_*, etc.)
cache.py → warming logic + state dicts (_warm_all_v1_caches, now imported from here)
routes/capability.py → tag="Capability" (sources, countries, region-types, generation-sources)
routes/regions.py → tag="Regions" (list regions, get region)
routes/forecasts.py → tag="Forecasts" (per-region, snapshot, timeseries, refresh)
routes/generation.py → tag="Generation" (per-region, snapshot, timeseries, refresh)

Swagger UI now shows 4 labelled sections. All 32 tests pass.
braddf and others added 30 commits May 12, 2026 15:29
* lint

* lint

* add protobuf back in

* move tests back to what was merged in #274
- created_time/init_time → created_utc/init_utc on all response models
  (ForecastResponse, ForecastSnapshot, RegionForecastMatrix) and cache meta
- timestamp query param → time_utc on /forecasts/snapshot and
  /generation/snapshot routes
…sponses

Added to ForecastResponse, GenerationResponse, RegionForecastValue,
RegionGenerationValue, RegionForecast, and RegionGeneration. Snapshot
routes build a UUID→name map from the locations fetch; period matrix
routes use the live location objects. Nation display_name override
applied via the new _location_display_name helper.
…arch

- Bump dp-sdk to v0.29.0 which adds location_names_filter to ListLocationsRequest
- Add location_names param to get_locations across StorageInterface, DP client,
  dummydb, and quartzdb
- _resolve_region_id now delegates name search to DP (single ListLocations call)
  instead of iterating all region types client-side
- Both period routes (/forecasts/period, /generation/period) now accept
  region_names as well as region_ids; the union of both sets is applied
…routes

Accepts UUID, 'national' slug, or region name — the param name now reflects
that any of the three formats is valid.
# Conflicts:
#	src/quartz_api/cmd/main.py
Instead of flooring to 'now' (which rarely has data), probe the region
with the highest gsp_id over the last 6 hours and use the latest
valid_timestamp as the snapshot time.
…lay names

Adds location_name_map to RegionTypeConfig for user-facing name overrides
without changing DP names. _location_display_name now checks the map before
falling back to loc.name. NL provinces mapped to lowercase names (e.g.
nl_region_2_friesland -> friesland). Compound province DP keys need
verification against the data platform.
…ifier

Region names now serve as the only user-facing region identifier. Removes
region_id/id UUID fields from all response models (RegionSummary, RegionDetail,
ForecastResponse, GenerationResponse, snapshot value types, and matrix region
types). Renames {region_id} path param to {region} and parent_id query param
to parent (accepts name, 'national', or UUID — resolved internally). Removes
region_ids UUID filter from period endpoints in favour of region_names only.
… registry

- Change intraday_models/intraday_default_model from string internal-name
  references to ForecastModel objects, consistent with forecast_models
- Add FM class as single source of truth for all model definitions — slugs
  and internal names are defined once and referenced everywhere
- Simplify intraday_api_names() now that it can iterate ForecastModel directly
- Update _resolve_forecast_model to use ForecastModel objects directly
- GSP forecast_models now includes blend_adjust; intraday restricted to blend_adjust
- Intraday auth tests read model names from config rather than hardcoding strings
…y subset

GSP forecast_models now includes pvnet_intraday and pvnet_day_ahead alongside
blend (no adjusted variants — trend adjuster not run at GSP level). Intraday
test now asserts that intraday_models is always a subset of forecast_models so
a misconfigured config fails with a clear message rather than a silent 400.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🎯 Quartz API v1

3 participants