Raster data registry, cached reading, custom api endpoints#3

Merged

karimbahgat merged 29 commits intomainfrom

data-registry-and-caching

Feb 26, 2026

Commits on Feb 23, 2026

first partial commit of dataset registry and caching
Karim Bahgat
committed
chore: allow arbitrary dhis2eo download funcs, get dataset cache info such as coverage, more
Karim Bahgat
committed
chore: define datasets router prefix centrally
Karim Bahgat
committed
complete aggregate pipeline with unit conversion, hacky constants module
Karim Bahgat
committed

Commits on Feb 24, 2026

make aggregation statistics dynamic and not hardcoded for each dataset
Karim Bahgat
committed
reorganize api so time and space aggregation happens hierarchically beneath dataset endpoint
Karim Bahgat
committed
move some funcs to utils.py
Karim Bahgat
committed
add raster with time period aggrgation download endpoint as part of the hierarchy
Karim Bahgat
committed
add dummy tiles endpoint
Karim Bahgat
committed
apply unit conversion earlier so it also affects raster download
Karim Bahgat
committed
switch to proper logging
Karim Bahgat
committed
check and raise dataset id errors more gracefully
Karim Bahgat
committed
validate target period type and skip temporal aggregation if same period type
Karim Bahgat
committed
fix misc dataset metadata
Karim Bahgat
committed
fix period type validation bug, redo json serialization and show correct period format
Karim Bahgat
committed
easier to switch out to different org units, cache builds to latest date by default
Karim Bahgat
committed
switch to more reliable cache background worker, fix dynamic bbox error
Karim Bahgat
committed
speedup unit conversion by doing it on fewer values, stabilize internal code so we always work with an xarray dataset, dask compute after time aggregation to avoid very slow dask graphs later
Karim Bahgat
committed
add cache optimization which builds zarr archive, dataset openers uses zarr file if it exists which is much faster
Karim Bahgat
committed

Commits on Feb 25, 2026

improve read and write of zarr cache
Karim Bahgat
committed
fix same period type aggregation error, clarify array subsetting in time aggregation
Karim Bahgat
committed
smaller chunk size for improved read speed, memory, and optimize build time for large countries like brazil
Karim Bahgat
committed
add simple map image preview endpoints for orgunits and raster endpoints
Karim Bahgat
committed
fix more proper preview map figure generation that avoids thread-unsafe issues
Karim Bahgat
committed

Commits on Feb 26, 2026

upgrade to dhis2eo v1.1.1, tweak preview figures
Karim Bahgat
committed
merge with main
Karim Bahgat
committed
remove old files, make datasets module work with new setup
Karim Bahgat
committed
update docstring for preview endpoints
Karim Bahgat
committed
some ruff fixes, ignore datasets folder for ruff and mypy checks
Karim Bahgat
committed