Add generic observation processes which combine the convolution with the noise model. #644
Conversation
cdc-mitzimorris
requested review from
SamuelBrand1,
damonbayer and
dylanhmorris
as code owners
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #644 +/- ##
==========================================
+ Coverage 96.98% 97.26% +0.28%
==========================================
Files 42 47 +5
Lines 1094 1282 +188
==========================================
+ Hits 1061 1247 +186
- Misses 33 35 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thank you for your contribution @cdc-mitzimorris 🚀! Your github-pages is ready for download 👉 here 👈! |
…ents, 'aggregate' instead of 'jurisdiction'
…ents, 'aggregate' instead of 'jurisdiction'
for more information, see https://pre-commit.ci
… into mem_generic_observations
for more information, see https://pre-commit.ci
… into mem_generic_observations
|
In preparing to add the |
| expected_counts_safe = jnp.nan_to_num(expected_counts, nan=0.0) | ||
| predicted_counts = self._predicted_obs(infections) | ||
| self._deterministic("predicted_counts", predicted_counts) | ||
| predicted_counts_safe = jnp.nan_to_num(predicted_counts, nan=0.0) |
There was a problem hiding this comment.
^ This feels dangerous to do automatically and without warning. I think I would rather force the user to do it themself in _predicted_obs, where appropriate. Is there a good reason not to?
There was a problem hiding this comment.
The nan_to_num is applied to predicted_counts from the latent infection process, not to user-provided observations. The NaN values are a structural artifact of the convolution burn-in (first len(delay_pmf)-1 days).
cf
PyRenew/pyrenew/observation/count_observations.py
Lines 145 to 148 in d103d0f
The nan_to_num is a pragmatic choice: the model doesn't crash, and the behavior is noted in the docstring. It's not a safeguard against misuse — it's just handling a structural artifact so the code runs.
dylanhmorris
left a comment
dylanhmorris
left a comment
There was a problem hiding this comment.
Thanks @cdc-mitzimorris! A few things to address and then I can re-review.
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
… into mem_generic_observations
for more information, see https://pre-commit.ci
… into mem_generic_observations
|
@dylanhmorris - ready for re-review - made all suggested changes to the tutorials and added arg "name" to observation processes so that user specifies the signal name. |
dylanhmorris
left a comment
dylanhmorris
left a comment
There was a problem hiding this comment.
Thanks @cdc-mitzimorris! Just a couple remaining questions.
|
changes made. |
|
@dylanhmorris - conversation resolved. |
dylanhmorris
left a comment
dylanhmorris
left a comment
There was a problem hiding this comment.
Still need the separate noise module.
There was a problem hiding this comment.
@cdc-mitzimorris this still needs to be implemented.
| Output preserves input timeline. First len(delay_pmf)-1 days return | ||
| -1 or ~0 (depending on noise model) due to NaN padding. |
There was a problem hiding this comment.
What is meant by ~0? and why does the padding change depending on the noise model?
There was a problem hiding this comment.
I think we should discuss this f2f in our upcoming meeting.
| times : ArrayLike | None | ||
| Day indices for sparse observations. None for dense observations. |
There was a problem hiding this comment.
index relative to what vector?
This PR adds work that was done in https://github.com/cdcent/cfa-pyrenew-hierarchical/pull/4 to PyRenew.
It adds the base observation process class, concrete implementations for Count processes and the abstract base class for Measurement processes, together with unit tests and two new tutorials for count and measurement observation processes respectively.
Once this PR and the work done in https://github.com/cdcent/cfa-pyrenew-hierarchical/pull/5 have been added to PyRenew, subsequent PRs will deprecate unused features and harmonize the documentation and tutorials.