Test: Assert specific evaluation scores for sample data

[musaqlain]

Description

Previously, test_evaluate only checked if the execution completed without errors, but did not assert the quality of the results. This PR adds assertions to ensure the model produces consistent precision and recall scores on the sample data.

Changes Made

• Relaxed Unit Test: I modified test_evaluate in tests/test_main.py to use looser bounds (> 0.7) so it acts as a sanity check for future models.
• New Benchmark Test: I created tests/test_benchmark.py which loads the specific model revision (SHA) using m.load_model(..., revision=...). This test asserts the exact precision/recall values (approx 0.80 / 0.72) to ensure reproducibility for this specific release.

Testing

• Ran pytest tests/test_main.py::test_evaluate locally.
• Result: Passed.

Linked Issue

Closes #1233

AI-Assisted Development

I utilized AI for sanity checking code logic and conducting an assisted review to identify potential missing resets and schema constraints.

• I used AI tools (e.g., GitHub Copilot, ChatGPT, etc.) in developing this PR
• I understand all the code I'm submitting

[jveitchmichaelis]

Thanks for the contribution. I would prefer a "benchmark" unit test that specifies the model via config as explicitly as possible (for example using a commit hash from hugging face, not just revision=main as that could also change), and then makes this check. Then we could add other tests for newer models etc.

We already have this assertion in the unit test in the PR:

assert np.round(results["box_precision"], 2) > 0.5
assert np.round(results["box_recall"], 2) > 0.5

which we could bump to 0.7 if we want tighter checks. But I don't think this test is the right place for a "close" assertion as it will break if we ever release an improved model.

Please could you also include the AI assistance declaration from the PR template? (you can see an example here)

[musaqlain]

Thanks for the guidance! I completely understand your point that separating the sanity checks from strict benchmarking is a better approach for long-term maintenance of this package. Now I have updated it accordingly also updated this PR's description. 👍

[ethanwhite]

This looks really close to me. Just one recommendation for clarifying a code comment. @jveitchmichaelis - do you see anything else?

[ethanwhite]

I'd change these comments to something like:

Check that precision and recall don't regress below reasonable baselines

[jveitchmichaelis]

We can remove this comment too

[jveitchmichaelis]

Looks like we also need to rebase, and there is another comment I would suggest removing as it doesn't provide much. I think that in the long run, we probably want to use tags or branches rather than opaque commit hashes but that's not an issue right now.