-
Notifications
You must be signed in to change notification settings - Fork 118
[feat] Support multiple aggregations and statistics at once when querying performance data #3585
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #3585 +/- ##
===========================================
+ Coverage 91.38% 91.40% +0.01%
===========================================
Files 62 62
Lines 13484 13505 +21
===========================================
+ Hits 12323 12344 +21
Misses 1161 1161 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
541d9b9 to
f9f4fcc
Compare
f9f4fcc to
79fd962
Compare
79fd962 to
6dfab74
Compare
6dfab74 to
cc60ca8
Compare
ekouts
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't finished my testing, but stats seems to be broken for --performance-report:
./bin/reframe -c my_checks/perf_vars.py -r --performance-report "now-1d:now/stats:/"
ReFrame Setup]
version: 4.10.0-dev.0+cc60ca88
....
[==========] Finished on Mon Jan 26 11:38:45 2026+0100
WARNING: failed to generate performance report: "pval (std)" not found
Current session stored with UUID: e0a328f6-0ff3-4eeb-85fb-eb0047f17040
Log file(s) saved in '/var/folders/19/lkdv1y8949g05ynf917f07mm0000gn/T/rfm-a76sgaqq.log'
When the database is disabled, reframe doesn't complain about this, just returns the default results.
This PR offers the following features:
statsaggregator. These includes mean, standard deviation, and 0.01, 0.05, 0.50, 0.95, 0.99 quantiles.mean,min,max, and also aggregations on values other than the performance, e.g.,min(num_tasks),min(pref)etc.--term-lhsand--term-rhsand are only meaningful with the--performance-compareand--performance-reportoptions.(lhs)and(rhs)respectively._Land_Rinstead of_Aand_B.Closes #3436.
Implementation details
The analytics layers is now using Polars through its Python bindings.
Managing the column names was the most challenging part of the implementation, as we wanted to keep a straightforward for selecting test attributes from the command line, specifying the left/right variants quickly and the same time produce tables with meaningful headers (e.g., include the aggregation in the column name, especially now, that multiple aggregations are supported at once). This is handled the the internal
_QueryMatchclass which replaces the very thin former_Matchnamed tuple.Todos
--table-format=prettystill uses|as a separator forjoin_uniq.Notes
If the CI does not pass for Python < 3.9, we should simply merge this PR after #3583 is addressed.This PR depends on #3596.