Allow for custom evaluation procedures

[PGijsbers]

Based on this thread. We should discuss whether we want to allow people to create custom evaluation procedures (e.g., “30 times 3-fold Crossvalidation”) and how we want to support this. Possible concerns include spotty availability (might not support the entire tasktype x procedure matrix) and splintering experimental results. But there are of course also obvious benefits :)

[Jayant-kernel]

@PGijsbers
I’m supporting custom eval procedures, but we should keep comparability.
i thhink
allow custom procedures only as a variant of a known base (e.g., base=10‑fold CV, params: folds/repeats/seed), and whitelist task types that can use it. That keeps leaderboards anchored on the base while still allowing flexibility. Happy to help draft a concrete schema/API if this direction makes sense.