Skip to content

Statistics for multiple reruns#1

Open
ageneric wants to merge 4 commits into
masterfrom
renew_statistics
Open

Statistics for multiple reruns#1
ageneric wants to merge 4 commits into
masterfrom
renew_statistics

Conversation

@ageneric
Copy link
Copy Markdown
Collaborator

Additions:

  • aggregate_regrading.py to perform repeated runs and collect statistics

Changes:

  • Updated documentation
  • Relocated unused configs to legacy/

Technical changes:

  • For readability, renamed "token_usage" to "text_generation" to better describe the functionality the script provides
  • Removed duplicate 'local' copies of functions related to reasoning models
  • Instead of setting reasoning effort to "medium", simply use the default reasoning effort (which is "medium" anyway)
  • Will upload result files in next commit (didn't want to bury the code changes in all the added files)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant