Skip to content

chore: drop metrics from schema#277

Open
viraatc wants to merge 1 commit intomainfrom
feat/viraatc-drop-metrics-schema
Open

chore: drop metrics from schema#277
viraatc wants to merge 1 commit intomainfrom
feat/viraatc-drop-metrics-schema

Conversation

@viraatc
Copy link
Copy Markdown
Collaborator

@viraatc viraatc commented Apr 10, 2026

What does this PR do?

based on offline discussion, we want to drop metrics from schema (it was unused anyways).

Type of change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor/cleanup

Related issues

Testing

  • Tests added/updated
  • All tests pass locally
  • Manual testing completed

Checklist

  • Code follows project style
  • Pre-commit hooks pass
  • Documentation updated (if needed)

@viraatc viraatc requested review from a team and Copilot April 10, 2026 01:13
@github-actions
Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes the unused metrics configuration section from the benchmark configuration schema and updates templates/examples/tests accordingly so configs no longer include or validate that field.

Changes:

  • Removed the Metrics Pydantic model and metrics field from BenchmarkConfig.
  • Updated config templates and example YAMLs to drop the metrics: section.
  • Updated unit tests to stop including/validating metrics in YAML/schema tests.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/unit/config/test_yaml_loader.py Removes metrics from a valid YAML config used in loader tests.
tests/unit/config/test_schema.py Removes Metrics import and the Metrics unit tests.
src/inference_endpoint/config/templates/submission_template.yaml Drops metrics section from submission template.
src/inference_endpoint/config/templates/online_template_full.yaml Drops metrics section from online full template.
src/inference_endpoint/config/templates/offline_template_full.yaml Drops metrics section from offline full template.
src/inference_endpoint/config/templates/eval_template.yaml Drops metrics section from eval template.
src/inference_endpoint/config/templates/concurrency_template_full.yaml Drops metrics section from concurrency full template.
src/inference_endpoint/config/schema.py Removes Metrics model and BenchmarkConfig.metrics field.
examples/08_Qwen3-VL-235B-A22B_Example/online_qwen3_vl_235b_a22b_shopify.yaml Drops metrics section from example config.
examples/08_Qwen3-VL-235B-A22B_Example/offline_qwen3_vl_235b_a22b_shopify.yaml Drops metrics section from example config.
examples/06_Llama2-70B_Example/online_llama2_70b_orca.yaml Drops metrics section from example config.
examples/05_Llama3.1-8B_Example/online_llama3_8b_cnn.yaml Drops metrics section from example config.
examples/05_Llama3.1-8B_Example/offline_llama3_8b_cnn.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/vllm_gptoss_120b_example.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/sglang_gptoss_120b_example.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/gptoss_120b_example.yaml Drops metrics section from example config.
examples/02_ServerBenchmarking/online_llama2_70b_cnn.yaml Drops metrics section from example config.
examples/02_ServerBenchmarking/offline_llama3_8b_cnn.yaml Drops metrics section from example config.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes the metrics collection configuration and associated logic from the codebase, including the Metrics model, its helper methods, and all references in YAML configuration templates and unit tests. I have no feedback to provide as the changes are straightforward and clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants