Restructure benchmarks skill and rename to kaggle-benchmarks#1012
Restructure benchmarks skill and rename to kaggle-benchmarks#1012nicholaskang-us wants to merge 3 commits into
Conversation
|
@nicholaskang-us I think @rosbo and @stevemessick has a plan to organize the skills. Let's consult them first. |
|
|
||
| ``` | ||
| kaggle benchmarks (alias: kaggle b) | ||
| ├── auth — Fetch Model Proxy credentials |
There was a problem hiding this comment.
How well does this play with kaggle auth login? (Do we really need both?)
There was a problem hiding this comment.
kaggle auth login authenticates you with Kaggle.
Then, using your Kaggle credentials, kaggle benchmarks auth fetches a model proxy token. You must be autenticated before you can fetch a model proxy token.
There was a problem hiding this comment.
i'm assuming this means this is fine and we don't need to make any change?
| @@ -1,400 +0,0 @@ | |||
| # Kaggle Benchmarks CLI Reference | |||
There was a problem hiding this comment.
This file is referenced from the main SKILL: https://github.com/Kaggle/kaggle-cli/blob/main/skills/SKILL.md.
This will cause a broken link.
The idea is to have one skill with references to the different resources (e.g. kernels, models, datasets, benchmarks, etc).
There was a problem hiding this comment.
to clarify, you mean that we just expect users to download the kaggle-cli skill ONLY? and they would say they want to write a benchmark task, which would invoke the main skill
There was a problem hiding this comment.
The current thinking followed the playwright-cli structure where you have a SKILL.md explaining cross-cutting CLI concerns (like auth/authz) and then link to references file stored in the references/ folder for each product. One explaining how to use benchmark, one for kernels, etc.
Following this structure: https://agentskills.io/home
But maybe it is best to really have separate skill per concrete task (like creating a benchmark)... We can experiment and see how the agent performs.
There was a problem hiding this comment.
I would suggest to put your skill focused on creating a benchmark task e2e in the kaggle-skills repo given it is cross-cutting (kaggle-cli & kaggle-benchmarks).
This PR restructures the benchmarks skill to adhere to the Agent Skills standard and completes the truncated content.
What problem are we solving
What changes are proposed