betterbench improvements

[brownsarahm]

• add option to keep a previous answer and show those
• make it write out without object declaration
• if CheckListItem class is not used more than current, just use a dictionary instead
• try to work the logic in so there's only one loop & one write (i.e. write blank if skipped; change q to just skip prompting and always use skip for the rest)
• use click.edit for the justifications

if it's a CLI itself, it cannot be called from within benchtool init

ideally, but less important

• switch module to be usable as python then re-add CLI
• improve layout of yaml format for betterbench, using example

instead of

# Design
"The tested capability, characteristic, or concept is defined": 
  - "Tested concept, capability, or characteristic not explicitly mentioned."
  - "Tested concept explicitly mentioned and need for definition acknowledged, but definition not provided."
  - "Tested concept, capability, or characteristic explicitly mentioned but not defined."
  - "Tested concept, capability, or characteristic explicitly mentioned and defined."

it would be better to have more structure

- category_name: Design
   criterion_text: "The tested capability, characteristic, or concept is defined" 
   criterion_id: definition
   rubric: 
       - na: ""
       - 0: "Tested concept, capability, or characteristic not explicitly mentioned."
       - 5:"Tested concept explicitly mentioned and need for definition acknowledged, but definition not provided."
       - 10:"Tested concept, capability, or characteristic explicitly mentioned but not defined."
       - 15:"Tested concept, capability, or characteristic explicitly mentioned and defined."

and then the per-benchmark wouldlook like

- criterion_id: definiton
   response
   justification
   score
   skipped 
   category_name: design

this will then also make the code way easier to read

[brownsarahm]

also allow the scores to be not just a multiple choice, but nay number, 0-15, giuded by the rubric

[AymanBx]

try to work the logic in so there's only one loop & one write (i.e. write blank if skipped; change q to just skip prompting and always use skip for the rest)
I want to make sure I understand correctly.

• I should remove the first loop that creates the empty checklist file
• Then make the q action skip question but continue to loop over the main_checklist to add the questions to the new checklist
• I'm thinking of making the click question stuff happen in a function call, are you ok with that idea?

[AymanBx]

use click.edit for the justifications

Can you please elaborate on this? Where would we need this for the justifications? Is it when the answer is yes? and what would be the edited text?

[AymanBx]

switch module to be usable as python then re-add CLI

By re-add CLI do you mean how I had it as
@benchtool.add_command(betterbench)
or something different

[brownsarahm]

I'm thinking of making the click question stuff happen in a function call, are you ok with that idea?

I dont understand. can you psuedocode?

switch module to be usable as python then re-add CLI

means: make betterbench checklist creation function as a python function that can accept all values or a dictionary, and then have CLI functionality in a separate moduel. can be either in the existing cli.py or a second cli focused odule

Can you please elaborate on this? Where would we need this for the justifications? Is it when the answer is yes? and what would be the edited text?

use that instead of whatver you have now when getting the justification text. it lets them use a real text editor.