How can I make a test that could merge multiple results of different command line into one result file

[zhiqiang-hhhh]

Hello, zilliz community~

I want to generate a result file with the same structure as vectordb_bench/results/QdrantCloud/result_20230727_standard_qdrantcloud.json (or any other result*.json in the results directory), so that I can update the recall & QPS page in my local WebUI.

I’ve already found that vectordb_bench/results/leaderboard_v2.json is generated by vectordb_bench/results/getLeaderboardDataV2.py. However, I still don’t know how to produce a single result file that merges outputs from multiple benchmark commands.

[alwayslove2013]

@zhiqiang-hhhh If you only intend to use the "QPS & Recall" page, there is no need to merge multiple result files into a single one. vdbbench will automatically collect all files in the results directory.

[zhiqiang-hhhh]

@zhiqiang-hhhh If you only intend to use the "QPS & Recall" page, there is no need to merge multiple result files into a single one. vdbbench will automatically collect all files in the results directory.

Thanks for reply.

I noticed that the results inside the results directory are automatically collected. However, for the QPS vs Recall sub-page, that page needs multiple groups of different recall values on the x-axis, each with corresponding QPS on the y-axis. If I run multiple tests with different parameters, I can indeed obtain different recall and QPS values. But this requires giving each test a different task-label (otherwise the generated result files have the same name), which means I must manually trigger the tests multiple times, and then manually select multiple task-labels on the QPS vs Recall page.

What I want is to write all these different test parameter groups into a single test configuration file and run everything once, using a single task-label. That would be much more convenient.

[alwayslove2013]

@zhiqiang-hhhh Apologize for the delay in getting back to you. From what I understand, you're hoping to run multiple sets of parameters for the same case in a single task execution.

Currently, this functionality isn't available through the web UI. However, you can achieve this using the batch-cli tool.