Currently we are using the running script (https://github.com/mmtk/running/tree/rust-mmtk) for configuring runs and executing benchmarks. However, there are some limitations of the running script, such as:
- It is designed for executing Java benchmarks, we probably cant use it for Javascript benchmarking.
- It assumes heap size for each benchmark and we can't set heap size for each build.
- It assumes the same iteration/invocation for all the benchmarks. We may want to run certain benchmarks with more invocations/iterations to get more reliable results.
We may consider using ReBench (https://github.com/smarr/ReBench). That is used by SOM across multiple SOM implementation. We need to investigate to see if ReBench is suitable.
Currently we are using the
runningscript (https://github.com/mmtk/running/tree/rust-mmtk) for configuring runs and executing benchmarks. However, there are some limitations of therunningscript, such as:We may consider using ReBench (https://github.com/smarr/ReBench). That is used by SOM across multiple SOM implementation. We need to investigate to see if ReBench is suitable.