XLG Inspector Vegeta Fork

This repository is a fork of Vegeta.

All credit for the original Vegeta load-testing tool goes to the upstream Vegeta authors.

Overview

This fork instruments Vegeta with additional signals required by XLG Inspector. It also includes analysis scripts and experiments used to generate the findings in our paper.

Our main contributions are:

• Instrumentation of Vegeta with signals needed by XLG Inspector.
• Analysis scripts for reproducing the paper’s figures and tables.
• Experiments using Vegeta to evaluate load-generation behavior.
• A streaming experiment, experiments/test_xlg_window.sh, that evaluates the XLG tool in an online setting.

Data

The raw data required to reproduce the results is available in the shared data directory:

Shared Google Drive Data Directory

This directory contains the raw data, intermediate outputs, and experiment results needed for reproducing the figures and tables described below.

Reproducing Figures and Tables

Figures 1 and 2

Figures 1 and 2 can be reproduced from the cpu_results.zip file.

---

Figure 3

Figure 3 can be regenerated by running Vegeta with HTTP/1 and HTTP/2 configurations.

HTTP/1

./vegeta attack \
  -targets=<target file or stdin target> \
  -rate=5000/s \
  -duration=<duration> \
  -http2=false \
  -metrics-csv=congestion_experiments/http1_metrics.csv \
  -output=<optional results.bin>

HTTP/2

./vegeta attack \
  -targets=<target file or stdin target> \
  -rate=5000/s \
  -duration=<duration> \
  -metrics-csv=congestion_experiments/http2_metrics.csv \
  -output=<optional results.bin>

The plot is generated using:

python scripts/plot_congestion.py

This script assumes the existence of:

• congestion_experiments/http1_metrics.csv
• congestion_experiments/http2_metrics.csv

Both files are available in the shared data directory under congestion_results/.

---

Figure 4

Figure 4 can be regenerated using the experiments and shell scripts in:

worker_cap_sweep_new/

This experiment assumes the existence of the manual stall server, which is provided in:

cmd/manual_stall_server/

The plot is generated by:

python plot_peak_server_completions.py

This plotting script assumes the existence of a runs/ directory containing the raw experiment data.

The corresponding raw data is available in the shared data directory as worker_cap_runs/.

---

Figure 5

Figure 5 can be reproduced by running:

python experiments/run_misleading_throughput.py

The results are plotted using:

python scripts/plot_misleading_throughput.py

This experiment assumes the existence of the burst server, which is provided in:

cmd/burst_server/

The raw data files are in the shared folder under misleading_data/

---

Table 4 and Figure 7

Figure 7 can be reproduced by running the full evaluation pipeline.

The relevant pipeline scripts are located in:

experiments_eval/
scripts_eval/

Depending on the environment, these scripts may require minor adaptation.

To send appropriate requests to the microservices, we provide:

python scripts/generate_targets.py

This script generates appropriate commands for HotelReservation.

A similar script was written for SocialNetwork, but it is not included in this repository.

The shared data directory includes all raw and intermediate data needed for this evaluation, including:

• .bin files containing server responses
• Intermediate outputs from the evaluation pipeline
• Experiment results for:
  • ExpServer_result.zip
  • HotelReservation_results.zip
  • SocialNetwork_results.zip

The confusion matrix and values in Table 4 were produced by running scripts_eval/stage_b_cross_rps.py

---

Table 2 and Figure 8

Table 2 and Figure 8 can be reproduced by running:

python plot_delay_ecdfs.py

This script uses

HotelReservation_results.zip
SocialNetwork_results.zip

---

Table 3

Table 3 can be reproduced from the data provided in the shared data directory.

The relevant files are:

streaming_overhead.zip
vanilla_overhead.zip

These are provided in the shared data folder as overhead_experiments/

Streaming XLG Experiment

The main streaming experiment is provided in:

experiments/test_xlg_window.sh

This script tests the XLG tool in a streaming fashion and represents the main contribution of this fork. It can really be run with any server as long as its running on localhost:8080 (obviously, this can be changed to), but we recommend running it with cmd/ramping_regime_shift_server/main.go, since that tickles every coniditon other than CPU contention.

Notes

Some experiments may require minor environment-specific modifications, particularly the full evaluation pipeline for Figure 7.

The shared data directory contains the raw data and intermediate outputs necessary to reproduce the reported results without rerunning every experiment from scratch.