Tag intermediate `initializationError` retries with `test.final_status=skip` in JUnit XML by cbeauchesne · Pull Request #11010 · DataDog/dd-trace-java

cbeauchesne · 2026-03-31T13:01:46Z

Motivation

When a JUnit setup method (e.g. @BeforeAll) fails and is retried via Gradle's retry plugin, Gradle generates a synthetic <testcase name="initializationError"> for each attempt. If the final retry succeeds, the build passes, but Test Optimization receives all intermediate failure entries with no indication that they were retried, making them appear as genuine failures in the dashboard.

What Does This Do

Add a doLast post-processor to every Test task that rewrites the JUnit XML reports after execution. For any suite with multiple initializationError testcases (i.e. retries occurred), all entries except the last one are tagged with:

<properties>                                                                                                                                                              
    <property name="dd_tags[test.final_status]" value="skip" />                                                                                                             
</properties>

The last entry is left unmodified, allowing Test Optimization to apply its default status inference based on the actual outcome. Files with only one (or zero) initializationError testcases are not modified.

The post-processor runs as a doLast action directly on the test task, keeping it within the task's up-to-date and caching scope so it doesn't interfere with downstream consumers of the JUnit reports.

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

buildSrc/src/main/kotlin/dd-trace-java.configure-tests.gradle.kts

github-actions · 2026-04-02T17:28:26Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

PerfectSlayer · 2026-04-07T13:32:20Z

.gitlab/TagInitializationErrors.java

+      System.err.println("File not found: " + xmlFile);
+      System.exit(1);
+    }
+    var doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(xmlFile);


❔ question: ‏Should we add some flags about entity resolution (for example) here to prevent security issue?

Which security issue do you have in mind ? The entire workflow and data are derivated from the public content of this repo, and the script itself can be modified during a PR.

I think @PerfectSlayer refers to XML external entity, or other tricks: https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html#java.

If it's those, everything that runs here is produced by the PR content, including the script that execute the command. So i'm don't think that there is any increase in the surface attack.

Indeed, that said I think it doesn't hurt to have them at least as a "processing" contract, and it's simple it's just configuring the factory.

PerfectSlayer · 2026-04-07T13:38:28Z

.gitlab/TagInitializationErrors.java

+ *
+ * <p>Gradle generates synthetic "initializationError" testcases in JUnit reports for setup methods.
+ * When a setup is retried and eventually succeeds, multiple testcases are created, with only the
+ * last one passing. All intermediate attempts are marked skip so Test Optimization is not misled.


🎯 suggestion: ‏It would help if you describe the expected changes here. You can re-use stuff from the PR description 😉

Done in 899ee14

I wonder if you could provide in the javadoc a sample of the non modified junit test file, and the expected output.

Also noteworthy to know, since this code is running on Java 25 it's possible to use markdown javadoc: https://blog.jetbrains.com/idea/2025/04/markdown-in-java-docs-shut-up-and-take-my-comments/

Addressed in 1b5a33e

pr-commenter · 2026-04-07T14:35:59Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/remove-initialization-error
git_commit_date	1775145366	1775807614
git_commit_sha	`2365c12`	`3494f99`
release_version	1.61.0-SNAPSHOT~2365c1251f	1.61.0-SNAPSHOT~3494f99b04

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775809499	1775809499
ci_job_id	1584314866	1584314866
ci_pipeline_id	107031680	107031680
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-kowv9og7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-kowv9og7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 10 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:petclinic:profiling:Remote Config	better [-53.954µs; -15.920µs] or [-9.135%; -2.695%]	555.714µs	590.651µs

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1061861
Total [baseline] (11.123 s) : 0, 11122890
Agent [candidate] (1.064 s) : 0, 1064231
Total [candidate] (11.113 s) : 0, 11112663
section appsec
Agent [baseline] (1.252 s) : 0, 1251744
Total [baseline] (11.184 s) : 0, 11184237
Agent [candidate] (1.249 s) : 0, 1248782
Total [candidate] (11.131 s) : 0, 11131358
section iast
Agent [baseline] (1.227 s) : 0, 1227077
Total [baseline] (11.417 s) : 0, 11416964
Agent [candidate] (1.232 s) : 0, 1231560
Total [candidate] (11.403 s) : 0, 11403212
section profiling
Agent [baseline] (1.191 s) : 0, 1190856
Total [baseline] (11.169 s) : 0, 11169336
Agent [candidate] (1.182 s) : 0, 1181915
Total [candidate] (11.113 s) : 0, 11112528

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	appsec	1.252 s	189.883 ms (17.9%)
Agent	iast	1.227 s	165.216 ms (15.6%)
Agent	profiling	1.191 s	128.995 ms (12.1%)
Total	tracing	11.123 s	-
Total	appsec	11.184 s	61.347 ms (0.6%)
Total	iast	11.417 s	294.074 ms (2.6%)
Total	profiling	11.169 s	46.445 ms (0.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.064 s	-
Agent	appsec	1.249 s	184.551 ms (17.3%)
Agent	iast	1.232 s	167.329 ms (15.7%)
Agent	profiling	1.182 s	117.684 ms (11.1%)
Total	tracing	11.113 s	-
Total	appsec	11.131 s	18.695 ms (0.2%)
Total	iast	11.403 s	290.549 ms (2.6%)
Total	profiling	11.113 s	-135.009 µs (-0.0%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (633.976 ms) : 0, 633976
BytebuddyAgent [candidate] (637.853 ms) : 0, 637853
AgentMeter [baseline] (29.787 ms) : 0, 29787
AgentMeter [candidate] (29.679 ms) : 0, 29679
GlobalTracer [baseline] (250.919 ms) : 0, 250919
GlobalTracer [candidate] (250.755 ms) : 0, 250755
AppSec [baseline] (32.376 ms) : 0, 32376
AppSec [candidate] (32.333 ms) : 0, 32333
Debugger [baseline] (60.436 ms) : 0, 60436
Debugger [candidate] (60.661 ms) : 0, 60661
Remote Config [baseline] (600.191 µs) : 0, 600
Remote Config [candidate] (612.583 µs) : 0, 613
Telemetry [baseline] (8.088 ms) : 0, 8088
Telemetry [candidate] (8.131 ms) : 0, 8131
Flare Poller [baseline] (8.203 ms) : 0, 8203
Flare Poller [candidate] (6.665 ms) : 0, 6665
section appsec
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.211 ms) : 0, 1211
BytebuddyAgent [baseline] (662.685 ms) : 0, 662685
BytebuddyAgent [candidate] (662.141 ms) : 0, 662141
AgentMeter [baseline] (12.104 ms) : 0, 12104
AgentMeter [candidate] (12.099 ms) : 0, 12099
GlobalTracer [baseline] (249.887 ms) : 0, 249887
GlobalTracer [candidate] (248.695 ms) : 0, 248695
AppSec [baseline] (185.025 ms) : 0, 185025
AppSec [candidate] (184.58 ms) : 0, 184580
Debugger [baseline] (66.712 ms) : 0, 66712
Debugger [candidate] (66.405 ms) : 0, 66405
Remote Config [baseline] (613.282 µs) : 0, 613
Remote Config [candidate] (599.511 µs) : 0, 600
Telemetry [baseline] (8.689 ms) : 0, 8689
Telemetry [candidate] (8.535 ms) : 0, 8535
Flare Poller [baseline] (3.669 ms) : 0, 3669
Flare Poller [candidate] (3.49 ms) : 0, 3490
IAST [baseline] (24.585 ms) : 0, 24585
IAST [candidate] (24.525 ms) : 0, 24525
section iast
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (802.56 ms) : 0, 802560
BytebuddyAgent [candidate] (806.836 ms) : 0, 806836
AgentMeter [baseline] (11.396 ms) : 0, 11396
AgentMeter [candidate] (11.462 ms) : 0, 11462
GlobalTracer [baseline] (239.377 ms) : 0, 239377
GlobalTracer [candidate] (239.401 ms) : 0, 239401
AppSec [baseline] (31.252 ms) : 0, 31252
AppSec [candidate] (31.967 ms) : 0, 31967
Debugger [baseline] (59.471 ms) : 0, 59471
Debugger [candidate] (58.82 ms) : 0, 58820
Remote Config [baseline] (531.435 µs) : 0, 531
Remote Config [candidate] (529.619 µs) : 0, 530
Telemetry [baseline] (15.004 ms) : 0, 15004
Telemetry [candidate] (14.952 ms) : 0, 14952
Flare Poller [baseline] (3.643 ms) : 0, 3643
Flare Poller [candidate] (4.056 ms) : 0, 4056
IAST [baseline] (25.892 ms) : 0, 25892
IAST [candidate] (26.01 ms) : 0, 26010
section profiling
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (692.622 ms) : 0, 692622
BytebuddyAgent [candidate] (690.665 ms) : 0, 690665
AgentMeter [baseline] (9.195 ms) : 0, 9195
AgentMeter [candidate] (9.101 ms) : 0, 9101
GlobalTracer [baseline] (209.428 ms) : 0, 209428
GlobalTracer [candidate] (206.515 ms) : 0, 206515
AppSec [baseline] (32.997 ms) : 0, 32997
AppSec [candidate] (32.539 ms) : 0, 32539
Debugger [baseline] (66.928 ms) : 0, 66928
Debugger [candidate] (65.436 ms) : 0, 65436
Remote Config [baseline] (590.651 µs) : 0, 591
Remote Config [candidate] (555.714 µs) : 0, 556
Telemetry [baseline] (7.944 ms) : 0, 7944
Telemetry [candidate] (7.79 ms) : 0, 7790
Flare Poller [baseline] (3.652 ms) : 0, 3652
Flare Poller [candidate] (3.503 ms) : 0, 3503
ProfilingAgent [baseline] (94.823 ms) : 0, 94823
ProfilingAgent [candidate] (93.362 ms) : 0, 93362
Profiling [baseline] (95.4 ms) : 0, 95400
Profiling [candidate] (93.916 ms) : 0, 93916

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067173
Total [baseline] (8.895 s) : 0, 8895138
Agent [candidate] (1.056 s) : 0, 1056317
Total [candidate] (8.862 s) : 0, 8861907
section iast
Agent [baseline] (1.222 s) : 0, 1222148
Total [baseline] (9.575 s) : 0, 9574900
Agent [candidate] (1.224 s) : 0, 1224400
Total [candidate] (9.577 s) : 0, 9577011

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.222 s	154.975 ms (14.5%)
Total	tracing	8.895 s	-
Total	iast	9.575 s	679.762 ms (7.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	iast	1.224 s	168.083 ms (15.9%)
Total	tracing	8.862 s	-
Total	iast	9.577 s	715.104 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.221 ms) : 0, 1221
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (639.602 ms) : 0, 639602
BytebuddyAgent [candidate] (632.513 ms) : 0, 632513
AgentMeter [baseline] (29.757 ms) : 0, 29757
AgentMeter [candidate] (29.268 ms) : 0, 29268
GlobalTracer [baseline] (251.474 ms) : 0, 251474
GlobalTracer [candidate] (248.525 ms) : 0, 248525
AppSec [baseline] (32.353 ms) : 0, 32353
AppSec [candidate] (32.096 ms) : 0, 32096
Debugger [baseline] (60.027 ms) : 0, 60027
Debugger [candidate] (59.623 ms) : 0, 59623
Remote Config [baseline] (597.629 µs) : 0, 598
Remote Config [candidate] (599.564 µs) : 0, 600
Telemetry [baseline] (8.146 ms) : 0, 8146
Telemetry [candidate] (8.096 ms) : 0, 8096
Flare Poller [baseline] (7.515 ms) : 0, 7515
Flare Poller [candidate] (8.209 ms) : 0, 8209
section iast
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (800.362 ms) : 0, 800362
BytebuddyAgent [candidate] (801.836 ms) : 0, 801836
AgentMeter [baseline] (11.397 ms) : 0, 11397
AgentMeter [candidate] (11.402 ms) : 0, 11402
GlobalTracer [baseline] (238.698 ms) : 0, 238698
GlobalTracer [candidate] (238.938 ms) : 0, 238938
AppSec [baseline] (31.679 ms) : 0, 31679
AppSec [candidate] (27.822 ms) : 0, 27822
Debugger [baseline] (57.429 ms) : 0, 57429
Debugger [candidate] (61.716 ms) : 0, 61716
Remote Config [baseline] (533.967 µs) : 0, 534
Remote Config [candidate] (528.162 µs) : 0, 528
Telemetry [baseline] (14.26 ms) : 0, 14260
Telemetry [candidate] (14.369 ms) : 0, 14369
Flare Poller [baseline] (3.977 ms) : 0, 3977
Flare Poller [candidate] (4.358 ms) : 0, 4358
IAST [baseline] (25.79 ms) : 0, 25790
IAST [candidate] (25.784 ms) : 0, 25784

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/remove-initialization-error
git_commit_date	1775145366	1775807614
git_commit_sha	`2365c12`	`3494f99`
release_version	1.61.0-SNAPSHOT~2365c1251f	1.61.0-SNAPSHOT~3494f99b04

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775810008	1775810008
ci_job_id	1584314867	1584314867
ci_pipeline_id	107031680	107031680
cpu_model	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-gyp1f57p 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-gyp1f57p 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 4 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-199.679µs; -77.214µs] or [-7.010%; -2.711%]	unsure [-614.072µs; -93.457µs] or [-7.435%; -1.132%]	unstable [-56.486op/s; +199.736op/s] or [-4.498%; +15.905%]	2.710ms	7.905ms	1327.406op/s	2.849ms	8.259ms	1255.781op/s
scenario:load:insecure-bank:iast_FULL:high_load	better [-426.712µs; -189.720µs] or [-6.954%; -3.092%]	better [-1075.331µs; -370.909µs] or [-7.315%; -2.523%]	unstable [-33.117op/s; +99.117op/s] or [-4.905%; +14.680%]	5.828ms	13.976ms	708.188op/s	6.136ms	14.699ms	675.188op/s
scenario:load:petclinic:profiling:high_load	better [-1.852ms; -0.812ms] or [-9.092%; -3.987%]	unsure [-2.582ms; -0.212ms] or [-8.059%; -0.663%]	unstable [-15.970op/s; +42.407op/s] or [-7.034%; +18.679%]	19.043ms	30.647ms	240.250op/s	20.375ms	32.045ms	227.031op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.728 ms) : 18537, 18919
.   : milestone, 18728,
appsec (19.041 ms) : 18849, 19233
.   : milestone, 19041,
code_origins (18.504 ms) : 18322, 18686
.   : milestone, 18504,
iast (18.61 ms) : 18422, 18797
.   : milestone, 18610,
profiling (20.56 ms) : 20347, 20772
.   : milestone, 20560,
tracing (18.301 ms) : 18122, 18481
.   : milestone, 18301,
section candidate
no_agent (18.683 ms) : 18495, 18871
.   : milestone, 18683,
appsec (19.693 ms) : 19492, 19894
.   : milestone, 19693,
code_origins (18.56 ms) : 18375, 18744
.   : milestone, 18560,
iast (18.293 ms) : 18112, 18474
.   : milestone, 18293,
profiling (19.427 ms) : 19232, 19622
.   : milestone, 19427,
tracing (18.848 ms) : 18659, 19038
.   : milestone, 18848,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.728 ms [18.537 ms, 18.919 ms]	-
appsec	19.041 ms [18.849 ms, 19.233 ms]	312.814 µs (1.7%)
code_origins	18.504 ms [18.322 ms, 18.686 ms]	-224.2 µs (-1.2%)
iast	18.61 ms [18.422 ms, 18.797 ms]	-118.27 µs (-0.6%)
profiling	20.56 ms [20.347 ms, 20.772 ms]	1.832 ms (9.8%)
tracing	18.301 ms [18.122 ms, 18.481 ms]	-426.599 µs (-2.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.683 ms [18.495 ms, 18.871 ms]	-
appsec	19.693 ms [19.492 ms, 19.894 ms]	1.01 ms (5.4%)
code_origins	18.56 ms [18.375 ms, 18.744 ms]	-123.598 µs (-0.7%)
iast	18.293 ms [18.112 ms, 18.474 ms]	-390.479 µs (-2.1%)
profiling	19.427 ms [19.232 ms, 19.622 ms]	743.653 µs (4.0%)
tracing	18.848 ms [18.659 ms, 19.038 ms]	165.242 µs (0.9%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.381 ms) : 1368, 1394
.   : milestone, 1381,
iast (3.651 ms) : 3596, 3705
.   : milestone, 3651,
iast_FULL (6.859 ms) : 6785, 6933
.   : milestone, 6859,
iast_GLOBAL (3.915 ms) : 3841, 3990
.   : milestone, 3915,
profiling (2.597 ms) : 2569, 2626
.   : milestone, 2597,
tracing (2.061 ms) : 2041, 2081
.   : milestone, 2061,
section candidate
no_agent (1.344 ms) : 1332, 1357
.   : milestone, 1344,
iast (3.451 ms) : 3411, 3490
.   : milestone, 3451,
iast_FULL (6.537 ms) : 6467, 6607
.   : milestone, 6537,
iast_GLOBAL (3.893 ms) : 3824, 3961
.   : milestone, 3893,
profiling (2.415 ms) : 2391, 2440
.   : milestone, 2415,
tracing (2.08 ms) : 2062, 2098
.   : milestone, 2080,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.381 ms [1.368 ms, 1.394 ms]	-
iast	3.651 ms [3.596 ms, 3.705 ms]	2.269 ms (164.3%)
iast_FULL	6.859 ms [6.785 ms, 6.933 ms]	5.478 ms (396.6%)
iast_GLOBAL	3.915 ms [3.841 ms, 3.99 ms]	2.534 ms (183.5%)
profiling	2.597 ms [2.569 ms, 2.626 ms]	1.216 ms (88.0%)
tracing	2.061 ms [2.041 ms, 2.081 ms]	679.474 µs (49.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.344 ms [1.332 ms, 1.357 ms]	-
iast	3.451 ms [3.411 ms, 3.49 ms]	2.106 ms (156.7%)
iast_FULL	6.537 ms [6.467 ms, 6.607 ms]	5.193 ms (386.3%)
iast_GLOBAL	3.893 ms [3.824 ms, 3.961 ms]	2.548 ms (189.6%)
profiling	2.415 ms [2.391 ms, 2.44 ms]	1.071 ms (79.7%)
tracing	2.08 ms [2.062 ms, 2.098 ms]	735.73 µs (54.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/remove-initialization-error
git_commit_date	1775145366	1775807614
git_commit_sha	`2365c12`	`3494f99`
release_version	1.61.0-SNAPSHOT~2365c1251f	1.61.0-SNAPSHOT~3494f99b04

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1775809577	1775809577
ci_job_id	1584314868	1584314868
ci_pipeline_id	107031680	107031680
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-3-gbij563v 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-3-gbij563v 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.969 s) : 14969000, 14969000
.   : milestone, 14969000,
appsec (14.569 s) : 14569000, 14569000
.   : milestone, 14569000,
iast (17.978 s) : 17978000, 17978000
.   : milestone, 17978000,
iast_GLOBAL (17.74 s) : 17740000, 17740000
.   : milestone, 17740000,
profiling (14.739 s) : 14739000, 14739000
.   : milestone, 14739000,
tracing (14.76 s) : 14760000, 14760000
.   : milestone, 14760000,
section candidate
no_agent (14.923 s) : 14923000, 14923000
.   : milestone, 14923000,
appsec (14.733 s) : 14733000, 14733000
.   : milestone, 14733000,
iast (18.033 s) : 18033000, 18033000
.   : milestone, 18033000,
iast_GLOBAL (18.013 s) : 18013000, 18013000
.   : milestone, 18013000,
profiling (14.921 s) : 14921000, 14921000
.   : milestone, 14921000,
tracing (14.858 s) : 14858000, 14858000
.   : milestone, 14858000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.969 s [14.969 s, 14.969 s]	-
appsec	14.569 s [14.569 s, 14.569 s]	-400.0 ms (-2.7%)
iast	17.978 s [17.978 s, 17.978 s]	3.009 s (20.1%)
iast_GLOBAL	17.74 s [17.74 s, 17.74 s]	2.771 s (18.5%)
profiling	14.739 s [14.739 s, 14.739 s]	-230.0 ms (-1.5%)
tracing	14.76 s [14.76 s, 14.76 s]	-209.0 ms (-1.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.923 s [14.923 s, 14.923 s]	-
appsec	14.733 s [14.733 s, 14.733 s]	-190.0 ms (-1.3%)
iast	18.033 s [18.033 s, 18.033 s]	3.11 s (20.8%)
iast_GLOBAL	18.013 s [18.013 s, 18.013 s]	3.09 s (20.7%)
profiling	14.921 s [14.921 s, 14.921 s]	-2.0 ms (-0.0%)
tracing	14.858 s [14.858 s, 14.858 s]	-65.0 ms (-0.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~3494f99b04, baseline=1.61.0-SNAPSHOT~2365c1251f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.493 ms) : 1481, 1505
.   : milestone, 1493,
appsec (2.559 ms) : 2504, 2614
.   : milestone, 2559,
iast (2.283 ms) : 2214, 2352
.   : milestone, 2283,
iast_GLOBAL (2.33 ms) : 2261, 2399
.   : milestone, 2330,
profiling (2.129 ms) : 2072, 2185
.   : milestone, 2129,
tracing (2.094 ms) : 2041, 2148
.   : milestone, 2094,
section candidate
no_agent (1.499 ms) : 1488, 1511
.   : milestone, 1499,
appsec (3.851 ms) : 3629, 4073
.   : milestone, 3851,
iast (2.287 ms) : 2218, 2356
.   : milestone, 2287,
iast_GLOBAL (2.33 ms) : 2261, 2400
.   : milestone, 2330,
profiling (2.137 ms) : 2080, 2194
.   : milestone, 2137,
tracing (2.094 ms) : 2040, 2147
.   : milestone, 2094,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.493 ms [1.481 ms, 1.505 ms]	-
appsec	2.559 ms [2.504 ms, 2.614 ms]	1.066 ms (71.4%)
iast	2.283 ms [2.214 ms, 2.352 ms]	790.137 µs (52.9%)
iast_GLOBAL	2.33 ms [2.261 ms, 2.399 ms]	837.151 µs (56.1%)
profiling	2.129 ms [2.072 ms, 2.185 ms]	636.078 µs (42.6%)
tracing	2.094 ms [2.041 ms, 2.148 ms]	601.574 µs (40.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.499 ms [1.488 ms, 1.511 ms]	-
appsec	3.851 ms [3.629 ms, 4.073 ms]	2.351 ms (156.8%)
iast	2.287 ms [2.218 ms, 2.356 ms]	787.881 µs (52.5%)
iast_GLOBAL	2.33 ms [2.261 ms, 2.4 ms]	830.957 µs (55.4%)
profiling	2.137 ms [2.08 ms, 2.194 ms]	637.724 µs (42.5%)
tracing	2.094 ms [2.04 ms, 2.147 ms]	594.276 µs (39.6%)

bric3 · 2026-04-09T10:21:18Z

.gitlab/TagInitializationErrors.java

+ *
+ * <p>Gradle generates synthetic "initializationError" testcases in JUnit reports for setup methods.
+ * When a setup is retried and eventually succeeds, multiple testcases are created, with only the
+ * last one passing. All intermediate attempts are marked skip so Test Optimization is not misled.


I wonder if you could provide in the javadoc a sample of the non modified junit test file, and the expected output.

Also noteworthy to know, since this code is running on Java 25 it's possible to use markdown javadoc: https://blog.jetbrains.com/idea/2025/04/markdown-in-java-docs-shut-up-and-take-my-comments/

.gitlab/TagInitializationErrors.java

bric3 · 2026-04-09T10:32:46Z

.gitlab/TagInitializationErrors.java

+    if (!modified) return;
+    var transformer = TransformerFactory.newInstance().newTransformer();
+    transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
+    transformer.transform(new DOMSource(doc), new StreamResult(xmlFile));


note: This modifies the file in-place. What happens if the app fails? Does it leaves invalid documents?

Good catch ! addressed in 3494f99 3494f99

bric3 · 2026-04-09T10:38:40Z

.gitlab/TagInitializationErrors.java

+      System.err.println("File not found: " + xmlFile);
+      System.exit(1);
+    }
+    var doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(xmlFile);


I think @PerfectSlayer refers to XML external entity, or other tricks: https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html#java.

Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com>

Removes Initialization error from junit XML

c42a128

cbeauchesne added the tag: ai generated Largely based on code generated by an AI or LLM label Mar 31, 2026

bric3 reviewed Mar 31, 2026

View reviewed changes

Address comments

1f3708d

cbeauchesne changed the title ~~Removes Initialization error from junit XML~~ Tag intermediate initializationError retries with test.final_status=skip in JUnit XML Mar 31, 2026

cbeauchesne added 3 commits April 1, 2026 19:18

Use java script

c044bba

apply the logic by classname

c1bbc59

Use JAVA_25_HOME

fc75f5f

cbeauchesne force-pushed the cbeauchesne/remove-initialization-error branch from eb73143 to fc75f5f Compare April 2, 2026 09:14

cbeauchesne added 4 commits April 2, 2026 14:39

Merge branch 'master' into cbeauchesne/remove-initialization-error

42fbf91

Merge branch 'master' into cbeauchesne/remove-initialization-error

492abbc

Do not tag if the property already exists

814275c

Merge branch 'master' into cbeauchesne/remove-initialization-error

4af16af

cbeauchesne marked this pull request as ready for review April 2, 2026 17:28

cbeauchesne requested a review from a team as a code owner April 2, 2026 17:28

cbeauchesne requested review from erikayasuda and removed request for a team April 2, 2026 17:28

cbeauchesne added type: enhancement Enhancements and improvements comp: tooling Build & Tooling labels Apr 3, 2026

bric3 changed the title ~~Tag intermediate initializationError retries with test.final_status=skip in JUnit XML~~ Tag intermediate initializationError retries with test.final_status=skip in JUnit XML Apr 3, 2026

PerfectSlayer reviewed Apr 7, 2026

View reviewed changes

Better comment

899ee14

bric3 reviewed Apr 9, 2026

View reviewed changes

cbeauchesne and others added 5 commits April 9, 2026 14:23

Update .gitlab/TagInitializationErrors.java

96b50d9

Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com>

Update .gitlab/TagInitializationErrors.java

e98c700

Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com>

Update .gitlab/TagInitializationErrors.java

d904031

Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com>

Add before/after in comment

1b5a33e

Write output to a temp file then atomically rename it

3494f99

Conversation

cbeauchesne commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

What Does This Do

Additional Notes

Contributor Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cbeauchesne commented Mar 31, 2026 •

edited

Loading

github-actions bot commented Apr 2, 2026 •

edited

Loading

pr-commenter bot commented Apr 7, 2026 •

edited

Loading