ci: add Testing Farm test via tmt for Konflux CI by Roshan-R · Pull Request #4515 · coreos/coreos-assembler

Roshan-R · 2026-03-31T12:13:53Z

Add tmt test infrastructure to hook into Konflux CI via Testing Farm

The new test pulls the newly built coreos-assembler container image and verifies that it can successfully build a Fedora CoreOS image from scratch using a fedora-coreos-config checkout.

gemini-code-assist

Code Review

This pull request introduces a TMT-based testing framework for coreos-assembler, including Tekton integration for Testing Farm, TMT plans, and a test script for containerized builds. The reviewer identified several improvement opportunities: replacing hardcoded Git references in the Tekton configuration with dynamic parameters, refining the list of required packages in the TMT plan to include podman, and enhancing the test script's reliability by ensuring directory existence and applying proper variable quoting.

gemini-code-assist · 2026-03-31T12:15:07Z

+    value: https://github.com/Roshan-R/coreos-assembler
+  - name: GIT_REF
+    value: tmt-testing


The GIT_URL and GIT_REF parameters are currently hardcoded to a personal fork and a specific branch (Roshan-R/coreos-assembler and tmt-testing). For integration into the main repository, these should be updated to point to the official repository or use dynamic Tekton parameters to ensure the tests are executed against the correct source code.

gemini-code-assist · 2026-03-31T12:15:07Z

+      - wget
+      - jq
+      - git


The packages wget, jq, and git are being installed in the test environment but do not appear to be used by the host-side test.sh script. The git command used in the test is executed inside the coreos-assembler container. Conversely, the script relies on podman, which should be explicitly listed if it is not guaranteed to be present in the base image.

- podman

gemini-code-assist · 2026-03-31T12:15:07Z

+
+# IMAGE_URL is the coreos-assembler image that is built by konflux for each run.
+export COREOS_ASSEMBLER_CONTAINER="$IMAGE_URL"
+export COSA_DIR=$HOME/workspace/build


It is recommended to ensure that the $COSA_DIR directory exists before it is mounted as a volume. If the directory is missing, podman might create it with root ownership, which can lead to permission issues for the builder user inside the container. Additionally, quoting the path is a best practice to handle potential spaces.

Suggested change

export COSA_DIR=$HOME/workspace/build

export COSA_DIR="$HOME/workspace/build"

mkdir -p "$COSA_DIR"

gemini-code-assist · 2026-03-31T12:15:07Z

+    podman run --rm --security-opt=label=disable --privileged \
+    -v=${COSA_DIR}:/srv/ --device=/dev/kvm \
+    --device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa ${COREOS_ASSEMBLER_CONTAINER} "$@";


The volume mount and image variable should be quoted to prevent word splitting or issues with special characters. Also, the trailing semicolon at the end of the command is unnecessary in a bash script.

Suggested change

podman run --rm --security-opt=label=disable --privileged \

-v=${COSA_DIR}:/srv/ --device=/dev/kvm \

--device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa ${COREOS_ASSEMBLER_CONTAINER} "$@";

podman run --rm --security-opt=label=disable --privileged \

-v "${COSA_DIR}:/srv/" --device=/dev/kvm \

--device=/dev/fuse --tmpfs=/tmp -v /var/tmp:/var/tmp --name=cosa "${COREOS_ASSEMBLER_CONTAINER}" "$@"

Roshan-R · 2026-04-01T09:19:22Z

/retest

Roshan-R · 2026-04-08T14:02:05Z

/retest

joelcapitao · 2026-04-14T08:14:55Z

/retest

Roshan-R · 2026-04-17T09:17:14Z

/retest

Roshan-R · 2026-04-20T09:44:45Z

/retest

Roshan-R · 2026-04-22T11:37:55Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a testing framework using tmt for coreos-assembler, including a new test plan, a build test script, and updates to the Tekton pipeline for memory resource management. Feedback suggests adding the fetch flag to the build command to ensure necessary metadata is downloaded, increasing the allocated disk space to prevent out-of-space errors during image generation, and removing a fixed container name in the podman command to avoid potential naming conflicts.

gemini-code-assist · 2026-04-22T11:40:17Z

+    set -x
+    podman run --rm --security-opt=label=disable --privileged \
+    -v="${COSA_DIR}":/srv/ --device=/dev/kvm \
+    --device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa "${COREOS_ASSEMBLER_CONTAINER}" "$@";


Using a fixed name --name=cosa for the podman container can lead to conflicts if the test environment is reused or if multiple tests are run in parallel on the same host. Since the container is run with --rm, the name is not strictly necessary for identification. It's safer to omit it.

Suggested change

--device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa "${COREOS_ASSEMBLER_CONTAINER}" "$@";

--device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp "${COREOS_ASSEMBLER_CONTAINER}" "$@";

jbtrystram · 2026-04-28T15:12:18Z

+mkdir -p "$COSA_DIR"
+cosa init --force https://github.com/coreos/fedora-coreos-config --branch testing-devel
+# Test if the newly built container can build the fcos image
+cosa build


let's also run a set of test.
See

coreos-assembler/.cci.jenkinsfile

Line 42 in 211dee6

kola(cosaDir: "/srv", addExtTests: ["${env.WORKSPACE}/ci/run-kola-self-tests"])

Roshan-R · 2026-04-30T07:12:22Z

/retest

Roshan-R · 2026-05-08T09:19:26Z

/retest

…e memory for prepare-sboms Bump the pipeline-docker-build-multi-platform-oci-ta bundle to the latest hash. The prepare-sboms step in the build-images task requires more memory with our current workload and was hitting OOM. Increase memory requests and limits to 2Gi to ensure reliable execution.

Add tmt test infrastructure to hook into Konflux CI via Testing Farm The new test pulls the newly built coreos-assembler container image and verifies that it can successfully build a Fedora CoreOS image from scratch using a fedora-coreos-config checkout. It will also run kola tests by `cosa init` into fedora-coreos-config and running `kola run` and also the upgrade test using `kola runupgrade`

c4rt0

Just a comment based on today's Jenkins experience.

c4rt0 · 2026-05-15T17:39:04Z

+    TEST_CASE: test-qemu
+
+/test-kola-upgrade:
+  duration: 1h


Just wondering here if 1h here is sufficient. Honestly I have no idea if Konflux is any faster then Jenkins, but just today we witnessed CDN timeouts that caused a lot of failures.

The tests have been consistently passing in about 25–35 minutes in previous runs, which is why I set the timeout to 1 hour. I also haven’t seen failures caused by CDN timeouts before. Do you think it makes sense to increase it to 2 hours just to be safe?

With Dusty's recent workaround (merged only today) the upgrade test should be more... resilient. His change detect's the CDN stall and restarts zincati. That said, this process still takes time, bumping said value could be some form of a safety net. I really don't have strong opinions here - just something to consider.

gemini-code-assist Bot reviewed Mar 31, 2026

View reviewed changes

Roshan-R force-pushed the tmt-testing branch from f2ff904 to 51baff3 Compare April 22, 2026 11:33

Roshan-R force-pushed the tmt-testing branch from 51baff3 to 814ea11 Compare April 22, 2026 11:38

gemini-code-assist Bot reviewed Apr 22, 2026

View reviewed changes

jbtrystram requested changes Apr 28, 2026

View reviewed changes

Roshan-R force-pushed the tmt-testing branch 2 times, most recently from 51dbef0 to fa6e4a4 Compare May 5, 2026 09:02

Roshan-R force-pushed the tmt-testing branch from 8ff4a94 to f3aa1b7 Compare May 13, 2026 07:57

Roshan-R force-pushed the tmt-testing branch from f3aa1b7 to ccf9af6 Compare May 13, 2026 07:58

Roshan-R force-pushed the tmt-testing branch from 3836375 to 3c5bfc2 Compare May 14, 2026 00:14

c4rt0 reviewed May 15, 2026

View reviewed changes

Roshan-R force-pushed the tmt-testing branch 2 times, most recently from 816f027 to 3c5bfc2 Compare May 18, 2026 08:56

tmt: add kola-self-tests plan for running self tests

80a3d1e

	export COSA_DIR=$HOME/workspace/build
	export COSA_DIR="$HOME/workspace/build"
	mkdir -p "$COSA_DIR"

	--device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa "${COREOS_ASSEMBLER_CONTAINER}" "$@";
	--device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp "${COREOS_ASSEMBLER_CONTAINER}" "$@";

Conversation

Roshan-R commented Mar 31, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Roshan-R commented Apr 1, 2026

Uh oh!

Roshan-R commented Apr 8, 2026

Uh oh!

joelcapitao commented Apr 14, 2026

Uh oh!

Roshan-R commented Apr 17, 2026

Uh oh!

Roshan-R commented Apr 20, 2026

Uh oh!

Roshan-R commented Apr 22, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

jbtrystram Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Roshan-R commented Apr 30, 2026

Uh oh!

Roshan-R commented May 8, 2026

Uh oh!

c4rt0 left a comment

Choose a reason for hiding this comment

Uh oh!

c4rt0 May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Roshan-R May 18, 2026

Choose a reason for hiding this comment

Uh oh!

c4rt0 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants