test: Add API test for partial pull with pullProgress flag by simek-m · Pull Request #28713 · containers/podman

simek-m · 2026-05-15T15:23:35Z

Progress reporting in container-libs/image did not update the progress channel for chunked layers.
This PR adds a test that builds a new image with chunked layers, pulls it using the REST API images/pull endpoint with the pullProgress flag introduced in Extend libpod pull API to show pull progress #28224 and verifies that the progress stream contains events for partial pulls too.

The fix for the container-libs bug needs to be merged first. The test currently fails as expected.
Depends-on: containers/container-libs#469

Log a correct container registry port in start_registry.

Current outcome of the test:

ok 336 [10-images] POST libpod/images/pull?reference=quay.io/libpod/alpine:latest&pullProgress=true [-d {}] : .status ('pulling') ~ .*success.*
ok 337 [10-images] POST libpod/images/pull?reference=quay.io/libpod/alpine:latest&pullProgress=true [-d {}] : .stream ('Trying to pull quay.io/libpod/alpine:latest...') ~ .*Artifact pulled successfully.*
ok 338 [10-images] pullProgress shows layer ID 161a397ac5ee9e631cd79cdfb5f0b36a861071d42e67afe98ed3cb69155ea5ff ('{"status":"pulling","stream":"Trying to pull quay.io/libpod/alpine:latest...
"}') ~ .*161a397ac5ee9e631cd79cdfb5f0b36a861071d42e67afe98ed3cb69155ea5ff.*
...+......+.....+...+....+.....+......+..........+.........+..+................+...+.....+....+...+++++++++++++++++++++++++++++++++++++++++++++*.......+.+..+....+......+.....+.........+.+...+.........+..+.+.....+.......+......+++++++++++++++++++++++++++++++++++++++++++++*..+.......+........+.......+...+..+.+......+...+..................+........+......+.........+.......+........+.+...+..+...............+.............+...+...+......+...+..+.....................+...+.......+........+.........+.......+...+.................................+..................+......+.........+..+.........+....+............+..............+..........+......+...+............+..+.......+...+.....+.+.....+....+...+........+.......+.........+.....+........................+......+................+.....+............+...............+.+...............+.....+.........+...+............+............+.+............+......+..+...........................+.......+...+...+...+++++
...........+........+++++++++++++++++++++++++++++++++++++++++++++*....+....+..+.......+++++++++++++++++++++++++++++++++++++++++++++*.....+...+..+.+..+............+....+..+...............+...+......+......+...+.........+.+..+....+...+........+.......+........+...+......+................+........+......+.+..+...+..........+........+..........+......+.........+..+............+..........+..+...............+....+.........+........+........................+...+..........+......+..+...+.........+.............+..+......+...............+.+........+...+...............+...+......+....+.................+....+........+..........+...+..................+..+....+......+.....+...+....+......+..+...+......+...............+...+.+............+..+..........+......+......+.........+......+.....+.+........+......+.+...............+..+.......+........+.+......+...+...+.....+.........................+............+..+...+......+..........+..+.........+.+...+...........+..........+.........+..............................+.....+.......+.........+...+..+....+..+.........+.+..+.....................+............+....+........+.............+........+....+..................+............+.....+.......+........+..........+...+..+.+..+....+....................+...............+...+.......+...........+..........+.....+....+.........+..+...+.+....................+...+.+............+........+.......+..+..........+...+..............+.+.................+.............+...+.........+...+..+.............+...+..+...............+.........+....+.....+....+..+....+..+............+..........+......+..+..................+.+..................+...+..............+.........+.....................+....+......+....................+...+.+..........................+.........+.+.........+.....+......+.+..+.+...........+......+...............+...+.......+.........+........+.......+.....+.+.....+....+...+...+...+.....+......+.............+.....+...............+.......+...+...+........+.+.....+..........+...+........+...+....+...+...+.....+......+.+............+............+........+..........+...+..............+..........+......+..+............+.+............+......+......+...+..+......+...+....+.................+....+..............+...............+.............+...+..+..........+.................+......+.........+...+...+......+...+.....................+......+...+......+....+......+..+.......+......+..+..................+....+...+.........+.....+...+.+......+.....+....+......+.............................................+..............+....+...+.........+.........+...............+..+..........+..+.......+.....+.+.....+...+......+....+.........+......+......+...+...............+..+.........+......+.........+................+..................+...............+.....+......+....+...+..................+............+..+............+...+......+.+.....+.........+................+..............+.......+.....+.......+..+++++
-----
# started registry (auth=none) on port 8081
ok 339 [10-images] POST libpod/images/pull?reference=localhost:5328/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : status=200
ok 340 [10-images] POST libpod/images/pull?reference=localhost:5328/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .status ('pulling') ~ .*success.*
ok 341 [10-images] POST libpod/images/pull?reference=localhost:5328/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .stream ('Trying to pull localhost:5328/chunked-test:latest...') ~ .*Starting to pull artifact.*
ok 342 [10-images] POST libpod/images/pull?reference=localhost:5328/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .stream ('Trying to pull localhost:5328/chunked-test:latest...') ~ .*Artifact pulled successfully.*
not ok 343 [10-images] pullProgress reports sha256:d684db2c15374e8c5797c138c5211bb8a0db426663c34042f14ff91519c4ba06 as progressComponentID
#  expected: ~ .*progressComponentID.*sha256:d684db2c15374e8c5797c138c5211bb8a0db426663c34042f14ff91519c4ba06.*
#    actual: {"status":"pulling","stream":"Trying to pull localhost:5328/chunked-test:latest...
"}
{"status":"pulling","stream":"Getting image source signatures
"}
{"status":"pulling","stream":"Copying blob sha256:d684db2c15374e8c5797c138c5211bb8a0db426663c34042f14ff91519c4ba06
"}
{"status":"pulling","stream":"Copying config sha256:a72eb130ce0f10876fdd2f367c24516ed3fd583f360a8da860b390232c3491b8
"}
{"status":"pulling","stream":"Starting to pull artifact","pullProgress":{"status":"pulling","total":542,"progressComponentID":"sha256:a72eb130ce0f10876fdd2f367c24516ed3fd583f360a8da860b390232c3491b8"}}
{"status":"pulling","stream":"Artifact pulled successfully","pullProgress":{"status":"success","current":542,"total":542,"progressComponentID":"sha256:a72eb130ce0f10876fdd2f367c24516ed3fd583f360a8da860b390232c3491b8"}}
{"status":"pulling","stream":"Writing manifest to image destination
"}
{"status":"success","images":["a72eb130ce0f10876fdd2f367c24516ed3fd583f360a8da860b390232c3491b8"],"id":"a72eb130ce0f10876fdd2f367c24516ed3fd583f360a8da860b390232c3491b8"}

Test with changes applied:

ok 334 [10-images] POST libpod/images/pull?reference=localhost:5000/idonotexist:latest&pullProgress=true [-d {}] : .response=500
ok 335 [10-images] POST libpod/images/pull?reference=quay.io/libpod/alpine:latest&pullProgress=true [-d {}] : status=200
ok 336 [10-images] POST libpod/images/pull?reference=quay.io/libpod/alpine:latest&pullProgress=true [-d {}] : .status ('pulling') ~ .*success.*
ok 337 [10-images] POST libpod/images/pull?reference=quay.io/libpod/alpine:latest&pullProgress=true [-d {}] : .stream ('Trying to pull quay.io/libpod/alpine:latest...') ~ .*Artifact pulled successfully.*
ok 338 [10-images] pullProgress shows layer ID 161a397ac5ee9e631cd79cdfb5f0b36a861071d42e67afe98ed3cb69155ea5ff ('{"status":"pulling","stream":"Trying to pull quay.io/libpod/alpine:latest...
"}') ~ .*161a397ac5ee9e631cd79cdfb5f0b36a861071d42e67afe98ed3cb69155ea5ff.*
....+.....+...+.+..+...+.+.........+.........+..+...+.+.....+......+.+............+...+++++++++++++++++++++++++++++++++++++++++++++*.+...+....+...+..+..........+........+...+....+...+...+...........+...+.+.....+.+++++++++++++++++++++++++++++++++++++++++++++*.+...+...+.+......+........+...............+.+...........+...+.+........+...+...+......+.........+.........+++++
.....+......+.....+...+.......+.....+.+.................+...............+.+..+..................+..........+.....+.......+...+...........+.........+..........+..+++++++++++++++++++++++++++++++++++++++++++++*............+...+..+....+...+..+.........+...............+......+.........+.+.....+.+..+...+.......+...........+....+...........+....+...+.....+............+.+..+..................+.+.........+...........+.+...........................+......+..+++++++++++++++++++++++++++++++++++++++++++++*.+.....+..........+++++
-----
# started registry (auth=none) on port 8081
ok 339 [10-images] POST libpod/images/pull?reference=localhost:5383/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : status=200
ok 340 [10-images] POST libpod/images/pull?reference=localhost:5383/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .status ('pulling') ~ .*success.*
ok 341 [10-images] POST libpod/images/pull?reference=localhost:5383/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .stream ('Trying to pull localhost:5383/chunked-test:latest...') ~ .*Starting to pull artifact.*
ok 342 [10-images] POST libpod/images/pull?reference=localhost:5383/chunked-test:latest&pullProgress=true&tlsVerify=false [-d {}] : .stream ('Trying to pull localhost:5383/chunked-test:latest...') ~ .*Artifact pulled successfully.*
ok 343 [10-images] pullProgress reports sha256:63ff4cd0ff8a821b4b8cfb03f5c6a2579d8000cc074fbb498226a7bbfcd0b5d1 as progressComponentID ('{"status":"pulling","stream":"Trying to pull localhost:5383/chunked-test:latest...
"}') ~ .*progressComponentID.*sha256:63ff4cd0ff8a821b4b8cfb03f5c6a2579d8000cc074fbb498226a7bbfcd0b5d1.*

Checklist

Ensure you have completed the following checklist for your pull request to be reviewed:

Certify you wrote the patch or otherwise have the right to pass it on as an open-source patch by signing all
commits. (git commit -s). (If needed, use git commit -s --amend). The author email must match
the sign-off email address. See CONTRIBUTING.md
for more information.
Referenced issues using Fixes: #00000 in commit message (if applicable)
Tests have been added/updated (or no tests are needed)
Documentation has been updated (or no documentation changes are needed)
All commits pass make validatepr (format/lint checks)
Release note entered in the section below (or None if no user-facing changes)

Does this PR introduce a user-facing change?

None.

mtrmac

Without vendoring the updated c/image this does not show that the new c/image passes…

… which is actually good now, because, the test is running here, against a not-yet-modified version of c/image, and it is passing; it probably shouldn’t.

mtrmac · 2026-05-15T16:06:08Z

 podman rmi -f quay.io/libpod/alpine:latest >/dev/null 2>&1 || true

+# Test pullProgress=true with zstd:chunked image from local registry
+start_registry


I see in the logs # started registry (auth=none) on port8081 vs.
POST libpod/images/pull?reference=localhost:5796

What is going on here? I can’t see any immediate explanation myself.

… the logging in start_registry is using a wrong variable.

Can you fix that as well, please? A separate commit would be nice.

Same in stop_registry.

simek-m · 2026-05-15T17:02:12Z

@mtrmac Yes, that was my intention to have the test failing here, but I'm surprised it actually passes. It definitely should not:(

I had it failing locally with the main version of ‘c/image‘ and passing with the bugfix version so it made me think it's correct.

I'll investigate what I did wrong, thanks.

Progress reporting in container-libs/image did not update the progress channel for chunked layers. Add test that builds a new image with chunked layers, pull it using the REST API and verify that the progress stream contains events for partial pulls too. Depends-on: containers/container-libs#469 Signed-off-by: Marek Simek <msimek@redhat.com>

simek-m · 2026-05-18T12:22:10Z

It worked locally because I had /etc/containers/storage.conf with the content of:

[storage.options.pull_options]
enable_partial_images = "true"

and for some weird reason assumed it's the default option (it's "false", of course).

The test passed on CI because it did not hit the code path at all (i.e. that's what the previous test exercises) 🤦‍♂️

Signed-off-by: Marek Simek <msimek@redhat.com>

mtrmac

Thanks! The explanation makes sense — let’s later see a passing version with a close-to-final implementation.

simek-m mentioned this pull request May 15, 2026

image/copy: Fix missing progress reporting for chunked layers containers/container-libs#848

Open

mtrmac requested changes May 15, 2026

View reviewed changes

simek-m force-pushed the test/api-chunked-pullprogress branch from 300c5dc to 54f93d1 Compare May 18, 2026 12:09

test/apiv2: Log correct port when starting registry

7819ab7

Signed-off-by: Marek Simek <msimek@redhat.com>

mtrmac self-requested a review May 18, 2026 18:10

mtrmac reviewed May 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Add API test for partial pull with pullProgress flag#28713

test: Add API test for partial pull with pullProgress flag#28713
simek-m wants to merge 2 commits into
containers:mainfrom
simek-m:test/api-chunked-pullprogress

simek-m commented May 15, 2026 •

edited

Loading

Uh oh!

mtrmac left a comment

Uh oh!

mtrmac May 15, 2026

Uh oh!

mtrmac May 15, 2026

Uh oh!

mtrmac May 18, 2026 •

edited

Loading

Uh oh!

simek-m commented May 15, 2026

Uh oh!

simek-m commented May 18, 2026

Uh oh!

mtrmac left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

simek-m commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current outcome of the test:

Test with changes applied:

Checklist

Does this PR introduce a user-facing change?

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

mtrmac May 15, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac May 15, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

simek-m commented May 15, 2026

Uh oh!

simek-m commented May 18, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

simek-m commented May 15, 2026 •

edited

Loading

mtrmac May 18, 2026 •

edited

Loading