Initial reimplementation of composefs-c#225
Open
cgwalters wants to merge 10 commits into
Open
Conversation
Collaborator
Author
|
There's definitely some sub-tasks to this and pieces that we need to break out. One that I'm realizing is that the dumpfile format is hardcoded to sha256-12. I guess we can just auto-detect from length (like we're doing in other places) but the more I think about this the more I feel we need to formalize it (as is argued in #224 ) So how about a magic comment in the dumpfile like or so? |
cgwalters
force-pushed
the
composefs-c-compat
branch
2 times, most recently
from
4d43b61 to
1871128
Compare
Collaborator
Author
|
Let's make the format layout a choice to avoid breaking sealed UKIs as is today |
cgwalters
force-pushed
the
composefs-c-compat
branch
3 times, most recently
from
8a5c48d to
9cb1923
Compare
This was referenced Mar 11, 2026
cgwalters
force-pushed
the
composefs-c-compat
branch
4 times, most recently
from
6eda766 to
dc1fed7
Compare
This was referenced Mar 17, 2026
cgwalters
force-pushed
the
composefs-c-compat
branch
from
dc1fed7 to
9a845fa
Compare
cgwalters
force-pushed
the
composefs-c-compat
branch
from
9a845fa to
9823c67
Compare
cgwalters
force-pushed
the
composefs-c-compat
branch
4 times, most recently
from
a8d6802 to
25cbbb1
Compare
8 tasks
cgwalters
force-pushed
the
composefs-c-compat
branch
2 times, most recently
from
895ccd1 to
8eeec80
Compare
Collaborator
Author
|
bootc-dev/bootc#1812 is related to this - we need to fix how we generate the EROFS. The problem in a nutshell is that https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9ed50b8231e37b1ae863f5dec8153b98d9f389b4 fixed a legitimate bug, but it also made the kernel parser less strict than it needed to be to fix the bug; we now accept symlinks that cross a block boundary, whereas before we didn't. |
cgwalters
force-pushed
the
composefs-c-compat
branch
2 times, most recently
from
ad51b62 to
4081268
Compare
cgwalters
force-pushed
the
composefs-c-compat
branch
4 times, most recently
from
6c48658 to
03b9a32
Compare
cgwalters
force-pushed
the
composefs-c-compat
branch
from
03b9a32 to
6234700
Compare
Collaborator
Author
|
OK, passing CI now, though re-reviewing I see a few more things to fix. |
cgwalters
force-pushed
the
composefs-c-compat
branch
6 times, most recently
from
43e33d1 to
123ea7a
Compare
Add set_write_concurrency() to Repository for overriding the default parallelism. Add read_filesystem_with_semaphore() as a public entry point that accepts an explicit Semaphore, and refactor the internal read_filesystem_impl() to centralize semaphore selection. Prep for wiring up --threads in mkcomposefs. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
The patch recipe referenced crates/cfsctl which was never a valid path; the crate has always been named composefs-ctl. Also relax the clean-tree check to allow untracked files (only committed changes need to match the pinned revision). Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
import_oci_layout() was opening the layout directory before calling ensure_writable(), so pulling into a read-only repo produced a misleading ENOENT error instead of a clear 'not writable' message. Move the write check to the top of the function, matching the existing skopeo pull path. Fixes privileged_pull_readonly_repo integration test. Signed-off-by: Colin Walters <walters@verbum.org>
For compatibility with the C composefs, we need to support writing directly to a flat XX/DIGEST path, without a leading `objects/`. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
The script hardcoded /usr/share/edk2/ovmf/OVMF_CODE.fd which is only present on Fedora. Probe a list of common paths (Ubuntu's ovmf package uses /usr/share/ovmf/OVMF.fd, Arch uses /usr/share/edk2/x64/OVMF.4m.fd) so the script works across distros without manual adjustment. Also add -machine q35, required on newer QEMU builds (e.g. RHEL10/CentOS Stream 10) where the default pc-i440fx machine type doesn't pair well with OVMF for EFI boot. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
The combined OVMF.qemuvars.fd with -bios hangs indefinitely on RHEL10/ CentOS Stream 10 QEMU (qemu-kvm 9.x). Use the split OVMF_CODE.fd + OVMF_VARS.fd files with -drive if=pflash and -machine q35 instead, which works correctly. Fall back to -bios with the combined image on distros that only ship the combined file (Ubuntu, Arch). Updated both testthing.py (which drives the example integration tests) and the fix-verity helper script (which runs the in-VM verity fixup pass). A temporary copy of OVMF_VARS.fd is made so UEFI can write to it without modifying the original system file. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
composefs-setup-root validates that the repo's meta.json has fs-verity enabled before trusting the repo. The dracut hook was only enabling verity on the content objects, so setup-root would see the repo as insecure and refuse to proceed. Switch the working directory to /sysroot/composefs (instead of the objects subdirectory) so we can enable verity on meta.json in addition to all the content objects. Also quote the loop variable and use the full relative path for clarity. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
The 30s default is tight on slower hardware (e.g. CentOS Stream 10 with OVMF pflash init overhead) — the VM boots successfully but just barely misses the window. 60s gives enough headroom while still being short enough to catch genuinely broken VMs. CI on Ubuntu with KVM acceleration boots well under 30s so the extra budget costs nothing. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
…info CLI Add support for generating V1 EROFS images compatible with the C composefs tools (mkcomposefs/composefs-info 1.0.8+). V1 uses compact inodes, BFS layout, and a simpler on-disk structure. Adds --erofs-version flag to cfsctl, new mkcomposefs and composefs-info compatibility subcommands, and RepositoryConfig for cleaner repo initialization. Note: this commit does not compile with --features oci (the default) until the following commit migrates OCI crate callers. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
Update the bootc reverse dependency CI to use the cgwalters/bootc adapt-composefs-rs-api branch which adapts bootc to the composefs-rs API changes (RepositoryConfig, FormatVersion, st_mtim_nsec). Assisted-by: OpenCode (Claude Sonnet 4)
cgwalters
force-pushed
the
composefs-c-compat
branch
from
123ea7a to
628f878
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Basically starting on composefs/composefs#423
3 key goals:
Assisted-by: OpenCode (Claude Sonnet 4)