Overlay arguments on DM-Verity / with overlayfs results in userxattr: Invalid argument

[unforgivablesin]

/bin/bwrap \
  --unshare-pid \
  --unshare-uts \
  --unshare-cgroup \
  --unshare-user \
  --new-session \
  --ro-bind-try /etc/ssl/certs/ca-bundle.crt /etc/ssl/certs/ca-bundle.crt \
  --ro-bind-try /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt \
  --ro-bind-try /etc/resolv.conf /etc/resolv.conf \
  --ro-bind-try /etc/hosts /etc/hosts \
  --ro-bind-try /etc/ld.so.preload /etc/ld.so.preload \
  --ro-bind-try /etc/ld.so.conf /etc/ld.so.conf \
  --ro-bind-try /etc/ld.so.cache /etc/ld.so.cache \
  --ro-bind-try /etc/ld.so.conf.d /etc/ld.so.conf.d \
  --ro-bind-try /etc/fonts /etc/fonts \
  --ro-bind-try /etc/passwd /etc/passwd \
  --ro-bind-try /etc/locale.conf /etc/locale.conf \
  --ro-bind-try /lib64 /lib64 \
  --ro-bind-try /lib /lib \
  --ro-bind-try /proc /proc \
  --ro-bind-try /dev /dev \
  --dev /dev \
  --proc /proc \
  --bind-try /home/user/.var/app/signal-desktop/tmp /tmp \
  --tmpfs /run \
  --dir /run/user/1000 \
  --setenv WAYLAND_DISPLAY wayland-1 \
  --setenv XDG_SESSION_TYPE wayland \
  --ro-bind-try /run/user/1000/wayland-1 /run/user/1000/wayland-1 \
  --ro-bind-try /run/user/1000/pulse /run/user/1000/pulse \
  --ro-bind-try /run/user/1000/pipewire-0 /run/user/1000/pipewire-0 \
  --ro-bind-try /sys /sys \
  --ro-bind-try /run/user/1000/keyring /run/user/1000/keyring \
  --bind-try /home/user/.var/app/signal-desktop/home /home/user \
  --setenv GTK_THEME Adwaita:dark \
  --bind-try /home/user/.config/mimeapps.list /home/user/.config/mimeapps.list \
  --setenv XDG_DATA_DIRS /var/lib/pharaoh/export/:/usr/local/share/:/usr/share/ \
  --ro-bind-try /run/user/1000/xdg-dbus-proxy/org.Pharaoh.signal-desktop-4cqlVvmT.sock /run/user/1000/bus \
  --ro-bind-try /var/lib/dbus/machine-id /var/lib/dbus/machine-id \
  --ro-bind-try /etc/machine-id /etc/machine-id \
  --setenv DBUS_SESSION_BUS_ADDRESS unix:path=/run/user/1000/bus \
  --ro-bind-try /persist/pharaoh/app/signal-desktop /persist/pharaoh/app/signal-desktop \
  --overlay-src /usr/lib \
  --overlay-src /persist/pharaoh/app//signal-desktop//usr/lib \
  --ro-overlay /usr/lib \
  --overlay-src /usr/bin \
  --overlay-src /persist/pharaoh/app//signal-desktop//usr/bin \
  --ro-overlay /usr/bin \
  --overlay-src /usr/share \
  --overlay-src /persist/pharaoh/app//signal-desktop//usr/share \
  --ro-overlay /usr/share \
  --symlink /usr/bin /bin \
  /persist/pharaoh/app/signal-desktop/usr/bin/signal-desktop

Hello! So this bwrap command is part of my package manager and works on most systems, however I am integrating it into my OS which has a DM verity / and overlayfs on for example /usr/lib/systemd. Now I suspect the issue lies with the overlayfs since I have no errors with just /usr/share (theres no overlayfs there) But when I run this command I get:

bwrap: Can't make overlay mount on /newroot/usr/lib with options lowerdir=/oldroot/persist/pharaoh/app/signal-desktop/usr/lib:/oldroot/usr/lib,userxattr: Invalid argument

To not rule out an alternative, I will also provide the end goal: I want to merge /persist/pharaoh/app//signal-desktop//usr/lib and /usr/lib into a single /usr/lib on the bubblewrap sandbox. The specific issue is with these arguments:

  --overlay-src /usr/lib \
  --overlay-src /persist/pharaoh/app//signal-desktop//usr/lib \
  --ro-overlay /usr/lib \

Now when I manually create an overlay it does seem to work correctly.

mount -t overlay overlay   -o lowerdir=/usr/lib:/persist/pharaoh/app/signal-desktop/usr/lib,upperdir=/tmp/upper,workdir=/tmp/work   /tmp/merged

I don't think It's helpful but this is my partition setup:

NAME     MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
sr0       11:0    1  1.4G  0 rom   
vda      254:0    0   30G  0 disk  
├─vda1   254:1    0  500M  0 part  /boot
├─vda2   254:2    0   10G  0 part  
│ └─root 253:0    0   10G  1 crypt /  <------------ DM VERITY
├─vda3   254:3    0  500M  0 part  
│ └─root 253:0    0   10G  1 crypt /   <--------- DM VERITY HASH DEVICE
├─vda4   254:4    0 10.3G  0 part  /persist <-------- READ WRITE FOR OVERLAYFS
└─vda5   254:5    0  8.8G  0 part  /home <--------- READ WRITE FOR USER

Is there any way to make bwrap work with this setup? I know I can do some really hacky stuff with LD_LIBRARY_PATH but then I have to iterate over all of the files in /usr/lib and add them to the bwrap command, put them in /usr/lib/host and make another directory in /usr/lib/signal for example, and then use LD_LIBRARY_PATH to specify both these directories, plus their sub directories (its not recursive :P)

Any help is strongly appreciated!

[rusty-snake]

Now when I manually create an overlay it does seem to work correctly.

Did you tested with run0/sudo or in a unprivileged user-namespace?

bwrap: Can't make overlay mount on /newroot/usr/lib with options lowerdir=/oldroot/persist/pharaoh/app/signal-desktop/usr/lib:/oldroot/usr/lib,userxattr: Invalid argument

Check dmesg.

[unforgivablesin]

Did you tested with run0/sudo or in a unprivileged user-namespace?

unprivileged. I would also like to keep it that way

check dmesg

This is the output:

[   73.116409] overlayfs: failed to clone lowerpath

[rusty-snake]

Can you check #680.

[unforgivablesin]

Ah yes, okay. Is there some other way how I can get the desired outcome?

[rusty-snake]

• Implement the merge logic yourself using --bind and symlinks
• fuse-overlayfs, (fuse) unionfs, ...