IOCTL command numbers encode struct size, defeating ABI versioning

[JOOpdenhoevel]

Summary

The ABI versioning protocol (the leading __u32 size field in every IOCTL argument struct) is silently neutralised by the Linux _IOR/_IOW/_IOWR macro family. The two mechanisms are mutually contradictory, and any struct addition will break userspace compatibility rather than extend it gracefully.

Background

The kernel ABI reference documents a size-negotiation protocol: the caller sets size = sizeof(struct ...), the kernel copies min(user_size, kernel_size) bytes in each direction, and unknown trailing fields are zero-filled. This is intended to let the driver and library evolve their structs independently.

The Conflict

_IOW, _IOR, and _IOWR embed sizeof(struct ...) directly into the IOCTL command number at compile time:

#define _IOW(type, nr, size)  _IOC(_IOC_WRITE, (type), (nr), sizeof(size))

As a result, the command number is different for every distinct struct size. When the kernel receives an IOCTL, the VFS layer matches the command number exactly against registered handlers. If the caller compiled against an older (or newer) version of the struct than the kernel, the encoded sizes differ, the command numbers do not match, and the kernel returns -ENOTTY — before the handler even runs, let alone before the size field can be inspected.

Concretely: if slash_ioctl_bar_info gains one new __u32 field (going from 24 to 28 bytes), a library compiled against the old header will compute SLASH_CTLDEV_IOCTL_GET_BAR_INFO as a different integer than the kernel, and every call will fail with -ENOTTY. The size-negotiation logic is never reached.

This affects every IOCTL in the current ABI: SLASH_CTLDEV_IOCTL_GET_BAR_INFO, SLASH_CTLDEV_IOCTL_GET_BAR_FD, SLASH_CTLDEV_IOCTL_GET_DEVICE_INFO, all four QDMA ioctls, and the three hotplug device-request ioctls.

Consequence

The size field versioning protocol provides no actual protection. Any struct change is a hard ABI break, identical to having no versioning at all.