io_uring_queue_init failed when using IORING_SETUP_SQPOLL, SQ_AFF, and COOP_TASKRUN with custom options

[TrKNguyen]

Hi, I'm encountering an issue when initializing Photon with io_uring options enabled.

Here is the log output:
/////
2025/05/31 10:15:34.596740|INFO |th=000058114BE9E970|iouring-wrapper.cpp:702|new_iouring_event_engine:Init event engine: iouring [is_master=1][setup_sqpoll=1][setup_sq_aff=1][sq_thread_cpu=2]
2025/05/31 10:15:34.596755|DEBUG|th=000058114BE9E970|reset_handle.cpp:27|ResetHandle:push [this=000058114BE89150]
2025/05/31 10:15:34.596767|INFO |th=000058114BE9E970|iouring-wrapper.cpp:557|check_register_file_support:iouring: register_files is enabled
2025/05/31 10:15:34.596791|INFO |th=000058114BE9E970|iouring-wrapper.cpp:123|init:io_uring_queue_init failed, removing IORING_SETUP_COOP_TASKRUN
/////
My code for initializing Photon looks like this:
////

photon::PhotonOptions opts;
opts.iouring_sq_thread_cpu = 1;
opts.iouring_sq_thread_idle_ms = 10;

uint64_t event_flags = INIT_EVENT_IOURING |
INIT_EVENT_IOURING_SQPOLL |
INIT_EVENT_IOURING_SQ_AFF;

if (photon::init(event_flags, INIT_IO_NONE, opts)) {
LOG_ERROR_RETURN(0, -1, "Photon init failed");
}
DEFER(photon::fini());

////

It looks like the kernel rejects IORING_SETUP_COOP_TASKRUN, and Photon silently disables it. I'm trying to understand:
Could someone please clarify:

1. Why io_uring_queue_init fails with IORING_SETUP_COOP_TASKRUN?

2. Should I explicitly disable IORING_SETUP_COOP_TASKRUN, or is this fallback expected behavior?

3. Are there known kernel limitations or conflicts between SQPOLL/SQ_AFF and COOP_TASKRUN?

Thank you so much for your help!

[beef9999]

Kernel version greater than 5.19 should have had support for IORING_SETUP_COOP_TASKRUN, it's beneficial to performance.

SQPOLL is not integrated in Photon yet.

[lihuiba]

@beef9999 We already have SQPOLL support. See #683.

[lihuiba]

2. Should I explicitly disable IORING_SETUP_COOP_TASKRUN, or is this fallback expected behavior?

Did you mean that you want to suppress the log output?

3. Are there known kernel limitations or conflicts between SQPOLL/SQ_AFF and COOP_TASKRUN?

As described in the code:

  if (p.flags & IORING_SETUP_COOP_TASKRUN) {
      // this seems to be conflicting with IORING_SETUP_SQPOLL,
      // at least in 6.4.12-1.el8.elrepo.x86_64
      p.flags &= ~IORING_SETUP_COOP_TASKRUN;
      LOG_INFO("io_uring_queue_init failed, removing IORING_SETUP_COOP_TASKRUN");
      goto retry;
  }

[TrKNguyen]

Hi again, and thanks for the previous clarification!

Since IORING_SETUP_COOP_TASKRUN and IORING_SETUP_SQPOLL are incompatible and Photon removes COOP_TASKRUN on failure, I’m wondering:

In a low-latency scenario (like HFT or real-time trading), is IORING_SETUP_SQPOLL with SQ_AFF considered strictly better than COOP_TASKRUN?

Are there benchmark results or internal insights comparing the performance impact of each under Photon?

For context, I’m currently running Linux kernel 6.5.0-41-generic, and I still see the conflict between COOP_TASKRUN and SQPOLL.

Given that, do you recommend a specific kernel version (or config) where both flags can safely coexist, or where Photon achieves optimal low-latency performance?

Thanks again for your support!

[IFOneDev]

Hello there:

Also same problem. Still working on it. Checked permissions and other recommendations found elsewhere. Still no progress.

[lihuiba]

is IORING_SETUP_SQPOLL with SQ_AFF considered strictly better than COOP_TASKRUN

Actually we don't have detailed comparison, but I have found an link that may be useful for you: https://atlarge-research.com/pdfs/2024-bingimarsson-msc_thesis.pdf

conflict between COOP_TASKRUN and SQPOLL.

See axboe/liburing#1075