Blocked when creating a namespace using ndctl.

[sarailQAQ]

I am using CXLMemSim in QEMU to emulate CXL Type 3 memory. However, when I tried to create a namespace using ndctl, the command took a very long time to execute (over ten minutes) and never succeeded.

root@localhost:~# ndctl create-namespace -m dax -r region0
qemu-system-x86_64: info: CXL Type3: Initializing CXLMemSim integration
qemu-system-x86_64: info: CXL Type3: CXLMemSim TCP mode - 127.0.0.1:9999
qemu-system-x86_64: info: CXL Type3: Active latency injection ENABLED (min=0 ns)
qemu-system-x86_64: info: CXL Type3: Establishing connection during init...
qemu-system-x86_64: info: CXL Type3: Connection attempt #1 to CXLMemSim server at 127.0.0.1:9999
qemu-system-x86_64: info: CXL Type3: Successfully connected to CXLMemSim (fd=97)
qemu-system-x86_64: info: CXL Type3: Request #1 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #2 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #3 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #4 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #5 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #6 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #7 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #8 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #9 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Request #10 (op=READ, connected=1, transport=TCP)
qemu-system-x86_64: info: CXL Type3: Latency injection stats @ 100000 ops: injected=0 times, total_delay=0 us, avg=0.0 ns/injection
[  369.887920] INFO: task daxctl:627 blocked for more than 122 seconds.
[  369.943400]       Not tainted 6.8.0 #4
[  369.994925] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
qemu-system-x86_64: info: CXL Type3: Latency injection stats @ 200000 ops: injected=0 times, total_delay=0 us, avg=0.0 ns/injection
qemu-system-x86_64: info: CXL Type3: Latency injection stats @ 300000 ops: injected=0 times, total_delay=0 us, avg=0.0 ns/injection
[  492.767445] INFO: task daxctl:627 blocked for more than 245 seconds.
[  492.859890]       Not tainted 6.8.0 #4
[  492.904303] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  615.665098] INFO: task daxctl:627 blocked for more than 368 seconds.
[  615.891293]       Not tainted 6.8.0 #4
[  616.042594] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
qemu-system-x86_64: info: CXL Type3: Latency injection stats @ 400000 ops: injected=0 times, total_delay=0 us, avg=0.0 ns/injection

[vickiegpt]

Since you are using tcp mode, you need to make sure your NIC is powerful enough to transport the memory requests.

[sarailQAQ]

QEMU and cxlmemsim_server are running on the same server, with QEMU configured to use a TAP device. The CPU usage of cxlmemsim_server is constantly at 100% (single-threaded). Is there a better way to determine if the bottleneck lies in the network?

[vickiegpt]

QEMU and cxlmemsim_server are running on the same server, with QEMU configured to use a TAP device. The CPU usage of cxlmemsim_server is constantly at 100% (single-threaded). Is there a better way to determine if the bottleneck lies in the network?

Can you instead using shm mode?

[sarailQAQ]

An error when using shm mode：

cxl region: cmd_create_region: created 1 region
qemu-system-x86_64: info: CXL Type3: Initializing CXLMemSim integration
qemu-system-x86_64: info: CXL Type3: CXLMemSim SHM mode - /cxlmemsim_pgas
qemu-system-x86_64: info: CXL Type3: Active latency injection ENABLED (min=0 ns)
qemu-system-x86_64: info: CXL Type3: Establishing connection during init...
qemu-system-x86_64: CXL Type3: SHM invalid magic (got 0x43584c4d454d5348, expected 0x43584c53484d454d)
qemu-system-x86_64: CXL Type3: Connection failed during init (will retry on first memory access)