Fix segmentation fault on Python 3.13

[shadchin]

Fix for #910 and #868

The problem is only with Python 3.13, in Python 3.14 PyObject_GC_New does this initialization itself, see python/cpython#123192

[provinzkraut]

Seems like a reasonable fix to me.

I'm curious though how you tracked it down to this?

Also, can you re-enable the regular tests for 3.13 again here?

msgspec/.github/workflows/ci.yml

Lines 63 to 64 in 1bc64dd

	# TODO: Enable this once fixed https://github.com/jcrist/msgspec/issues/910
	# - "3.13"

[shadchin]

I'm curious though how you tracked it down to this?

It was difficult :) The short version:

From gdb bt found following calls:

4  make_dict_from_instance_attributes
#5  _PyObject_MaterializeManagedDict_LockHeld
#6  _PyObject_MaterializeManagedDict
#7  ensure_managed_dict

In next step, I found that values was not initialized:

(gdb) frame 4
#4  0x00000000023d7c59 in make_dict_from_instance_attributes (interp=0x2fcc2e8 <_PyRuntime+104400>, keys=0x7ffff4e24f20, values=0x7ffff54dc070) at /home/shadchin/arc/arcadia/contrib/tools/python3/Objects/dictobject.c:6736
6736                track += _PyObject_GC_MAY_BE_TRACKED(val);
(gdb) info locals
val = <unknown at remote 0xcdcdcdcdcdcdcdcd>
i = 0
used = 1
track = 0
size = 30
res = 0x7ffff1b535b0
(gdb) print values
$1 = (PyDictValues *) 0x7ffff54dc070
(gdb) print keys
$2 = (PyDictKeysObject *) 0x7ffff4e24f20
(gdb) x/10gx values
0x7ffff54dc070: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc080: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc090: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc0a0: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc0b0: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd

Then I had to study what managed dict/PyDictValues and Struct_alloc. I found that it's true for 3.13 that we don't initialize PyDictValues, looked at the CPython code, how it does it, found the _PyObject_InitInlineValues call.

[ofek]

Do we know why 3.14 is fine?

[provinzkraut]

@shadchin seems like _PyObject_InitInlineValues isn't available?

https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228

[shadchin]

Do we know why 3.14 is fine?

Python 3.14+ does this initialization itself - https://github.com/python/cpython/blob/main/Python/gc.c#L2377-L2379

[provinzkraut]

Maybe it's worth asking upstream about how to deal with this without relying on private APIs?

Not like we don't rely on private APIs elsewhere, so I wouldn't consider it a blocker, but if there's a way, that might be nice?

[shadchin]

@shadchin seems like _PyObject_InitInlineValues isn't available?

https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228

We have our own build system, fix is working for us, but I forgot to check on a clean environment, sorry. I tried to make a fix but it didn't help. There's a conflict with write_u64.

I'll go upstream with it.