Uncaught Exceptions breakpoint lifetime extends exception object

[maxbachmann]

Issue

When enabling "Uncaught Exceptions breakpoint" with debugpy this lifetime extends exceptions and in return their traceback until the next unwind event afterwards. This leads to behaviour changes under the debugger. The following minimal reproducer shows this behaviour change

import weakref

class Test:
    def test(self):
        raise Exception

  a = Test()
  b = weakref.ref(a)
  try:
      a.test()
  except:
      pass
  finally:
      a = None

  # None unless running under debugpy with "Uncaught Exceptions breakpoint" enabled
  print(b())

since Exception tracebacks contain a lot of references this can keep a lot of things alive for longer than expected.

This wasn't an issue prior to the sys.monitoring based implementation for Python 3.12.

Original Message

I am unsure what exactly changes since we didn't use debugpy in around a year. In the meantime we also switched from Python 3.11 to Python 3.13.
Our code base relies on objects being destroyed e.g. so signals are disconnected when debugging our code base.

When attempting to use debugpy again with our recent versions we experience crashes because the objects are kept alive via a frame object and so the signals aren't disconnected. Presumably this is caused by changes like the python 3.12 low impact monitoring.

Is it known what causes this + what would be the best way to avoid this?

[rchiodo]

Do you have some code that reproduces the issue? It's likely our eval that's adding a reference to something. 3.12 changed how everything worked.

[maxbachmann]

Not right now but I can try to write a smaller reproducer.

[maxbachmann]

signal.py
mySignal.cpp

I finally managed to make a reproducer for my issue I can share

The repo includes a signal.py to debug and a mySignal.cpp for a C++ signal that is required to trigger the issue.
To compile the C++ signal on Linux I did run

pip install pybind11
c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) mySignal.cpp -o mySignal$(python3-config --extension-suffix)

Running signal.py without debugpy on python3.11/3.13 just prints switch twice and then exits.
Running it under debugpy (python3.11) in vs code also just prints switch twice and then exits.
Running it under debugpy (python3.13) in vs code triggers the assert in line 72.

This occurs because the reference is kept alive -> the signal continues to call it when I already expect it to be dead.
I don't have any breakpoints or watched variables.

[rchiodo]

Thanks for the repro. Might be a while before somebody gets to this though.

I was going to try this. It should theoretically show where the references are coming from.

import objgraph
objgraph.show_backrefs(obj, max_depth=5)

[maxbachmann]

I did slighly extend using

def main():
    a = StateMachine(GameState)
    a.execute()
    import weakref

    state = weakref.ref(a.state)
    while not shutdown:
        nextFrame()
        b = state()
        if b is not None:
            import objgraph
            objgraph.show_backrefs([b], max_depth=5)
        b = None

    a.leave()

without debugpy this exports nothing. With debugpy it exports

which isn't particularly helpful. The refcount at this point is 3. It's worth noting that the lifetime is only extended. So after the next nextFrame call it is eventually dropped. However this is already too late since it leads to the signal getting called on the already "dead" object.

[rchiodo]

If it is eventually dropped, that sounds like a potential normal case? How can you be sure that references will immediately go to zero when you get rid of your reference?

I'd hazard a guess that it's something in the sys.monitoring callbacks that requires another function to be called until it gets rid of all of its references.

[maxbachmann]

There probably is no strict guarantee on anything since as far as I know Python generally has no lifetime guarantees.
It doesn't appear like something in the sys.monitoring system itself since I can just subscribe to all events without running into this:

TOOL_ID = 1
monitoring.use_tool_id(TOOL_ID, "demo-monitor")
EVENTS = 0
for name in dir(monitoring.events):
    if name.isupper() and name != "NO_EVENTS":
        event = getattr(monitoring.events, name)
        EVENTS |= event
        monitoring.register_callback(TOOL_ID, event, on_call)

monitoring.set_events(TOOL_ID, EVENTS)

I don't see why without any breakpoints/watch variables debugpy would have to continue holding references after the function returned. Especially since it didn't have to do so in the past.

Overall it just isn't a great experience when behaviour changes under a debugger.

[rchiodo]

Debuggers try to not impact the debuggee, but that's not always possible - especially with soft mode debuggers like debugpy is. For example, the debugger definitely causes other modules to load into the program.

I'd say references that go away after a longer period of time is likely an expectation of running a debugger.

But maybe in this case it's a simple fix. Like we clear out some variables after a callback.