Using disposable objects in Orchestration functions

[MarcinJuraszek]

Is there any guidance on how to correctly use objects that implement IDisposable in Orchestration functions?

Let's consider something like this:

using (IActivityScope scope = _activityScopeFactory.CreateScope())
{
    await context.CallActivityAsync("TestActivity1");
    await context.CallActivityAsync("TestActivity2");

    // (...) etc.
}

Unless we're doing something wrong, the current behavior we're observing is that the scope object will be created multiple times, every time the orchestration is re-run to be replayed when the various activity complete. However, the Dispose on said scope only ever gets called once, as the orchestration completes?

Is there a way to somehow get it to always run?

We're doing this for logging and instrumentation, so there's no risk of leaking expensive stuff, but it's still quite annoying because the data we gather with this flow is just not correct. We'd love to get any insight on how to do it the right way.

[MarcinJuraszek]

I think this one might be similar? Azure/azure-functions-durable-extension#3190

[cgillum]

Disposables in orchestrator code is an interesting scenario, for which we don't currently have any guidance.

However, logging in orchestrator code can definitely be problematic if the logging code is not safe for replay. Can you explain your end-to-end scenario a bit more?

[MarcinJuraszek]

There are a couple things we're trying to do:

• measure the overall execution duration for an orchestration instance
• associate the orchestrator execution logs with the correct TraceId coming from the callers (and later propagate it down to Activity Functions as well).

For both, we've created a class that implements IDisposable and internally uses Activity class and other similar things (e.g. various Metrics) to achieve the above. It uses the constructor as "Start" point, and Dispose as "End" point for all this. We use the same wrapper in non-orchestration flows (activity functions, and outside of Azure Functions and DTFx completely). So the wrappers know nothing about DTFx.

Things work just fine everywhere, except in Orchestrations, where we've noticed many more "Starts" than "Ends", which made me start investigating. That's how I observed that using statements and disposables in orchestrator code doesn't work as I would have expected it to work.

[cgillum]

Regarding your second bullet point, have you seen the distributed trace work that we recently did for DTFx? Samples are here: https://github.com/Azure/durabletask/tree/main/samples/DistributedTraceSample. This should cover propagating trace IDs from client, to orchestration, to activities, etc.

Regarding the point about many more starts than ends, one thing you could do is use context.IsReplaying and only create and dispose your trace activity when the value is false. I don't know if you'll be able to use the using syntax for this, but it's one workaround you can consider.

[MarcinJuraszek]

For distributed tracing - we already have a sink configured with Geneva. Should this be as simple as calling .AddSource("DurableTask.Core") in our setup? It looks really exciting and would let us simplify a lot of things.

I can certainly use context.IsReplaying to only log metrics and overall duration once. Just wanted to make sure that'sthe recommendation and there isn't anything simple we are missing that would make using and IDisposable work as expected.

[MarcinJuraszek]

Also, what's the version the tracing is part of, just so I can make sure we're at the right latest? We're on 3.1.0 and I don't see any traces from "DurableTask.Core" emitted after adding it as source.

We're using dotnet-isolated mode. Does that matter? Are those traces emitted by the host or the worker process?

[MarcinJuraszek]

There's one more use case:

• we are using log enrichment to add extra information like userId, tenantId, etc. to all logs for requests, orchestrations and activities. We use a try/catch/finally to make sure we set it at the beginning of the orchestration and clean in finally to not leak it out. Sounds like that's just not going to work?