Struct DIM escape analysis driven deabstraction

[NinoFloris]

Default interface methods on structs currently require boxing for their this parameter. At the same time the JIT often has full type information. Either it sees a concrete struct type being boxed just beforehand, or it's a constrained generic call which allows for full specialization. Even then DIM calls heap allocate, introducing unexpected *per call* costs. These calls look cheap, either as constrained calls on a generic type parameter, or as the well known box undo pattern where a struct is cast to an interface and immediately used for a call. Users reasonably expect both paths to be allocation free for known struct types.

The status quo is unfortunate because most DIMs are trivial: they return constants, throw exceptions, or delegate to other methods on the same interface. In practice, this almost never escapes, which likely holds for the vast majority of DIM implementations out there.

As far back as 2019 I opened an issue (dotnet/roslyn#35008) requesting a C#-recognized attribute to warn implementers of DIMs that should be overridden. This would help us, normal library authors, do better interface evolution. Last year I suggested that for structs implementing DIM containing interfaces, partially doing so might warrant at least a compiler notice, as this can lead to very unexpected allocations. Neither proposal has received substantive replies.

JIT optimization of DIM calls on structs could significantly reduce the cases where this particular boxing cliff shows up. A DIM can be deabstracted and synthesized as an actual interface implementation: when this does not meaningfully escape and is not used as a metadata target.

This gives us a conservative initial rule:

• this is never boxed.
• this never escapes except where it is used as the instance value for a call where:
  • The call target is declared on the exact interface instantiation being specialized.
  • The call target is implemented by the struct (i.e., not a DIM).

A more broadly useful, but more complex, extended rule allows valid call targets to be other DIMs, which must themselves fully conform to the stated rules.

This approach enables a DIM to call any other methods on its interface (which may themselves be DIMs) without boxing, while avoiding calls to object methods or needing to trace complex interface member resolution.

To get there the JIT could synthesize a method implementation for the struct. In practice I assume true method synthesis is hindered by the runtime <-> JIT boundary. Limiting the optimization to inlining, where the DIM would need to be a profitable candidate, isn't so bad: they generally will be good candidates (and may justify a decent bonus factor). To satisfy the extended rule profitably, any contained DIM calls *must* be inlined as well, otherwise we may end up turning one box into multiple.

For the call setup a stack copy has identical mutation semantics to a boxed struct, and since this does not escape and is never used as a metadata target, heap identity is not required.

Mechanically this gives the JIT a reasonably straightforward transformation:

• Call setup: replace the box with a stack copy of the struct.
• DIM body transformation:
  • Replace all ldarg.0 with ldloca.s 0 (ref to the stack copy).
  • Replace all implemented interface method calls on this (verified to target the interface instantiation only) with direct calls on the struct, passing ref this.
  • Recursively rewrite and inline any calls to other DIMs (again, verified to be conformant).

DIM escape analysis might be able to use the same infrastructure the JIT has recently built up for stack allocation of reference types. Conservative by default, falling back to boxing when it can't be certain.

[tannergooding]

The boxing here is likely not the expensive part, it's rather going to be the copy of the struct. The copy must be preserved as it is otherwise an observable change in behavior for when the DIM mutates the underlying value. Avoiding either the box or the copy requires inlining and other analysis to prove that it is safe and non side-effecting.

With that all being said, we also expect most instances have overridden the implementation because the DIM does allocate, does copy, is likely a suboptimal implementation, etc. So is it really worth the explicit effort to target and optimize this scenario in the JIT when it is expected to be uncommon and any of the normal escape analysis will already be covering the trivial cases?

-- DIMs aren't really there for convenience or perf; they are there so you can extend an interface without it being a binary breaking change and its expected that you override the implementation with something more optimal.

[NinoFloris]

The boxing here is likely not the expensive part, it's rather going to be the copy of the struct.

The copy may be necessary to preserve semantics, but it’s not clear to me that the allocation cost is negligible for average sized structs, let alone specialization wrappers. The allocation is strictly additive to the copy, carrying real end to end costs: GC pressure, generation promotions, or even serious additional latency if the allocation budget is exceeded or the region is full. By contrast, a copy does none of this, and for specialization wrappers the copy really is as good as a no-op.

Avoiding either the box or the copy requires inlining and other analysis to prove that it is safe and non side-effecting.

Yes, as laid out in the proposal. I did not dive into how to avoid the copy itself, as that could just rely on more general mechanisms not unique to this DIM optimization.

Note that the JIT already handles closely related patterns, which look syntactically identical:

((IFace)s).Method();       // optimized, avoids heap allocation
((IFace)s).DefaultMethod(); // currently always allocates

If Method is an explicit interface implementation, using this pattern is even the only way to make this call (without using a constrained call helper), and it was still optimized at one point. For most users looking at this snippet, the DIM case is reasonably expected to behave similarly to the regular interface call. People would either expect them both to box, or neither.

Granted, as we know, enabling the DIM case to behave like this is more complex, but that complexity does not demerit that these expectations are reasonable.

With that all being said, we also expect most instances have overridden the implementation

There is currently zero tooling that helps people actually override DIMs systematically: no compiler warning, analyzer, or diagnostic. I mentioned this at the start of the issue for this very reason. Currently, when a DIM is added in a release and its existing callers begin using it immediately, implementers receive zero hints about this, unless they happened to read the release notes or inspected the interface again. Saying "we expect most instances have overridden the implementation" without providing any mechanisms to guide developers is optimistic at best.

Until (if) we get opt-in reference type specialization, people will use interface constraints and structs if they need specialization, and in that context, DIMs are the only mechanism to provide defaults.

Other, protocol or trait based, ecosystems enable exactly this, even though they generally come with a self type and/or friendly syntax for 'instance' methods over static receiver methods. One would have to argue against the merits of those ecosystems before dismissing the value of such an approach in .NET where we have a syntactically similar feature that has no real alternatives (snaking ref TSelf into a static getter method that would otherwise be an instance property is not a reasonable alternative).

The BCL may not do this kind of specialization-dependent type design, but it would likely use it quite a bit if opt-in reference type specialization allowed such designs to revolve around abstract classes. That would be a great alternative, but we're much further from that point than starting to do just the slightest bit of DIM optimization. The capability to specialize is just too useful in data-centric code where composition is necessary to manage complexity but virtual call overhead is unacceptable.

To be clear, this issue is describing what an initial optimization could look like, not arguing it must be done now. I wanted the design on record because with the recent investments into escape analysis it will become increasingly cheaper to implement.

[tannergooding]

but it’s not clear to me that the allocation cost is negligible for average sized structs

The GC is designed so that allocations are highly efficient, in most cases it is cheaper than even malloc in native. While long lived allocations can have some increased impact, particularly when freeing the data, the actual cost for typical user code ends up being the copy instead.

which look syntactically identical

Something appearing similar or even identical doesn't really matter; the implementation details of the two methods are what become impactful and a DIM has a fundamentally different signature and implementation, which is where its cost and overhead comes from.

The box is elidable for the former case because the underlying method it binds against is taking a ref MyStruct this parameter. It is not elidable for the latter because it is taking a IFace this parameter. This means that the former can be optimized with or without inlining, by simply ensuring that a relevant copy of the data occurs. The latter, however, can only be optimized with inlining and may require significant optimization on top of that based on what the method body was doing.

People would either expect them both to box, or neither.

This is an incorrect expectation because of the differences in implementation that were touched on above. There is a lot of nuance when it comes to compilers, optimizations, and the general patterns that various code emits. One must take each scenario independently with the associated context to determine what happens, it is not simply enough to just look at how two patterns appear similar in high level code.

There is currently zero tooling that helps people actually override DIMs systematically: no compiler warning, analyzer, or diagnostic

This is arguably the "real" problem and what needs addressing.

The underlying issue here isn't that DIMs allocate or box and adding an optimization to avoid those doesn't actually fix the issue, at least not properly. It simply masks the problem a little bit more, but still leaves the largest percentage of perf on the table, particularly since the DIM is likely suboptimal for derived types anyways and is simply trying to polyfill in a way that avoids it being a breaking change.

DIMs are the only mechanism to provide defaults.

This is really a misunderstanding of the DIMs feature IMO. DIMs are not really a convenience feature like that and aren't there in the same way that abstract classes use virtual methods to provide a default implementation. They are rather a binary compatibility feature which come with large caveats for value types and a few other scenarios, they exist as a fallback for the case where a user hasn't provided the actual implementation yet.

The expectations we have for how users have written code come from having a fairly good understanding of how users do write code on average. That the majority of code is reference types, that value types tend to be specialized and for perf oriented scenarios where users are likely more mindful about keeping up to date with such changes, etc.

[aromaa]

Theres also proposal in dotnet/csharplang#9969

[NinoFloris]

Let's get concrete about allocation costs, here's a benchmark. The write barrier alone as part of copying the struct into the box will cost around what the full stack copy for a typical 16 byte struct will be. And a microbenchmark can't even capture the knock-on effects of GC pressure. It's hard to characterize a 4x difference as negligible, or more importantly, that the copy is the expensive part.

Method	Mean	Error	StdDev	Ratio	Gen0	Allocated	Alloc Ratio
Box	5.432 ns	0.0402 ns	0.0376 ns	1.00	0.0051	32 B	1.00
Copy	1.469 ns	0.0111 ns	0.0092 ns	0.27	-	-	0.00

using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<BoxVsCopyBenchmark>();

[MemoryDiagnoser]
public class BoxVsCopyBenchmark
{
    readonly ValueTask<int> _source = new(42); // 16 bytes

    [Benchmark(Baseline = true)]
    [MethodImpl(MethodImplOptions.NoInlining)]
    public object Box() => _source;

    [Benchmark]
    [MethodImpl(MethodImplOptions.NoInlining)]
    public ValueTask<int> Copy() => _source;
}

Something appearing similar or even identical doesn't really matter; the implementation details of the two methods are what become impactful and a DIM has a fundamentally different signature and implementation, which is where its cost and overhead comes from.

The box is elidable for the former case because the underlying method it binds against is taking a ref MyStruct this parameter. It is not elidable for the latter because it is taking a IFace this parameter. This means that the former can be optimized with or without inlining, by simply ensuring that a relevant copy of the data occurs. The latter, however, can only be optimized with inlining and may require significant optimization on top of that based on what the method body was doing.

I addressed all of this in the proposal, you're responding as if I didn't.

This is really a misunderstanding of the DIMs feature IMO.

This is getting off topic from the proposal, but since you raised it: looking at the history, the original proposal (dotnet/csharplang#288) is titled 'A Tour of Default Interface Methods for C# (traits)' and explicitly states that the feature 'provides the elements of the traits language feature.' The proposal (https://github.com/dotnet/csharplang/blob/main/proposals/csharp-8.0/default-interface-methods.md#motivation) lists binary compatibility, traits, and interop with Java and Swift as motivations. Reducing the feature to just one of those isn't a historically correct assessment.

Additionally, the LDM did discuss a warning for structs not implementing default methods during the design phase and deferred it as 'more suited for an analyzer.' This was seven years ago. No analyzer exists today.

Features get refined and repurposed as platforms evolve all the time, and the technology adapts to how people actually use things rather than insisting it must only be used as originally designed. Without alternative mechanisms for specialization, interface constrained type parameters over struct implementations is how you do it. And once you're there, DIMs are the only mechanism for providing defaults.

The expectations we have for how users have written code come from having a fairly good understanding of how users do write code on average

The argument that "value type users tend to be specialized and mindful" is circular. You argue based on an understanding that's internal to the team, from which you conclude nobody needs it because nobody does it. While in fact, library authors can't ship specialization-based designs in good faith today. It used to be because we could not evolve interfaces, which DIMs set out to solve, but now we can't because if we ever need to add a DIM the boxing cost makes them untenable, while implementers receive no indication they should override them either.

Tooling and optimization should be complementary here: one helps developers know when to override, the other makes the default path not silently costly.

Previously I steelmanned one alternative: opt-in reference type specialization with abstract classes, noting we're much further from that than from basic DIM optimization. Shipping self types (e.g. IFace<self TSelf>) is another far away alternative. Notably, it could include a way for authors to use e.g. self.Call() instead of this calls, giving the JIT a source compatible signal that those calls can be specialized. It would only need to verify during importation that this isn't used to mark the DIM as deabstractable.

Every language that supports trait-like composition either avoids these issues by design (C++ via template substitution, Rust and Swift via generic specialization), or has DIMs without value types, avoiding the problem (Java, Kotlin). All these languages have many users with varying levels of experience. In .NET we do have both value types and trait-like patterns, but we keep penalizing the combination without offering a viable alternative.

[tannergooding]

Let's get concrete about allocation costs, here's a benchmark. The write barrier alone as part of copying the struct into the box will cost around what the full stack copy for a typical 16 byte struct will be. And a microbenchmark can't even capture the knock-on effects of GC pressure. It's hard to characterize a 4x difference as negligible, or more importantly, that the copy is the expensive part.

This is not representative of costs for real code. It's trying to isolate a specific metric and then ignoring all the downstream things that also come into play, such as how it impacts inlining, how it impacts other optimizations, that other examples are going to be doing real work and not simply returning a value, etc.

The real cost is not some 4x and the real cost it going to differ quite significantly for other reasons as well.

But this is really missing the point. The real issue at the heart of what you're raising isn't a lack of JIT optimization here, it's that users don't have a way to get notified that they failed to implement a DIM on a value type and so will end up with potential pessimizations. The fix is to provide such an analyzer, which anyone can do (it does not have to be the core team), and which will be far more impactful. -- There are an unlimited number of analyzers that could be written and feature work to be done, the .NET team has to focus on what is most impactful across the entire product and ecosystem, which means some like the one raised here may not be prioritized.