UniversalBGTask crashes with STOWED_EXCEPTION

[tipa]

Describe the bug

I added support for background tasks as described in the documentation: https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/applifecycle/background-tasks

It works for most users, but I am seeing a lot of crash reports (in the Partner Center) for all my apps published in the Microsoft Store that are using background tasks.
With WinAppSDK 1.7, it was mostly ACCESS_VIOLATION errors and they have been discussed in this issue (which for some reason has been moved to the microsoft/microsoft-ui-xaml repository).

Now, after upgrading to WinAppSDK 1.8, I am still seeing crashes (seemingly the same amount - way over 10.000 crashes per day!), but with error code STOWED_EXCEPTION. Since the amount of crashes exceeds the "devices affected", it looks like the crash is happening repeatedly for the same machines.

Here are some stack traces as reported by the Microsoft Partner Center:

STOWED_EXCEPTION_80004005_Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll!_CallSettingFrame_LookupContinuationIndex

Frame  Image                                                                 Function
0      combase.dll                                                           RoOriginateLanguageException
1      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::hresult_error::originate
2      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     `winrt::to_hresult'::`1'::catch$23
3      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     _CallSettingFrame_LookupContinuationIndex
4      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     __FrameHandler4::CxxCallCatchBlock
5      ntdll.dll                                                             RcFrameConsolidation
6      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::to_hresult
7      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     `winrt::impl::root_implements_winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::factory_implementation::Task,winrt::Windows::Foundation::IActivationFactory_::NonDelegatingGetTrustLevel'::`1'::catch$0
8      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     _CallSettingFrame_LookupContinuationIndex
9      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     __FrameHandler4::CxxCallCatchBlock
10     ntdll.dll                                                             RcFrameConsolidation
11     Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::impl::produce_winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task,winrt::Windows::ApplicationModel::Background::IBackgroundTask_::Run
12     biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::RunInternal
13     biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::Run
14     twinapi.appcore.dll                                                   Windows::ApplicationModel::Core::BackgroundTaskWrapper::ThreadProc
15     ntdll.dll                                                             TppWorkpExecuteCallback
16     ntdll.dll                                                             TppWorkerThread
17     kernel32.dll                                                          BaseThreadInitThunk
18     ntdll.dll                                                             RtlUserThreadStart

STOWED_EXCEPTION_80004002_Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll!winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task::Run

Frame  Image                                                                 Function
0      combase.dll                                                           RoOriginateLanguageException
1      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::hresult_error::originate
2      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::hresult_no_interface::hresult_no_interface
3      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task::Run
4      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::impl::produce_winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task,winrt::Windows::ApplicationModel::Background::IBackgroundTask_::Run
5      biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::RunInternal
6      biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::Run
7      twinapi.appcore.dll                                                   Windows::ApplicationModel::Core::BackgroundTaskWrapper::ThreadProc
8      ntdll.dll                                                             TppWorkpExecuteCallback
9      ntdll.dll                                                             TppWorkerThread
10     kernel32.dll                                                          BaseThreadInitThunk
11     ntdll.dll                                                             RtlUserThreadStart

STOWED_EXCEPTION_80004005_Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll!Unknown

Frame  Image                                                                 Function
0      combase.dll                                                           RoOriginateLanguageException
1      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
2      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
3      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
4      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
5      ntdll.dll                                                             RcFrameConsolidation
6      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
7      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
8      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
9      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
10     ntdll.dll                                                             RcFrameConsolidation
11     Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     -
12     biwinrt.dll                                                           -
13     biwinrt.dll                                                           -
14     twinapi.appcore.dll                                                   Windows::ApplicationModel::Core::BackgroundTaskWrapper::ThreadProc
15     ntdll.dll                                                             TppWorkpExecuteCallback
16     ntdll.dll                                                             TppWorkerThread
17     kernel32.dll                                                          BaseThreadInitThunk
18     ntdll.dll                                                             RtlUserThreadStart

STOWED_EXCEPTION_800706ba_Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll!winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task::Run

Frame  Image                                                                 Function
0      combase.dll                                                           RoOriginateLanguageException
1      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::hresult_error::originate
2      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::hresult_error::hresult_error
3      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::throw_hresult
4      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task::Run
5      Microsoft.Windows.ApplicationModel.Background.UniversalBGTask.dll     winrt::impl::produce_winrt::Microsoft::Windows::ApplicationModel::Background::UniversalBGTask::implementation::Task,winrt::Windows::ApplicationModel::Background::IBackgroundTask_::Run
6      biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::RunInternal
7      biwinrt.dll                                                           Windows::ApplicationModel::Background::CBackgroundTaskInstance::Run
8      twinapi.appcore.dll                                                   Windows::ApplicationModel::Core::BackgroundTaskWrapper::Run
9      twinapi.appcore.dll                                                   Windows::ApplicationModel::Core::BackgroundTaskWrapper::ThreadProc
10     ntdll.dll                                                             TppWorkpExecuteCallback
11     ntdll.dll                                                             TppWorkerThread
12     kernel32.dll                                                          BaseThreadInitThunk
13     ntdll.dll                                                             RtlUserThreadStart

NuGet package version

Windows App SDK 1.8.1: 1.8.250916003

Packaging type

Packaged (MSIX)

Windows version

Windows 11 version 24H2 LTSC (26100, June Update)

IDE

Visual Studio 2022

Additional context

Crashes happen on various Windows versions, e.g. Windows 11 10.0.26100, Windows 11 10.0.22631, Windows 10 10.0.19045, ...

[godlytalias]

I had a look into these crash dumps, and the crash is happening as CoCreateInstance fails and throws exception. On further checking the etl files, below are some relevant ones I am seeing

Warning: PackagedCom entry for {B949185E-9ADC-4FD4-84B9-DAF90D64A3FB} in package 49297T.Partl.ClockOut_2.15.3.0_x64__jr9bq2af9farr (and possibly others) exists but is not visible to the caller. Lookup was performed without per-user state, which may indicate that the caller was elevated, non-user, or otherwise blocks per-user registrations (e.g. UWP and some OS processes).

Package 49297T.Partl.ClockOut_2.15.3.0_x64__jr9bq2af9farr is not singleton or system registered

Call security blocked activation call: CLSID:{B949185E-9ADC-4FD4-84B9-DAF90D64A3FB}, Client PID (image):6654 (C:\WINDOWS\system32\backgroundTaskHost.exe), Server PID (image): 0x60c4 (C:\Program Files\WindowsApps\49297T.Partl.ClockOut_2.15.3.0_x64__jr9bq2af9farr\WorkingHours.exe)

Can you once check whether the LaunchAndActivationPermissions are set appropriately as done in the sample here

[tipa]

Yes, LaunchAndActivationPermissions is set exactly like that.
I think the background task is working as intended for most users, and if LaunchAndActivationPermissions wasn't set properly, it wouldn't work for any?

[tipa]

@godlytalias please let me know if you need any additional information from me.
Is it possible that crash happens when the WinAppSDK Runtime hasn't been properly installed alongside WorkingHours? Or has been uninstalled forcefully later on? I've heard from instances where this happened.
For one of my apps I am now seeing almost 15.000 (!) crashes per day - and it's still an upwards trend

[godlytalias]

@tipa Its not related to WinAppSDK Runtime, I can see the UniversalBGTask operations being executed, but the above COM class IDs mentioned is from your app right? Errors say that the CoCreateInstance for these COM classes fail as it is not found in the package.

@DrusTheAxe Whether you have any clue on why these package related errors happen, #5870 (comment)

[tipa]

@godlytalias Yes, that is the GUID for my background task. It's certainly in the app, you can test it yourself :) As I said, from what I can tell the background task basically works for most users.
Isn't there a way to handle this error in your code so that it doesn't crash my app?

[godlytalias]

@tipa Ignoring CoCreateInstance failures would be problematic, In that case we may not be able to know of real issues where the tasks are not running. Right path here would be to identify the issue with the package here and get it fixed. Anyway user experience would be same in this case as well as if we simply ignore the failure. Crash would be for backgroundtaskhost.exe process, not the app process

[tipa]

@godlytalias I'd totally prefer if you could find the underlying issue, but from a developer perspective handling or not handling the error does make a difference. At the moment, my crash reporting in the Partner Center is totally flooded with those crashes and it makes it so much harder to identify the crashes I am actually able to fix. Additionally, getting so many crashes can have an impact on the Store ranking algorithm and potentially cause my app to become less visible in the Store.

Would this crash happen when not calling CoRegisterClassObject?
I've identified many cases in which my app throws an exception on startup (caused by the WinAppSDK - I've already reported them here). I am currently catching those errors and gracefully exit the app, without calling CoRegisterClassObject (I call that method in my Apps constructor).

[godlytalias]

If the COM server don't do CoRegisterClassObject, that would fail the CoCreateInstance for sure. But I am not sure whether that is the only cause for crashes here.
@DrusTheAxe may have some clues from package deployment perspective.

[tipa]

I made sure to call CoRegisterClassObject before anything else in my app, as first instruction in the Program.Main() method (I'm using a single-instanced app, if that's of relevance here) - without luck.

I have found yet another oddity with this background task implementation. When I omit RuntimeIdentifier in my csproj file, the background task just never fires, and I also don't see a crash. Why?

Going through the sample, I also noted this line in which ComServer.CoRevokeObject is called. Is this of importance? If yes, why and if no, why is it included in the sample?

Same question about this part. Is WindowsAppSDKBackgroundTask=true in the csproj really required? It seems to work without it in my apps. Why is it documented as being required?

In the meatime, my crash count keeps rising every day and since there doesn't seem to be any fix on the horizon, I will at some point have to consider to remove the entire background task from my app again.

[godlytalias]

If you omit RuntimeIdentifier, I guess it may not be deploying the universalbgtask.dll as well, you can see whether the dll is getting included in the package. Also about the point 'seeing a crash', these crashes in the spikes are from backgroundtaskhost.exe, which would not crash the 'main application process'. I think you may have to look into 'Diagnostic Data Viewer' -> Problem Reports tab / search for Microsoft.Windows.BrokerInfrastructure.* telemetry to see what is the status of the tasks (If that is not the way you're checking crash).

ComServer.CoRevokeObject does the cleanup operations when COM server shuts down, so that COM doesn't call into a dead server handle. This may be required when application closes. This can be a catch if not being done.

WindowsAppSDKBackgroundTask=true is not required for C# projects, as the binaries gets placed during publish operations otherwise as well. Its required for cppwinrt projects only.

If you're having so much crash count, there should be some way to repro the issue, if you can share a repro, it would be easy to figure out root cause (as mentioned above you should not be looking for the app process crashing, it would be backgroundtaskhost.exe which would be crashing). From the crash dumps what I can find out is that, the COM server looks for creating instance of the background task class registered, but that fails.

[tipa]

Also about the point 'seeing a crash', these crashes in the spikes are from backgroundtaskhost.exe, which would not crash the 'main application process'.

When I'm referring to a "crash" then I mean a "Crash" as reported by the Partner Center in the "Health" tab and these are a problem for me.

I think you may have to look into 'Diagnostic Data Viewer' -> Problem Reports tab / search for Microsoft.Windows.BrokerInfrastructure.* telemetry to see what is the status of the tasks (If that is not the way you're checking crash).

I don't think these crashes are coming from my machines, so I won't be able to gather contextual information in the Diagnostic Data Viewer. Searching for Microsoft.Windows.BrokerInfrastructure.* yields no results.

ComServer.CoRevokeObject does the cleanup operations when COM server shuts down, so that COM doesn't call into a dead server handle. This may be required when application closes. This can be a catch if not being done.

• If this is required (is it really?) why is it not mentioned in the documentation here?
• Looking at just that very documentation, it is noted that

  The background task class itself—and all other classes in the background task project—need to be public.
  

  but is this really the case? My BackgroundTask is internal and it works as expected - why? Or is this even the correct documentation for my case? It contains code about RegistrationServices that is nowhere to be found in the example app. It is really confusing
• I'm also using the CoRegisterClassObject approach to integrate into the Windows widget panel, and that component doesn't crash at all - at least I see no related crash reports in the Partner Center
• I can try to call CoRevokeObject and report back if that changes things, but I don't have much hope to be honest, given all my previous changes didn't improve the situation either. If using this method is really required, why is it not documented anywhere, only used in the example project? Additionally, from what I can tell basd on a few quick tests, the App deconstructor is never called.

WindowsAppSDKBackgroundTask=true is not required for C# projects, as the binaries gets placed during publish operations otherwise as well. Its required for cppwinrt projects only.

Would you agree that the sample should be fixed then? It currently is in the C# project sample with the note This tag is required for the usage of WinAppSDK Background Task API

If you're having so much crash count, there should be some way to repro the issue, if you can share a repro, it would be easy to figure out root cause (as mentioned above you should not be looking for the app process crashing, it would be backgroundtaskhost.exe which would be crashing). From the crash dumps what I can find out is that, the COM server looks for creating instance of the background task class registered, but that fails.

As I wrote before, I don't think the crashes originate from my machine, nor from machines of the majority of users. The Partner Center reports that the 15.000 crashes originate from just 460 devices (and I have much more daily active users). So that makes 32 crashes per devices on an average day. I am using a background task with a 15 minute TimeTrigger - that would perfectly match a regular 8 hour work day (4 crashes per hour)

What I can do is share a trimmed down example project that shows how I have implemented the background task. Would that help?

[godlytalias]

You're referring to the Platform SDK Documentation which have been there for quite some time. I don't have the context of all those points. However I feel its fair point that the background task class need to be public as it need to be invoked by an external component.
CoRevokeClassObject is required however, I can see that in the sample as well in the doc link you shared, More details can be seen in the CoRevokeClassObject docs as well, https://learn.microsoft.com/en-us/windows/win32/api/combaseapi/nf-combaseapi-corevokeclassobject (not CoRevokeObject, winappsdk sample to be fixed)
WindowsAppSDKBackgroundTask=true is to be fixed in C# case from winappsdk targets file itself, Currently it is working through C# publishing mechanism, hence even the apps that don't require background task ends up having those universalbgtask binaries. In upcoming versions this need to be fixed that only apps which have this flag will have those binaries packaged.
I would suggest to fix the revoke operation and once see whether crash count goes down