[android][clr] No peer certificates when executing System.Net.Http.Functional.Tests on Android emulator

[matouskozak]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1297669
Build error leg or test failing: System.Net.Http.Functional.Tests
Runtime changes: a92d749...e02e618 (TBD - confirm)

                 02-17 19:11:51.890 10934 25477 I DOTNET  : 	[IGNORED] System.Net.Http.Functional.Tests.SocketsHttpHandler_DiagnosticsTest_Http11.SendAsync_OperationCanceledException_RecordsActivitiesWithCorrectErrorInfo
                 02-17 19:11:51.891 10934 25477 I DOTNET  : System.Net.Http.Functional.Tests.SocketsHttpHandler_DiagnosticsTest_Http11 1.4314913 ms
                 02-17 19:11:51.891 10934 25477 I DOTNET  : 
                 02-17 19:11:51.891 10934 25477 I DOTNET  : Test collection for System.Net.Http.Functional.Tests.SocksProxyTest_Http1_Async
                 02-17 19:11:51.893 10934 25477 D DOTNET  : GetJNIEnv: Registering JNI thread detach. env ptr 0x7dcd5de753c0. Key: -2147483632
                 02-17 19:11:53.125 10934 25477 D DOTNET  : DetachThreadFromJNI: Detaching thread from JNI
                 02-17 19:11:54.064 10934 10945 I unctional.Test: Background young concurrent copying GC freed 129004(5772KB) AllocSpace objects, 0(0B) LOS objects, 24% free, 14MB/18MB, paused 381.045ms total 425.702ms
                 02-17 19:11:54.900 10934 25377 D DOTNET  : GetJNIEnv: Registering JNI thread detach. env ptr 0x7dcd5de75500. Key: -2147483632
                 02-17 19:11:54.900 10934 25377 W System.err: javax.net.ssl.SSLPeerUnverifiedException: No peer certificates
                 02-17 19:11:54.901 10934 25377 W System.err: 	at com.android.org.conscrypt.ActiveSession.checkPeerCertificatesPresent(ActiveSession.java:314)
                 02-17 19:11:54.901 10934 25377 W System.err: 	at com.android.org.conscrypt.ActiveSession.getPeerCertificates(ActiveSession.java:180)
                 02-17 19:11:54.901 10934 25377 W System.err: 	at com.android.org.conscrypt.ExternalSession.getPeerCertificates(ExternalSession.java:106)
                 02-17 19:11:54.901 10934 25377 W System.err: 	at com.android.org.conscrypt.Java7ExtendedSSLSession.getPeerCertificates(Java7ExtendedSSLSession.java:129)
                 02-17 19:11:54.901 10934 25377 W System.err: 	at com.android.org.conscrypt.Java7ExtendedSSLSession.getPeerCertificates(Java7ExtendedSSLSession.java:31)
                 02-17 19:11:55.010 10934 25485 I DOTNET  : 	[PASS] System.Net.Http.Functional.Tests.SocksProxyTest_Http1_Async.TestLoopbackAsync

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": [
      "System.Net.Http.Functional.Tests",
      "System.err: javax.net.ssl.SSLPeerUnverifiedException: No peer certificates"
  ],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1297669
Error message validated: [System.Net.Http.Functional.Tests System.err: javax.net.ssl.SSLPeerUnverifiedException: No peer certificates]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/17/2026 8:00:42 PM UTC

Report

Build	Definition	Test	Pull Request
1308845	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124842
1308825	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124731
1308820	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124843
1308792	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124833
1308779	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124701
1308718	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124778
1308614	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124838
1308592	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124837
1308167	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124810
1308204	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124819
1308580	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124836
1308519	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124834
1308505	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124833
1308461	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124817
1308071	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124738
1308498	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#121906
1308476	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124832
1308412	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124830
1308467	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124831
1308424	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124760
1308450	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123830
1308435	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124818
1308404	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124791
1308290	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124736
1308179	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124813
1308300	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124822
1308213	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124821
1308220	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124820
1308080	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.LargeUriAndHeaders_Works	#124782
1308149	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#120275
1308138	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124805
1308132	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124765
1308115	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution
1307983	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124203
1308022	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124300
1307890	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#122649
1308009	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124809
1307919	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124804
1297561	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124519
1307873	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124806
1307813	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124801
1307837	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124650
1306322	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124684
1307754	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124798
1307676	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124753
1307694	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124516
1307684	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124794
1307597	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124796
1307590	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123323
1307570	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124764
1307543	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124649
1307524	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124566
1307536	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124791
1306121	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124567
1307513	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124792
1307261	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124589
1307507	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124590
1307493	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124273
1307470	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124593
1307463	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123251
1307360	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124789
1307356	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.LargeUriAndHeaders_Works	#124788
1307341	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124650
1307351	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124646
1307312	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123138
1307170	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#122526
1307166	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution
1307150	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124397
1307039	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123939
1307035	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124586
1307000	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124738
1306959	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution
1306766	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124731
1306821	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123614
1306872	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124203
1306890	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124778
1306655	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#120275
1306756	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124685
1306682	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.LargeUriAndHeaders_Works	#124773
1306737	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#122533
1306689	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124772
1306554	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124763
1306712	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124774
1306733	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124775
1306700	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124736
1306478	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.LargeUriAndHeaders_Works	#124682
1306526	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124663
1306511	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124589
1306503	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124771
1306472	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#123830
1306421	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124244
1306379	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124268
1306413	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#122898
1306398	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124731
1303542	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124560
1306363	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124720
1306299	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#122649
1306325	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124761
1306334	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124760
1306311	dotnet/runtime	System.Net.Http.Functional.Tests.WorkItemExecution	#124649
Displaying 100 of 474 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
68	433	474

[EgorBo]

@dotnet/ncl Can someone take a look at this? it has 365 hits per week on CI, all JIT PRs are always red because of it.

[ManickaP]

This is android specific AFAICS, isn't it?
cc @simonrozsival

[simonrozsival]

@ManickaP yes

[wfurt]

we may need to disable the offending tests if real fix drags too long.....

[vitek-karas]

Investigation Summary: Android Emulator CI Failure

Root Cause

The failures are caused by an Android Emulator version upgrade (36.2.x → 36.4.9) picked up during Helix VMSS re-imaging on Feb 14-15.

Timeline

Date	Event
Nov 12, 2025	Base Helix image built (Helix-Ubuntu-2204-2025-11-12) — emulator ≤36.2.12
Feb 10, 2026	Google releases Android Emulator 36.4.9 Stable with major networking changes (netsimd replaces QEMU pipe networking, libslirp stack updated, Lavapipe as default software renderer)
Feb 11-14	dotnet-helix-machines PRs (including PR 57760 adding linux-ubuntu-disable-unattended-upgrades to Android queues) trigger VMSS re-imaging
Feb 14, 20:01 UTC	Last green rolling build (1294907) — likely ran on a VM not yet re-imaged
Feb 15, 08:01 UTC	First red rolling build (1295230) — re-imaged VM with emulator 36.4.9

Evidence

• Not a code change: The 6 commits between the last green and first red builds touch zero SSL/HTTP/Android files.
• Systemic: Verified across 5 failing builds on 5 different Helix machines — all show identical symptoms (timeout at 1800s, exit code 70, 0 test results).
• QEMU pipe errors: Failing builds show 6-11 Failed to open QEMU pipe 'qemud:network': Invalid argument errors vs ~1 in green builds. This is consistent with emulator 36.2.6+ changing the WiFi packets handler to go through netsimd instead of QEMU pipes — API 29 system images still try the old qemud:network pipe, which no longer works properly.
• Unpinned emulator version: The Helix onboarding script (linux-ubuntu-android-emulator.sh) uses sdkmanager ... "emulator" which always installs the latest version — so re-imaging picked up 36.4.9.

Recommended Fix

Pin the Android emulator version in the dotnet-helix-machines repo's linux-ubuntu-android-emulator.sh to 36.2.12 (last known-good stable from Oct 13, 2025), or to 36.3.10 (Dec 4, 2025 stable). This prevents future uncontrolled emulator upgrades from breaking CI.

[wfurt]

Do you know what package name we need to pin @vitek-karas ? Also, there was debate with @simonrozsival and @akoeplinger that we may switch to docker images. That would give you full control over all aspects. But I'm not sure if we are ready....but it would be nice.

[vitek-karas]

I'm not convinced that pinning the version is the right solution - we already have way too many things which need "regular" manual updates - it adds too much cost to the maintenance. Historically I'm not aware of Android emulator updates causing problems - so if this is a one-off - I would personally prefer to keep it auto-updating and dealing with rare occasional breaks when they happen.
Similar for docker images - yes - it would give us full control - but it comes with the same maintenance cost.