Console log: 'Microsoft.Extensions.Hosting.Functional.Tests' from job 9f894df3-bb7c-4704-8fed-02d2c108a8c7 (azurelinux.3.arm64.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-helix-arm64v8@sha256:71ebe961ba52031c21a19a0b5f32cb03be8b246bb63ccd2111c8dbc2b8af2a84 on a00AK6G running $HELIX_CORRELATION_PAYLOAD/scripts/29d73e75c7f048268fdc95858154b916/execute.sh in /datadisks/disk1/work/B48709B1/w/BA0A09CE/e max 2700 seconds Output: [BEGIN EXECUTION] + cd /root/helix/work/workitem/e + mkdir -p /datadisks/disk1/dumps/ + /root/helix/work/correlation/scripts/29d73e75c7f048268fdc95858154b916/execute.sh + export __TestArchitecture=arm64 + __TestArchitecture=arm64 + ./RunTests.sh --runtime-path /root/helix/work/correlation ========================= Begin custom configuration settings ============================== export XUNIT_HIDE_PASSING_OUTPUT_DIAGNOSTICS=1 ========================== End custom configuration settings =============================== ----- start Mon May 18 12:45:00 UTC 2026 =============== To repro directly: ===================================================== pushd . /root/helix/work/correlation/dotnet exec --runtimeconfig Microsoft.Extensions.Hosting.Functional.Tests.runtimeconfig.json --depsfile Microsoft.Extensions.Hosting.Functional.Tests.deps.json xunit.console.dll Microsoft.Extensions.Hosting.Functional.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing popd =========================================================================================================== /root/helix/work/workitem/e /root/helix/work/workitem/e Discovering: Microsoft.Extensions.Hosting.Functional.Tests (method display = ClassAndMethod, method display options = None) Discovered: Microsoft.Extensions.Hosting.Functional.Tests (found 2 test cases) Starting: Microsoft.Extensions.Hosting.Functional.Tests (parallel test collections = on [2 threads], stop on fail = off) Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ShutdownTestRun [FAIL] System.ComponentModel.Win32Exception : An error occurred trying to start process 'pgrep' with working directory '/root/helix/work/workitem/e'. No such file or directory Stack Trace: /_/src/libraries/System.Diagnostics.Process/src/Microsoft/Win32/SafeHandles/SafeProcessHandle.Unix.cs(369,0): at Microsoft.Win32.SafeHandles.SafeProcessHandle.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, IDictionary`2 env, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, Boolean usesTerminal, SafeHandle[] inheritedHandles, Holder& waitStateHolder, Boolean throwOnNoExec) /_/src/libraries/System.Diagnostics.Process/src/Microsoft/Win32/SafeHandles/SafeProcessHandle.Unix.cs(213,0): at Microsoft.Win32.SafeHandles.SafeProcessHandle.StartCore(ProcessStartInfo startInfo, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, SafeHandle[] inheritedHandles, Holder& waitStateHolder) /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs(363,0): at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, SafeHandle[] inheritedHandles) /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs(1272,0): at System.Diagnostics.Process.Start() /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs(1383,0): at System.Diagnostics.Process.Start(ProcessStartInfo startInfo) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(93,0): at Microsoft.Extensions.Internal.ProcessExtensions.RunProcessAndWaitForExit(String fileName, String arguments, TimeSpan timeout, String& stdout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(44,0): at Microsoft.Extensions.Internal.ProcessExtensions.GetAllChildIdsUnix(Int32 parentId, ISet`1 children, TimeSpan timeout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(33,0): at Microsoft.Extensions.Internal.ProcessExtensions.KillTree(Process process, TimeSpan timeout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(17,0): at Microsoft.Extensions.Internal.ProcessExtensions.KillTree(Process process) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/Deployers/ApplicationDeployer.cs(126,0): at Microsoft.Extensions.Hosting.IntegrationTesting.ApplicationDeployer.ShutDownIfAnyHostProcess(Process hostProcess) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/Deployers/SelfHostDeployer.cs(179,0): at Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer.Dispose() /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/ShutdownTests.cs(107,0): at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ExecuteShutdownTest(String testName, String shutdownMechanic) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/ShutdownTests.cs(34,0): at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ShutdownTestRun() --- End of stack trace from previous location --- Output: | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Deploying [Variation] :: Runtime=CoreClr, Arch=x64, Publish=True | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Executing dotnet /root/helix/work/workitem/e/Microsoft.Extensions.Hosting.TestApp.dll | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Removing environment variable ASPNETCORE_ENVIRONMENT | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: SET DOTNET_STARTMECHANIC=Run | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Started dotnet. Process Id : 28 | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Application ready | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: dotnet stdout: Started | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Attempting to cancel process 28 Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ShutdownTestWaitForShutdown [FAIL] System.ComponentModel.Win32Exception : An error occurred trying to start process 'pgrep' with working directory '/root/helix/work/workitem/e'. No such file or directory Stack Trace: /_/src/libraries/System.Diagnostics.Process/src/Microsoft/Win32/SafeHandles/SafeProcessHandle.Unix.cs(369,0): at Microsoft.Win32.SafeHandles.SafeProcessHandle.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, IDictionary`2 env, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, Boolean usesTerminal, SafeHandle[] inheritedHandles, Holder& waitStateHolder, Boolean throwOnNoExec) /_/src/libraries/System.Diagnostics.Process/src/Microsoft/Win32/SafeHandles/SafeProcessHandle.Unix.cs(213,0): at Microsoft.Win32.SafeHandles.SafeProcessHandle.StartCore(ProcessStartInfo startInfo, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, SafeHandle[] inheritedHandles, Holder& waitStateHolder) /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs(363,0): at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo, SafeFileHandle stdinHandle, SafeFileHandle stdoutHandle, SafeFileHandle stderrHandle, SafeHandle[] inheritedHandles) /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs(1272,0): at System.Diagnostics.Process.Start() /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs(1383,0): at System.Diagnostics.Process.Start(ProcessStartInfo startInfo) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(93,0): at Microsoft.Extensions.Internal.ProcessExtensions.RunProcessAndWaitForExit(String fileName, String arguments, TimeSpan timeout, String& stdout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(44,0): at Microsoft.Extensions.Internal.ProcessExtensions.GetAllChildIdsUnix(Int32 parentId, ISet`1 children, TimeSpan timeout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(33,0): at Microsoft.Extensions.Internal.ProcessExtensions.KillTree(Process process, TimeSpan timeout) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/ProcessExtensions.cs(17,0): at Microsoft.Extensions.Internal.ProcessExtensions.KillTree(Process process) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/Deployers/ApplicationDeployer.cs(126,0): at Microsoft.Extensions.Hosting.IntegrationTesting.ApplicationDeployer.ShutDownIfAnyHostProcess(Process hostProcess) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/IntegrationTesting/src/Deployers/SelfHostDeployer.cs(179,0): at Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer.Dispose() /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/ShutdownTests.cs(107,0): at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ExecuteShutdownTest(String testName, String shutdownMechanic) /_/src/libraries/Microsoft.Extensions.Hosting/tests/FunctionalTests/ShutdownTests.cs(41,0): at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ShutdownTestWaitForShutdown() --- End of stack trace from previous location --- Output: | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Deploying [Variation] :: Runtime=CoreClr, Arch=x64, Publish=True | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Executing dotnet /root/helix/work/workitem/e/Microsoft.Extensions.Hosting.TestApp.dll | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Removing environment variable ASPNETCORE_ENVIRONMENT | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: SET DOTNET_STARTMECHANIC=WaitForShutdown | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Started dotnet. Process Id : 43 | [2026-05-18T12:45:01] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Application ready | [2026-05-18T12:45:02] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: dotnet stdout: Started | [2026-05-18T12:45:02] Microsoft.Extensions.Hosting.IntegrationTesting.SelfHostDeployer Information: Attempting to cancel process 43 Finished: Microsoft.Extensions.Hosting.Functional.Tests === TEST EXECUTION SUMMARY === Microsoft.Extensions.Hosting.Functional.Tests Total: 2, Errors: 0, Failed: 2, Skipped: 0, Time: 0.670s /root/helix/work/workitem/e ----- end Mon May 18 12:45:02 UTC 2026 ----- exit code 1 ---------------------------------------------------------- ulimit -c value: unlimited cat /proc/sys/kernel/core_pattern: /datadisks/disk1/dumps/core.%u.%p cat /proc/sys/kernel/core_uses_pid: 1 cat: /proc/sys/kernel/coredump_filter: No such file or directory cat /proc/sys/kernel/coredump_filter: Looking around for any Linux dumps... Looking for files matching core.* ... The '__IsXUnitLogCheckerSupported' env var is not set. + export _commandExitCode=0 + _commandExitCode=0 + python /root/helix/work/correlation/reporter/run.py https://dev.azure.com/dnceng-public/ public 39604340 eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Ilh0LW83aERicHVwQXotWlBtNkh4Q0ZXUzNjSSIsImtpZCI6Ilh0LW83aERicHVwQXotWlBtNkh4Q0ZXUzNjSSJ9.eyJhdWQiOiI0OTliODRhYy0xMzIxLTQyN2YtYWExNy0yNjdjYTY5NzU3OTgiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC83MmY5ODhiZi04NmYxLTQxYWYtOTFhYi0yZDdjZDAxMWRiNDcvIiwiaWF0IjoxNzc5MTA2MjUwLCJuYmYiOjE3NzkxMDYyNTAsImV4cCI6MTc3OTE5Mjk1MCwiYWlvIjoiazJGZ1lOQThtK3BYOC9iWVZLMEFxN1FuRGUxemd2K3Q5WE9aWmhCeXpTdlIzTzE1eDNZQSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJpZHR5cCI6ImZtaSIsInJoIjoiMS5BUm9BdjRqNWN2R0dyMEdScXkxODBCSGJSNnlFbTBraEUzOUNxaGNtZkthWFY1Z0FBQUFhQUEuIiwic3ViIjoiL2VpZDEvYy9wdWIvdC92NGo1Y3ZHR3IwR1JxeTE4MEJIYlJ3L2EvcklTYlNTRVRmMEtxRnlaOHBwZFhtQS9icy9oLzZmY2M5MmU1LTczYTctNGY4OC04ZDEzLWQ5MDQ1YjQ1ZmIyNy9pL2M3NzNmMmMyLTUxMjAtNDIwNy1hZmUyLWFmYWYzNWE4YmMwYSIsInRpZCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsInV0aSI6Ijd0VU52d1FzdDBhLU42dnJ3TGtBQUEiLCJ2ZXIiOiIxLjAiLCJ4bXNfYXR0ciI6eyJySVNiU1NFVGYwS3FGeVo4cHBkWG1BIjp7ImFkb19leHAiOjE3NzkxMzU5NTAsImFkb19zY3AiOiJhcHBfdG9rZW4iLCJhdWkiOiI3NWQyNjk5Zi03YTkwLTRkN2UtYTRmZC02MzY0Zjk4NWNhMWYiLCJCdWlsZElkIjoiY2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5OzE0MjQ2NDUiLCJEZWZJZCI6IjEyOSIsImpvYnJlZiI6IjNiMjg5MTQ0LTBjNGYtNGEyYy05ZDQ3LWNkNjZiODc1MWQ2NzpmN2U0ZDg2OS04ZjEyLTUzMWQtNmViYy1jNGI2MGIyYTEzYmMiLCJvcmNoaWQiOiIzYjI4OTE0NC0wYzRmLTRhMmMtOWQ0Ny1jZDY2Yjg3NTFkNjcuYnVpbGQuYnVpbGRfbGludXhfYXJtNjRfZGVidWdfbW9ub19taW5paml0X2xpYnJhcmllc3Rlc3RzLl9fZGVmYXVsdCIsInBjYyI6Ilt7XCJTY29wZVwiOlwiMTI5O2NiYjE4MjYxLWM0OGYtNGFiYi04NjUxLThjZGNiNTQ3NDY0OTtyZWZzL2hlYWRzL2NvcGlsb3QvZW5hYmxlLWZ1bmN0aW9uYWwtdGVzdHMtb24taG9zdGluZztkb3RuZXQvcnVudGltZVwiLFwiUGVybWlzc2lvblwiOjF9LHtcIlNjb3BlXCI6XCIxMjk7Y2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5O3JlZnMvaGVhZHMvbWFpbjtkb3RuZXQvcnVudGltZVwiLFwiUGVybWlzc2lvblwiOjF9LHtcIlNjb3BlXCI6XCIxMjk7Y2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5O3JlZnMvaGVhZHMvbWFpbjtkb3RuZXQvcnVudGltZVwiLFwiUGVybWlzc2lvblwiOjF9LHtcIlNjb3BlXCI6XCIxMjk7Y2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5O3JlZnMvcHVsbC8xMjgyNTIvbWVyZ2U7ZG90bmV0L3J1bnRpbWVcIixcIlBlcm1pc3Npb25cIjozfSx7XCJTY29wZVwiOlwiMTI5O2NiYjE4MjYxLWM0OGYtNGFiYi04NjUxLThjZGNiNTQ3NDY0OTtyZWZzL2hlYWRzL21hc3Rlcjtkb3RuZXQvcnVudGltZVwiLFwiUGVybWlzc2lvblwiOjF9XSIsInBwaWQiOiJ2c3RmczovLy9CdWlsZC9CdWlsZC8xNDI0NjQ1In19LCJ4bXNfaWRyZWwiOiI2IDMzIiwieG1zX3JkIjoiMC5BVmtBcHY4S0JRZ0NFZ0VhRWhRSUNSSVFvS1hiY1h5blpFdTh4UFg1aS1Kbl9oSVVDQXNTRUx2RmRXWGh3bGxXZnBUZnc1UzR4OEFpSkFnREVpREktMlNoYnBIMElEcVJseWlJQjhpbmlTV0xNUHZ2blFxUjZYVjlrb00yX0EifQ.bDE6FZO4dc0ueQldJ9pEFkLs3unyXuL1QtlgqlVgTYG0gMnGeCmZNSovsAOBwVUiGPOSgxOVjanZZzjpoIe5ZNOlCrP1-Tnba-G-Fz1CdMbIZKav1IP7GiGngeEvsgl5FnWmKsC-fLgEgUmQlCqzefr-t3SdRKOgqYDaSb229GUIIgdpJTCAHPCBX0L8_JfT9utvyS19sI8tyaIFU9yfKYku5LW8N_RN-WS61VAYDQObfSiJrhkxdKTJJBtRIosAAQQbVu3RZFzdkYcpmfE2VnlIZ8SPTjknrQXyue0m5YyaNErF5EEBwgfr_O-BUWatIpCvC7WD6hzUsU2SU8djsw /home/helixbot/.vsts-env/lib/python3.12/site-packages/azure/__init__.py:5: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources 2026-05-18T12:45:03.555Z INFO run.py managed_identity(151) __init__ ManagedIdentityCredential will use IMDS with client_id: 3423231b-6b9e-4177-8817-76276b647404 2026-05-18T12:45:03.555Z INFO run.py azure_utils(32) get_credential_and_access_token VMSSManagedIdentity VmssMIClientIdForUpload Auth type will be used 2026-05-18T12:45:03.555Z INFO run.py _universal(535) on_request Request URL: 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED&client_id=REDACTED' Request method: 'GET' Request headers: 'Metadata': 'REDACTED' 'User-Agent': 'azsdk-python-identity/1.25.3 Python/3.12.9 (Linux-6.6.130.1-3.azl3-aarch64-with-glibc2.38)' No body was attached to the request 2026-05-18T12:45:03.565Z INFO run.py _universal(581) on_response Response status: 200 Response headers: 'Content-Type': 'application/json; charset=utf-8' 'Server': 'IMDS/150.870.65.2020' 'x-ms-request-id': 'e72aab52-e4ff-437d-aa30-3dc3c345ce83' 'Date': 'Mon, 18 May 2026 12:45:03 GMT' 'Content-Length': '2468' 2026-05-18T12:45:03.566Z INFO run.py msal_managed_identity_client(165) _get_token_base ImdsCredential.get_token succeeded 2026-05-18T12:45:03.566Z INFO run.py decorators(24) wrapper ManagedIdentityCredential.get_token succeeded 2026-05-18T12:45:03.566Z INFO run.py azure_utils(64) get_credential_and_access_token Credentials are valid 2026-05-18T12:45:03.567Z INFO run.py msal_managed_identity_client(165) _get_token_base ImdsCredential.get_token_info succeeded 2026-05-18T12:45:03.567Z INFO run.py decorators(24) wrapper ManagedIdentityCredential.get_token_info succeeded 2026-05-18T12:45:03.620Z INFO run.py run(48) main Beginning reading of test results. 2026-05-18T12:45:03.620Z INFO run.py __init__(42) read_results Searching '/root/helix/work/workitem/e' for test results files 2026-05-18T12:45:03.621Z INFO run.py __init__(48) read_results Found results file /root/helix/work/workitem/e/testResults.xml with format xunit 2026-05-18T12:45:03.621Z INFO run.py __init__(42) read_results Searching '/root/helix/work/workitem/uploads' for test results files 2026-05-18T12:45:03.622Z INFO run.py packing_test_reporter(30) report_results Packing 2 test reports to '/root/helix/work/workitem/e/__test_report.json' 2026-05-18T12:45:03.622Z INFO run.py packing_test_reporter(33) report_results Packed 13295 bytes + exit 0 + _commandExitCode=0 + chmod -R 777 /datadisks/disk1/dumps + exit 0 [END EXECUTION] Exit Code:0