{"id":3070,"date":"2025-12-16T18:15:58","date_gmt":"2025-12-16T18:15:58","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2025\/12\/16\/microsoft-testing-platform-now-fully-supported-in-azure-devops\/"},"modified":"2025-12-16T18:15:58","modified_gmt":"2025-12-16T18:15:58","slug":"microsoft-testing-platform-now-fully-supported-in-azure-devops","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2025\/12\/16\/microsoft-testing-platform-now-fully-supported-in-azure-devops\/","title":{"rendered":"Microsoft.Testing.Platform Now Fully Supported in Azure DevOps"},"content":{"rendered":"<p>Earlier this year, we announced that <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/mtp-adoption-frameworks\/\">all major .NET test frameworks now support Microsoft.Testing.Platform<\/a>. The next logical step? Making sure it works seamlessly with your CI\/CD pipelines. Today, we\u2019re announcing comprehensive Azure DevOps integration for Microsoft.Testing.Platform.<\/p>\n<h2>What\u2019s New in Azure DevOps<\/h2>\n<p>Azure DevOps now provides first-class support for Microsoft.Testing.Platform with two key improvements:<\/p>\n<ul>\n<li><strong>Run tests<\/strong> using the familiar DotNetCoreCLI task; no more workarounds required.<\/li>\n<li><strong>Handle test retries<\/strong> intelligently\u2014publish multiple TRX files from retry attempts with proper grouping and exit codes.<\/li>\n<\/ul>\n<p>Whether you\u2019re migrating from VSTest or starting fresh with Microsoft.Testing.Platform, the experience is now smooth and intuitive.<\/p>\n<h2>About Microsoft.Testing.Platform<\/h2>\n<p>If you\u2019re new to Microsoft.Testing.Platform, here are some helpful resources to get you up to speed:<\/p>\n<ul>\n<li><a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/testing\/\">Testing in .NET<\/a> gives an overview of the testing tools used in .NET. In particular, it clarifies the difference between a test platform and a test framework.<\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/testing\/unit-testing-with-dotnet-test\">Testing with \u2018dotnet test\u2019<\/a> explains the different modes that the \u2018dotnet test\u2019 command can operate in.<\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/testing\/microsoft-testing-platform-intro\">Microsoft.Testing.Platform overview<\/a> covers what Microsoft.Testing.Platform is and how to run tests with it.<\/li>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/dotnet-test-with-mtp\/\">Enhance your CLI testing workflow with the new dotnet test<\/a> shows the \u2018dotnet test\u2019 support for Microsoft.Testing.Platform that was added in .NET 10 SDK.<\/li>\n<\/ul>\n<p>In short, Microsoft.Testing.Platform is a modern alternative to VSTest.<\/p>\n<h2>Running Tests in Azure DevOps<\/h2>\n<p>The DotNetCoreCLI task now supports Microsoft.Testing.Platform starting in version 2.263.0!<\/p>\n<p>You have two main options for running tests with Microsoft.Testing.Platform in Azure DevOps:<\/p>\n<h3>Option 1: Use the DotNetCoreCLI task (Recommended)<\/h3>\n<p>The <a href=\"https:\/\/learn.microsoft.com\/azure\/devops\/pipelines\/tasks\/reference\/dotnet-core-cli-v2\">DotNetCoreCLI Azure DevOps task<\/a> now <a href=\"https:\/\/github.com\/microsoft\/azure-pipelines-tasks\/pull\/21315\/\">supports Microsoft.Testing.Platform<\/a> starting in version 2.263.0.<\/p>\n<pre><code class=\"language-yml\">- task: DotNetCoreCLI@2\n  displayName: 'Run tests'\n  inputs:\n    command: 'test'\n    projects: '**\/*Tests.csproj'\n    arguments: '--no-build --report-trx'<\/code><\/pre>\n<p><strong>What changed?<\/strong> Microsoft.Testing.Platform uses different command-line flags than VSTest. For example, use <code>--report-trx<\/code> instead of <code>--logger trx<\/code>. See the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/testing\/microsoft-testing-platform-intro\">Microsoft.Testing.Platform CLI documentation<\/a> for the full list of options.<\/p>\n<h3>Option 2: Run \u2018dotnet test\u2019 directly<\/h3>\n<p>You can run the <code>dotnet test<\/code> command directly using a script or command-line task. This gives you maximum flexibility and doesn\u2019t require specific Azure DevOps task updates.<\/p>\n<pre><code class=\"language-yml\">- task: CmdLine@2\n  displayName: 'Run tests'\n  inputs:\n    script: 'dotnet test --no-build --report-trx'<\/code><\/pre>\n<p><strong>When to use this?<\/strong> Choose this approach if you need custom scripting around test execution or want to avoid dependencies on specific task versions.<\/p>\n<h3>Migrating from VSTest task<\/h3>\n<p>The <a href=\"https:\/\/learn.microsoft.com\/azure\/devops\/pipelines\/tasks\/reference\/vstest-v3\">VSTest Azure DevOps task<\/a> was designed exclusively for VSTest and will not support Microsoft.Testing.Platform.<\/p>\n<p><strong>If you\u2019re currently using the VSTest task:<\/strong><\/p>\n<ol>\n<li>Switch to Option 1 (DotNetCoreCLI task) for the most similar experience<\/li>\n<li>Update your command-line arguments to use Microsoft.Testing.Platform syntax<\/li>\n<li>Test your pipeline to ensure test discovery and execution work as expected<\/li>\n<\/ol>\n<p>Both Option 1 and Option 2 provide the same test execution capabilities as the VSTest task.<\/p>\n<h2>Publishing test results with retry support<\/h2>\n<p>Here\u2019s the challenge: when you use the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Testing.Extensions.Retry\">Retry extension<\/a> to automatically re-run failed tests, Microsoft.Testing.Platform generates a separate TRX file for each attempt. Previously, the <code>PublishTestResults<\/code> task treated these as independent test runs, leading to two problems:<\/p>\n<ul>\n<li><strong>Incorrect exit codes<\/strong>: The task would fail even when tests eventually passed after retry<\/li>\n<li><strong>Confusing UI<\/strong>: Test results appeared as separate runs instead of grouped retry attempts<\/li>\n<\/ul>\n<p>We\u2019ve solved this! The <code>PublishTestResults<\/code> task now intelligently handles multiple TRX files from retry attempts, correctly grouping them and setting appropriate exit codes.<\/p>\n<h3>What you need to know<\/h3>\n<p>Microsoft.Testing.Platform uses the standard TRX format (also known as VSTest test results format) through the <a href=\"https:\/\/nuget.org\/packages\/Microsoft.Testing.Extensions.TrxReport\">Microsoft.Testing.Extensions.TrxReport<\/a> package. The <a href=\"https:\/\/learn.microsoft.com\/azure\/devops\/pipelines\/tasks\/reference\/publish-test-results-v2\">PublishTestResults task<\/a> has always supported TRX files. What\u2019s new is the intelligent handling of retry scenarios.<\/p>\n\n<div class=\"alert alert-info\">\n<p class=\"alert-divider\"><i class=\"fabric-icon fabric-icon--Info\"><\/i><strong>Required Configuration<\/strong><\/p>\n<p>To enable retry-aware test result publishing, set the <code>AllowPtrToDetectTestRunRetryFiles<\/code> variable to <code>true<\/code> in your pipeline. This opt-in flag activates the new behavior that correctly interprets multiple TRX files as retry attempts rather than separate test runs.<\/p>\n<p><strong>Important:<\/strong> This variable must be set at the pipeline level (in your YAML file or pipeline variables). It will not work if set at the project or organization level.<\/p>\n<\/div>\n<h3>Complete pipeline example<\/h3>\n<p>Here\u2019s a full pipeline demonstrating test execution with retries and proper result publishing:<\/p>\n<pre><code class=\"language-yml\">trigger:\n- main\n\njobs:\n- job: MyJob\n  variables:\n    AllowPtrToDetectTestRunRetryFiles: true\n\n  steps:\n\n  # Your usual steps to install .NET SDK, restore, and build.\n\n  - task: CmdLine@2\n    displayName: 'Run tests'\n    inputs:\n      script: 'dotnet test --no-build --report-trx --retry-failed-tests 3 --results-directory TestResults'\n\n  - task: PublishTestResults@2\n    displayName: 'Publish test results'\n    inputs:\n      testResultsFormat: 'VSTest'\n      testResultsFiles: '**\/*.trx'\n      mergeTestResults: true\n      failTaskOnFailedTests: true\n    condition: succeededOrFailed()<\/code><\/pre>\n<p><strong>Key points in this pipeline:<\/strong><\/p>\n<ul>\n<li>The <code>AllowPtrToDetectTestRunRetryFiles: true<\/code> variable enables retry-aware behavior<\/li>\n<li>The <code>--retry-failed-tests 3<\/code> flag tells the test platform to retry failed tests up to 3 times<\/li>\n<li>The <code>condition: succeededOrFailed()<\/code> ensures test results publish even if tests fail<\/li>\n<\/ul>\n<h3>Understanding the retry outcomes<\/h3>\n<p>Let\u2019s see how the Azure DevOps UI displays results across different retry scenarios. Imagine a test suite with one stable test that always passes and one flaky test with intermittent failures.<\/p>\n<h4>Scenario 1: All tests pass on first attempt<\/h4>\n<p>When all tests pass immediately, you see standard success indicators. The <code>PublishTestResults<\/code> task completes successfully.<\/p>\n<p><img data-opt-id=1417283790  fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2025\/12\/passed-on-first-run.webp\" alt=\"Azure DevOps Tests UI showing tests passing on first run\" \/><\/p>\n<h4>Scenario 2: Test passes after retry<\/h4>\n<p>This is where the new behavior shines. The UI shows the test failed on attempts 1 and 2 but succeeded on attempt 3. Even though there were initial failures, the <code>PublishTestResults<\/code> task passes because the test eventually succeeded. This is the correct behavior for handling flaky tests.<\/p>\n<p><img data-opt-id=1042928912  fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2025\/12\/passed-on-re-run.webp\" alt=\"Azure DevOps Tests UI showing tests passing on re-run\" \/><\/p>\n<h4>Scenario 3: Test fails all retry attempts<\/h4>\n<p>When a test fails across all attempts, the UI groups all failures together for easy review. You can examine each individual failure to identify patterns or differences. The <code>PublishTestResults<\/code> task fails as expected when <code>failTaskOnFailedTests<\/code> is true.<\/p>\n<p><img data-opt-id=367740408  data-opt-src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2025\/12\/failed-on-all-runs.webp\"  decoding=\"async\" src=\"data:image/svg+xml,%3Csvg%20viewBox%3D%220%200%20100%%20100%%22%20width%3D%22100%%22%20height%3D%22100%%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%3Crect%20width%3D%22100%%22%20height%3D%22100%%22%20fill%3D%22transparent%22%2F%3E%3C%2Fsvg%3E\" alt=\"Azure DevOps Tests UI showing tests failing for all attempts\" \/><\/p>\n\n<div class=\"alert alert-success\">\n<p class=\"alert-divider\"><i class=\"fabric-icon fabric-icon--Lightbulb\"><\/i><strong>Why This Matters<\/strong><\/p>\n<p>The distinction between VSTest and Microsoft.Testing.Platform is important here: VSTest never had built-in retry support, so this scenario couldn\u2019t occur naturally. With Microsoft.Testing.Platform\u2019s Retry extension, you get automatic retry handling with proper Azure DevOps integration; no custom scripting required.<\/p><\/div>\n<h2>Get Started Today<\/h2>\n<p>Azure DevOps now has full support for Microsoft.Testing.Platform\u2014from running tests to publishing retry results with intelligent handling.<\/p>\n<p><strong>Ready to migrate?<\/strong><\/p>\n<ol>\n<li>Update to .NET SDK 10<\/li>\n<li>Add the Retry extension to your test project<\/li>\n<li>Set <code>AllowPtrToDetectTestRunRetryFiles: true<\/code> in your pipeline<\/li>\n<li>Run your tests with <code>--retry-failed-tests<\/code> flag<\/li>\n<\/ol>\n<p><strong>Need help?<\/strong> We\u2019re here for you:<\/p>\n<ul>\n<li>Ask questions: <a href=\"https:\/\/github.com\/microsoft\/testfx\/discussions\/new\/choose\">GitHub Discussions<\/a><\/li>\n<li>Report issues: <a href=\"https:\/\/github.com\/microsoft\/testfx\/issues\/new\/choose\">GitHub Issues<\/a><\/li>\n<li>Learn more: <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/testing\/microsoft-testing-platform-intro\">Microsoft.Testing.Platform documentation<\/a><\/li>\n<\/ul>\n<p>The post <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/microsoft-testing-platform-azure-retry\/\">Microsoft.Testing.Platform Now Fully Supported in Azure DevOps<\/a> appeared first on <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\">.NET Blog<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Earlier this year, we announced that all major .NET test frameworks now support Microsoft.Testing.Platform. The next logical step? Making sure [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3071,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-3070","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet"],"_links":{"self":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/3070","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/comments?post=3070"}],"version-history":[{"count":0,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/3070\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media\/3071"}],"wp:attachment":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media?parent=3070"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/categories?post=3070"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/tags?post=3070"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}