{"id":4701,"date":"2026-07-31T15:12:10","date_gmt":"2026-07-31T15:12:10","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2026\/07\/31\/from-generated-code-to-trusted-code-with-a-unit-test-agent\/"},"modified":"2026-07-31T15:12:10","modified_gmt":"2026-07-31T15:12:10","slug":"from-generated-code-to-trusted-code-with-a-unit-test-agent","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2026\/07\/31\/from-generated-code-to-trusted-code-with-a-unit-test-agent\/","title":{"rendered":"From generated code to trusted code with a unit-test agent"},"content":{"rendered":"<p>A common request to a coding agent is only one line:<\/p>\n<blockquote>\n<p>Generate unit tests.<\/p>\n<\/blockquote>\n<p>That request leaves important questions open. Which code needs tests? Which<br \/>\ntest framework does the project use? Where should the tests go? How does the<br \/>\nbuild find them? What should the tests check?<\/p>\n<p>The agent can learn from the repository, write the tests, and prove that they<br \/>\nwork.<\/p>\n<p>This is why we built<br \/>\n<a href=\"https:\/\/github.com\/dotnet\/skills\/blob\/main\/plugins\/dotnet-test\/agents\/code-testing-generator.agent.md\"><code>code-testing-generator<\/code><\/a>.<br \/>\nIt is an open-source, polyglot agent for unit-test generation. You can find it<br \/>\nin the<br \/>\n<a href=\"https:\/\/github.com\/dotnet\/skills\/tree\/main\/plugins\/dotnet-test\"><code>dotnet-test<\/code> plugin<\/a><br \/>\nin <a href=\"https:\/\/github.com\/dotnet\/skills\"><code>dotnet\/skills<\/code><\/a>.<\/p>\n<p>The agent writes <strong>unit tests<\/strong>. It isolates the code under test and mocks<br \/>\nexternal services and other outside dependencies.<\/p>\n<p>Integration, end-to-end, browser, and performance tests are outside its current<br \/>\nscope.<\/p>\n<p>It does more than make the tests pass. It checks the assertions, the requested<br \/>\nscenarios, and whether the repository\u2019s normal test run finds the new tests.<\/p>\n<h2>What happens after the prompt<\/h2>\n<p>The agent does not start writing tests immediately. It learns from the<br \/>\nrepository first, then plans, writes, and checks the tests.<\/p>\n<p><img data-opt-id=2016006668  fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/unit-testing-agent-workflow.png\" alt=\"Workflow from repository research through planning, implementation, test execution, and quality checks\" \/><\/p>\n<h3>First, learn from the repository<\/h3>\n<p>The agent searches the repository for the code that needs tests. It detects the<br \/>\nlanguage and test framework. It also looks for existing tests that show where<br \/>\nnew tests belong and how they should look.<\/p>\n<p>It also finds the correct commands to build and run the tests.<\/p>\n<p>This prevents a common problem. A new test project can build and pass on its<br \/>\nown but never run in continuous integration (CI) because it was not added to<br \/>\nthe solution or the repository\u2019s test command. The agent checks how the<br \/>\nrepository discovers tests and confirms that the new tests appear there.<\/p>\n<h3>Next, choose the right amount of work<\/h3>\n<p>A single method does not need a large plan. A whole solution does.<\/p>\n<p>The agent chooses from three paths:<\/p>\n<ul>\n<li><strong>Direct:<\/strong> Read the relevant code, write the tests, and validate the result.<\/li>\n<li><strong>Single pass:<\/strong> Research and plan once, then implement that plan.<\/li>\n<li><strong>Iterative:<\/strong> Repeat the cycle to cover a large request or reach a coverage<br \/>\ngoal.<\/li>\n<\/ul>\n<h3>Then, plan and write the tests<\/h3>\n<p>For larger tasks, the agent lists the code that needs tests. It starts with<br \/>\nsimple code and then moves to code with more dependencies. It maps each<br \/>\nbehavior to a test file.<\/p>\n<p>The plan follows the request. A request for one module should not change every<br \/>\ntest project in the repository.<\/p>\n<p>The agent follows the local conventions and runs the tests as it works. If the<br \/>\ngenerated code does not compile, it fixes it. If an assertion is wrong, it<br \/>\nreads the source and corrects the test.<\/p>\n<p>It does not change production code during test generation. It also avoids unit<br \/>\ntests that call external URLs, open ports, or depend on exact timing.<\/p>\n<h3>Finally, check that the tests are useful<\/h3>\n<p>A test can pass but provide little or no added value. For example, it may only<br \/>\ncheck that a result is not null. It may test the wrong method. It may even pass<br \/>\nif the method always returns a default value.<\/p>\n<p>The agent checks for these problems before it finishes:<\/p>\n<ol>\n<li>It considers small code changes that should make the tests fail. This is a<br \/>\nlightweight form of mutation testing.<\/li>\n<li>It looks for weak or missing assertions.<\/li>\n<li>It checks that every requested scenario has a matching test.<\/li>\n<li>It builds the full workspace and runs the full test suite.<\/li>\n<li>It confirms that the repository\u2019s test command can find the new tests.<\/li>\n<\/ol>\n<p>This workflow turns a short prompt into a tested result.<\/p>\n<h2>What we measured<\/h2>\n<p>In our latest benchmark, the agent completed 140 of 152 tasks. Stock Copilot,<br \/>\nwhich used the same tool without our plugin, completed 120 with the same model.<br \/>\nThat is a <strong>92.1% completion rate versus 78.9%<\/strong>, with 63% fewer failures.<\/p>\n<p>The biggest gains came from prompts like the one at the start of this post. We<br \/>\ncall these <strong>vague prompts<\/strong> because they leave most decisions to the agent.<\/p>\n<h3>Vague prompts show why the workflow matters<\/h3>\n<p>Vague prompts produced the clearest result. The agent passed 79 of 89 tasks,<br \/>\ncompared with 59 for stock Copilot. Failures fell from 30 to 10.<\/p>\n<p>We used our internal unit-testing benchmark. It contains 152 tasks from real<br \/>\nrepositories. Some prompts are detailed. Others are vague and leave most<br \/>\ndecisions to the agent.<\/p>\n<p>We compared four setups: stock GitHub Copilot, stock Claude Code, our agent in<br \/>\nGitHub Copilot, and our agent in Claude Code. The results below focus on the<br \/>\nGitHub Copilot comparison unless we say otherwise. Both Copilot setups used the<br \/>\nsame model and prompts.<\/p>\n<p>A task passed only when:<\/p>\n<ul>\n<li>The repository built.<\/li>\n<li>All tests passed.<\/li>\n<li>The agent added at least one test.<\/li>\n<li>The agent did not remove any existing tests.<\/li>\n<\/ul>\n<p>The full breakdown shows that detailed prompts were tied:<\/p>\n<table>\n<thead>\n<tr>\n<th>Prompt type<\/th>\n<th>Specialized agent<\/th>\n<th>Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Vague prompts, 89 tasks<\/td>\n<td>79 (88.8%)<\/td>\n<td>59 (66.3%)<\/td>\n<\/tr>\n<tr>\n<td>Detailed prompts, 63 tasks<\/td>\n<td>61 (96.8%)<\/td>\n<td>61 (96.8%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><img data-opt-id=879049612  fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/vague-prompt-results.png\" alt=\"Specialized resolved 88.8% of vague prompts versus 66.3% for stock; both resolved 96.8% of detailed prompts\" \/><\/p>\n<p>That is <strong>67% fewer failures<\/strong> on vague prompts. All 20 net gains came from<br \/>\nthese tasks. This matches the goal of the agent: do the research that the<br \/>\ndeveloper did not put in the prompt.<\/p>\n<p>The same pattern appeared when the prompt pointed to a code change. The<br \/>\nbenchmark has 15 tasks that ask for tests for a specific diff. The agent passed<br \/>\nall 15. Stock Copilot passed none.<\/p>\n<h3>More successful results, not simply more tests<\/h3>\n<p>Both setups ran the same 152 tasks. This lets us compare each result directly.<\/p>\n<p><img data-opt-id=1036130189  data-opt-src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/paired-benchmark-outcomes.png\"  decoding=\"async\" src=\"data:image/svg+xml,%3Csvg%20viewBox%3D%220%200%20100%%20100%%22%20width%3D%22100%%22%20height%3D%22100%%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%3Crect%20width%3D%22100%%22%20height%3D%22100%%22%20fill%3D%22transparent%22%2F%3E%3C%2Fsvg%3E\" alt=\"Paired outcomes: both resolved 119 tasks, specialized only 21, stock only one, and neither 11\" \/><\/p>\n<ul>\n<li>Both passed 119 tasks.<\/li>\n<li>Only the specialized agent passed 21 tasks.<\/li>\n<li>Only stock Copilot passed one task.<\/li>\n<li>Both failed 11 tasks.<\/li>\n<\/ul>\n<table>\n<thead>\n<tr>\n<th>Metric<\/th>\n<th>Specialized agent<\/th>\n<th>Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Tasks completed<\/td>\n<td>140\/152 (92.1%)<\/td>\n<td>120\/152 (78.9%)<\/td>\n<\/tr>\n<tr>\n<td>Solutions building<\/td>\n<td>148<\/td>\n<td>145<\/td>\n<\/tr>\n<tr>\n<td>Final test suites passing<\/td>\n<td>149<\/td>\n<td>147<\/td>\n<\/tr>\n<tr>\n<td>Tests generated<\/td>\n<td>6,963<\/td>\n<td>7,129<\/td>\n<\/tr>\n<tr>\n<td>Average final line coverage<\/td>\n<td>72.4%<\/td>\n<td>72.2%<\/td>\n<\/tr>\n<tr>\n<td>Average final branch coverage<\/td>\n<td>49.8%<\/td>\n<td>49.1%<\/td>\n<\/tr>\n<tr>\n<td>Average task time<\/td>\n<td>359 seconds<\/td>\n<td>380 seconds<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The specialized agent generated 2.3% fewer tests, with effectively the same<br \/>\naverage coverage. It also completed more tasks and was about 5.5% faster on<br \/>\naverage. The gain came from reliability, not from producing more tests.<\/p>\n<h3>The benefit extends across models and languages<\/h3>\n<p>The benchmark contains 45 .NET tasks. They cover several repositories and<br \/>\ndifferent task sizes. We ran the same comparison with three models:<\/p>\n<table>\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Specialized agent<\/th>\n<th>Stock Copilot<\/th>\n<th>Failure reduction<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Claude Opus 4.8<\/td>\n<td>43\/45 (95.6%)<\/td>\n<td>35\/45 (77.8%)<\/td>\n<td>80%<\/td>\n<\/tr>\n<tr>\n<td>GPT-5.5<\/td>\n<td>41\/45 (91.1%)<\/td>\n<td>36\/45 (80.0%)<\/td>\n<td>56%<\/td>\n<\/tr>\n<tr>\n<td>Claude Haiku 4.5<\/td>\n<td>34\/45 (75.6%)<\/td>\n<td>25\/45 (55.6%)<\/td>\n<td>45%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><img data-opt-id=320328365  data-opt-src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/csharp-failures-by-model.png\"  decoding=\"async\" src=\"data:image/svg+xml,%3Csvg%20viewBox%3D%220%200%20100%%20100%%22%20width%3D%22100%%22%20height%3D%22100%%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%3Crect%20width%3D%22100%%22%20height%3D%22100%%22%20fill%3D%22transparent%22%2F%3E%3C%2Fsvg%3E\" alt=\"C# unresolved tasks: specialized versus stock was 2 to 10 on Opus, 4 to 9 on GPT-5.5, and 11 to 20 on Haiku\" \/><\/p>\n<p>The workflow helped every model. With Opus, it added eight wins with no losses.<br \/>\nWith Haiku, it completed nine more C# tasks than stock Haiku.<\/p>\n<p>The result also suggests that a strong workflow can lift a mid-tier model close<br \/>\nto the best result. Across all 152 tasks, specialized GPT-5.5 reached 90.1%.<br \/>\nThat was within two points of specialized Opus and more than 11 points above<br \/>\nstock Opus.<\/p>\n<p>The agent includes guidance for .NET, Python, TypeScript, JavaScript, Java, Go,<br \/>\nRuby, Rust, Swift, Kotlin, PowerShell, and C++. It learns each repository\u2019s<br \/>\nconventions instead of applying C# patterns everywhere.<\/p>\n<p>Beyond .NET, the same Opus run covered several of these languages:<\/p>\n<table>\n<thead>\n<tr>\n<th>Language<\/th>\n<th>Specialized agent<\/th>\n<th>Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Python, 15 tasks<\/td>\n<td>13 (86.7%)<\/td>\n<td>6 (40.0%)<\/td>\n<\/tr>\n<tr>\n<td>Go, 15 tasks<\/td>\n<td>15 (100%)<\/td>\n<td>10 (66.7%)<\/td>\n<\/tr>\n<tr>\n<td>PowerShell, 10 tasks<\/td>\n<td>7 (70.0%)<\/td>\n<td>8 (80.0%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The agent passed every Go task and more than doubled the Python completion<br \/>\nrate. It also passed five Go and five Python tasks that targeted a specific<br \/>\ncode change.<\/p>\n<p>The result was not better in every language. Stock Copilot passed one more<br \/>\nPowerShell task. These groups are also small, so we see them as useful signals,<br \/>\nnot promises for every repository.<\/p>\n<h3>A second, harder benchmark<\/h3>\n<p>We also tested the workflow on 44 unit-test tasks from<br \/>\n<a href=\"https:\/\/github.com\/scaleapi\/SWE-Atlas\">SWE Atlas<\/a>. This benchmark checks<br \/>\nrequirements and uses code changes to see whether the tests can catch bugs.<\/p>\n<table>\n<thead>\n<tr>\n<th>Metric<\/th>\n<th>Specialized agent<\/th>\n<th>Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Tasks completed<\/td>\n<td>16\/44 (36.4%)<\/td>\n<td>12\/44 (27.3%)<\/td>\n<\/tr>\n<tr>\n<td>Wins by only one setup<\/td>\n<td>4<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>Passing generated tests<\/td>\n<td>550<\/td>\n<td>493<\/td>\n<\/tr>\n<tr>\n<td>Tests that caught injected bugs<\/td>\n<td>360<\/td>\n<td>316<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The specialized agent completed four more tasks, with no stock-only wins.<\/p>\n<p>The completion rates are much lower than in our internal benchmark. SWE Atlas<br \/>\nis hard, and there is still a lot to improve.<\/p>\n<h2>What we learned and what comes next<\/h2>\n<p>The clearest gain was reliability. The agent was more likely to deliver unit<br \/>\ntests that built, passed, and matched the request. The difference was largest<br \/>\nwhen the prompt was vague or linked to a code change.<\/p>\n<p>Once both setups produced a valid result, neither led every quality measure.<br \/>\nStock Copilot scored slightly higher on assertions and coverage. The<br \/>\nspecialized agent scored slightly higher on test hygiene, such as clean<br \/>\nstructure and avoiding slow or fragile patterns.<\/p>\n<p>These groups contain different tasks. The specialized group includes harder<br \/>\ntasks that stock Copilot did not complete, so this is not a direct quality<br \/>\nranking. We see these results as guidance for the next improvements: deeper<br \/>\nassertions and error-path tests while keeping strong test hygiene.<\/p>\n<p>Efficiency remained close after accounting for completed tasks. The agent used<br \/>\nabout 3.2% more recorded tokens per completed task. These counts include cached<br \/>\ninput, so they do not directly represent cost.<\/p>\n<p>These results cover unit tests only. We are exploring where the same workflow<br \/>\ncould help with other types of testing, but we have no committed plans to<br \/>\nannounce.<\/p>\n<h2>Try the agent<\/h2>\n<p>The agent, skills, and language guidance are open source in<br \/>\n<a href=\"https:\/\/github.com\/dotnet\/skills\"><code>dotnet\/skills<\/code><\/a>.<\/p>\n<p>You can use the plugin in GitHub Copilot CLI. It is also available in Visual<br \/>\nStudio Code and VS Code Insiders through plugin support, which is still in<br \/>\npreview. We are also working on support for Visual Studio.<\/p>\n<p>Add the marketplace and install the plugin in GitHub Copilot CLI:<\/p>\n<pre><code class=\"language-text\">\/plugin marketplace add dotnet\/skills\r\n\/plugin install dotnet-test@dotnet-agent-skills<\/code><\/pre>\n<p>Restart the CLI. Select <code>code-testing-generator<\/code> from the list of agents. Then<br \/>\ntry the prompt from the start of this post:<\/p>\n<pre><code class=\"language-text\">Generate unit tests.<\/code><\/pre>\n<p>Review the plan, the tests, and the final checks. You can also ask for tests for<br \/>\none function, one module, or a specific coverage goal.<\/p>\n\n<div class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/github.com\/dotnet\/skills\/tree\/main\/plugins\/dotnet-test\" target=\"_blank\">Explore the dotnet-test plugin<\/a><\/div>\n<p>Good tests are not only generated. They are planned, built, run, and checked.<br \/>\nThat is the trust loop we are building.<\/p>\n<p>The post <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/polyglot-unit-testing-agent\/\">From generated code to trusted code with a unit-test agent<\/a> appeared first on <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\">.NET Blog<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>A common request to a coding agent is only one line: Generate unit tests. That request leaves important questions open. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":4702,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-4701","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet"],"_links":{"self":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/4701","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/comments?post=4701"}],"version-history":[{"count":0,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/4701\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media\/4702"}],"wp:attachment":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media?parent=4701"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/categories?post=4701"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/tags?post=4701"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}