{"id":1228,"date":"2024-09-12T11:13:15","date_gmt":"2024-09-12T11:13:15","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/09\/12\/performance-improvements-in-net-9\/"},"modified":"2024-09-12T11:13:15","modified_gmt":"2024-09-12T11:13:15","slug":"performance-improvements-in-net-9","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/09\/12\/performance-improvements-in-net-9\/","title":{"rendered":"Performance Improvements in .NET 9"},"content":{"rendered":"

Each year, summer arrives to find me daunted and excited to write about the performance improvements in the upcoming release of .NET. \u201cDaunted,\u201d because these posts, covering .NET 8<\/a>, .NET 7<\/a>, .NET 6<\/a>, .NET 5<\/a>, .NET Core 3.0<\/a>, .NET Core 2.1<\/a>, and .NET Core 2.0<\/a>, have garnered a bit of a reputation I want to ensure the next iteration lives up to. And \u201cexcited,\u201d because there\u2019s such an abundance of material to cover due to just how much goodness has been packed into the next .NET release, I struggle to get it all written down as quickly as my thoughts whirl.<\/p>\n

And so, every year, I start these posts talking about how the next release of .NET is the fastest and best release to date. That\u2019s true for .NET 9 as well, of course, but the statement that .NET 9 is the fastest and best release of .NET to date is now a bit\u2026 mundane. So, let\u2019s spice it up a bit. How about\u2026 a haiku?<\/p>\n

As the falcon flies,
\n.NET 9 speeds joy into
\nDevelopers’ hearts.<\/p>\n

Or, maybe a limerick:<\/p>\n

In the coding world, there’s a star,
\n.NET 9, the best by far.
\nWith speed that’s supreme,
\nIt’s every coder’s dream,
\nTaking development to a new par.<\/p>\n

A little gimmicky? Maybe something more classical, a sonnet perhaps:<\/p>\n

In realms of code where brilliance finds its way,
\n.NET 9 shines with an unmatched array.
\nIts speed and grace, a marvel to behold,
\nTransforming tasks to treasures, fast and bold.<\/p>\n

Developers, with joy, embrace its might,
\nTheir projects soar, efficiency in sight.
\nNo longer bound by limits of the past,
\nIn .NET 9, their dreams are built to last.<\/p>\n

Its libs, a symphony of pure delight,
\nTurning complex to simple, dim to light.
\nWith every line of code, a masterpiece,
\nIn .NET 9, dev burdens find release.<\/p>\n

Oh, wondrous .NET 9, you light the way,
\nIn your embrace, our future’s bright as day.<\/p>\n

Ok, so, yeah, I should stick to writing software rather than poetry (something with which my college poetry professor likely agreed). Nevertheless, the sentiment remains: .NET 9 is an incredibly exciting release. More than 7,500 pull requests (PRs) have merged into dotnet\/runtime<\/a> in the last year, of which a significant percentage have touched on performance in one way, shape, or form. In this post, we\u2019ll take a tour through over 350 PRs that have all found their way into packing .NET 9 full of performance yumminess. Please grab a large cup of your favorite hot beverage, sit back, settle in, and enjoy.<\/p>\n

Benchmarking Setup<\/h2>\n

In this post, I\u2019ve included micro-benchmarks to showcase various performance improvements. Most of these benchmarks are implemented using BenchmarkDotNet<\/a> v0.14.0<\/a>, and, unless otherwise noted, there is a simple setup for each.<\/p>\n

To follow along, first make sure you have .NET 8<\/a> and .NET 9<\/a> installed. The numbers I share were gathered using the .NET 9 Release Candidate.<\/p>\n

Once you have the appropriate prerequisites installed, create a new C# project in a new benchmarks directory:<\/p>\n

dotnet new console -o benchmarks
\ncd benchmarks<\/p>\n

The resulting directory will contain two files: benchmarks.csproj, which is the project file with information about how the application should be compiled, and Program.cs, which contains the code for the application. Replace the entire contents of benchmarks.csproj with this:<\/p>\n

<Project Sdk=”Microsoft.NET.Sdk”><\/p>\n

<PropertyGroup>
\n <OutputType>Exe<\/OutputType>
\n <TargetFrameworks>net9.0;net8.0<\/TargetFrameworks>
\n <LangVersion>Preview<\/LangVersion>
\n <ImplicitUsings>enable<\/ImplicitUsings>
\n <Nullable>enable<\/Nullable>
\n <AllowUnsafeBlocks>true<\/AllowUnsafeBlocks>
\n <ServerGarbageCollection>true<\/ServerGarbageCollection>
\n <\/PropertyGroup><\/p>\n

<ItemGroup>
\n <PackageReference Include=”BenchmarkDotNet” Version=”0.14.0″ \/>
\n <\/ItemGroup><\/p>\n

<\/Project><\/p>\n

The preceding project file tells the build system we want:<\/p>\n

to build a runnable application, as opposed to a library.
\nto be able to run on both .NET 8 and .NET 9, so that BenchmarkDotNet can build multiple versions of the application, one to run on each version, in order to compare the results.
\nto be able to use all of the latest features from the C# language even though C# 13 hasn\u2019t officially shipped yet.
\nto automatically import common namespaces.
\nto be able to use nullable reference type annotations in the code.
\nto be able to use the unsafe keyword in the code.
\nto configure the garbage collector (GC) into its \u201cserver\u201d configuration, which impacts the trade-offs it makes between memory consumption and throughput. This isn\u2019t required, but it\u2019s how most services are configured.
\nto pull in BenchmarkDotNet v0.14.0 from NuGet so that we\u2019re able to use the library in Program.cs.<\/p>\n

For each benchmark, I\u2019ve then included the full Program.cs source; to test it, just replace the entire contents of your Program.cs with the shown benchmark. Each test may be configured slightly differently from others, in order to highlight the key aspects being shown. For example, some tests include the [MemoryDiagnoser(false)] attribute, which tells BenchmarkDotNet to track allocation-related metrics, or the [DisassemblyDiagnoser] attribute, which tells BenchmarkDotNet to find and share the assembly code for the test, or the [HideColumns] attribute, which removes some output columns that BenchmarkDotNet might otherwise emit but that are unnecessary clutter for our needs in this post.<\/p>\n

Running the benchmarks is then simple. Each test includes a comment at its top for the dotnet command to use to run the benchmark. It\u2019s typically something like this:<\/p>\n

dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

That:<\/p>\n

builds the benchmarks in a Release build. Compiling for Release is important as both the C# compiler and the JIT compiler have optimizations that are disabled for Debug. Thankfully, BenchmarkDotNet warns if Debug is accidentally used:
\n\/\/ Validating benchmarks:
\n\/\/ * Assembly Benchmarks which defines benchmarks is non-optimized
\nBenchmark was built without optimization enabled (most probably a DEBUG configuration). Please, build it in RELEASE.
\nIf you want to debug the benchmarks, please see https:\/\/benchmarkdotnet.org\/articles\/guides\/troubleshooting.html#debugging-benchmarks.<\/p>\n

targets .NET 8 for the host project. There are multiple builds involved here: the \u201chost\u201d application you run with the above command, which uses BenchmarkDotNet, which will in turn generate and build an application per target runtime. Because the code for the benchmark is compiled into all of these, you typically want the host project to target the oldest runtime you\u2019ll be testing, so that building the host application will fail if you try to use an API that\u2019s not available in all of the target runtimes.
\nruns all of the benchmarks in the whole program. If you don\u2019t specify the –filter argument, BenchmarkDotNet will prompt you to ask which benchmarks to run. By specifying \u201c*\u201d, we\u2019re saying \u201cdon\u2019t prompt, just run \u2019em all.\u201d You can also specify an expression to filter down which subset of the tests you want invoked.
\nruns the tests on both .NET 8 and .NET 9.<\/p>\n

Throughout the post, I\u2019ve shown many benchmarks and the results I received from running them. Unless otherwise stated (e.g. because I\u2019m demonstrating an OS-specific improvement), the results shown for benchmarks are from running them on Linux (Ubuntu 22.04) on an x64 processor.<\/p>\n

BenchmarkDotNet v0.14.0, Ubuntu 22.04.3 LTS (Jammy Jellyfish) WSL
\n11th Gen Intel Core i9-11950H 2.60GHz, 1 CPU, 16 logical and 8 physical cores
\n.NET SDK 9.0.100-rc.1.24452.12
\n [Host] : .NET 9.0.0 (9.0.24.43107), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI<\/p>\n

My standard caveat: these are micro-benchmarks, often measuring operations that take very short periods of time, but where improvements to those times add up to be impactful when executed over and over and over. Different hardware, different operating systems, what other processes might be running on your machine, who you had breakfast with this morning, and the alignment of the planets can all impact the numbers you get out. In short, the numbers you see are unlikely to match exactly the numbers I share here; however, I\u2019ve chosen benchmarks that should be broadly repeatable.<\/p>\n

With all that out of the way, let\u2019s do this!<\/p>\n

JIT<\/h2>\n

Improvements in .NET show up at all levels of the stack. Some changes result in large improvements in one specific area. Other changes result in small improvements across many things. When it comes to broad-reaching impact, there are few areas of .NET that result in changes more broadly-impactful than those changes made to the Just In Time (JIT) compiler. Code generation improvements help make everything better, and it\u2019s where we\u2019ll start our journey.<\/p>\n

PGO<\/h3>\n

In Performance Improvements in .NET 8<\/a>, I called out the enabling of dynamic profile guided optimization (PGO) as my favorite feature in the release, so PGO seems like a good place to start for .NET 9.<\/p>\n

As a brief refresher, dynamic PGO is a feature that enables the JIT to profile code and use what it learns from that profiling to help it generate more efficient code based on the exact usage patterns of the application. The JIT utilizes tiered compilation, which allows code to be compiled and then re-compiled, possibly multiple times, achieving something new each time the code is compiled. For example, a typical method might start out at \u201ctier 0,\u201d where the JIT applies very few optimizations and has a goal of simply getting to functional assembly as quickly as possible. This helps with startup performance, as optimizations are one of the most costly things a compiler does. Then the runtime tracks the number of times the method is invoked, and if the number of invocations trips over a particular threshold, such that it seems like performance could actually matter, the JIT will re-generate code for it, still at tier 0, but this time with a bunch of additional instrumentation injected into the method, tracking all manner of things that could help the JIT better optimize, e.g. for a given virtual dispatch, what is the most common type on which the call is being performed. Then after enough data has been gathered, the JIT can compile the method yet again, this time at \u201ctier 1,\u201d fully optimized, also incorporating all of the learnings from that profile data. This same flow is relevant as well for code that\u2019s already been pre-compiled with ReadyToRun (R2R), except instead of instrumenting tier 0 code, the JIT will generate optimized, instrumented code on its way to generating a re-optimized implementation.<\/p>\n

In .NET 8, the JIT in particular paid attention to PGO data about types and methods involved in virtual, interface, and delegate dispatch. In .NET 9, it\u2019s also able to use PGO data to optimize casts. Thanks to dotnet\/runtime#90594<\/a>, dotnet\/runtime#90735<\/a>, dotnet\/runtime#96597<\/a>, dotnet\/runtime#96731<\/a>, and dotnet\/runtime#97773<\/a>, dynamic PGO is now able to track the most common input types to cast operations (castclass\/isinst, e.g. what you get from doing operations like (T)obj or obj is T), and then when generating the optimized code, emit special checks that add fast paths for the most common types. For example, in the following benchmark, we have a field of type A initialized to a type C that\u2019s derived from both B and A. Then the benchmark is type checking the instance stored in that A field to see whether it\u2019s a B or anything derived from B.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private A _obj = new C();<\/p>\n

[Benchmark]
\n public bool IsInstanceOf() => _obj is B;<\/p>\n

public class A { }
\n public class B : A { }
\n public class C : B { }
\n}<\/p>\n

That IsInstanceOf benchmark results in the following disassembly on .NET 8:<\/p>\n

; Tests.IsInstanceOf()
\n push rax
\n mov rsi,[rdi+8]
\n mov rdi,offset MT_Tests+B
\n call qword ptr [7F3D91524360]; System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
\n test rax,rax
\n setne al
\n movzx eax,al
\n add rsp,8
\n ret
\n; Total bytes of code 35<\/p>\n

but now on .NET 9, it produces this:<\/p>\n

; Tests.IsInstanceOf()
\n push rbp
\n mov rbp,rsp
\n mov rsi,[rdi+8]
\n mov rcx,rsi
\n test rcx,rcx
\n je short M00_L00
\n mov rax,offset MT_Tests+C
\n cmp [rcx],rax
\n jne short M00_L01
\nM00_L00:
\n test rcx,rcx
\n setne al
\n movzx eax,al
\n pop rbp
\n ret
\nM00_L01:
\n mov rdi,offset MT_Tests+B
\n call System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
\n mov rcx,rax
\n jmp short M00_L00
\n; Total bytes of code 62<\/p>\n

On .NET 8, it\u2019s loading the reference to the object and the desired method token for B, and calling the CastHelpers.IsInstanceOfClass JIT helper to do the type check. On .NET 9, instead it\u2019s loading the method token for C, which it saw during profiling to be the most common type used, and then comparing that against the actual object\u2019s method token. If they match, since the JIT knows that C derives from B, it then knows the object is in fact a B. If they don\u2019t match, then it jumps down to the fallback path where it does the same thing that was being done on .NET 8, loading the reference and the desired method token for B and calling IsInstanceOfClass.<\/p>\n

It\u2019s also capable of optimizing for the negative case where the cast most often fails. Consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private object _obj = “hello”;<\/p>\n

[Benchmark]
\n public bool IsInstanceOf() => _obj is Tests;
\n}<\/p>\n

On .NET 9, we get this assembly:<\/p>\n

; Tests.IsInstanceOf()
\n push rbp
\n mov rbp,rsp
\n mov rsi,[rdi+8]
\n mov rcx,rsi
\n test rcx,rcx
\n je short M00_L00
\n mov rax,offset MT_System.String
\n cmp [rcx],rax
\n jne short M00_L01
\n xor ecx,ecx
\nM00_L00:
\n test rcx,rcx
\n setne al
\n movzx eax,al
\n pop rbp
\n ret
\nM00_L01:
\n mov rdi,offset MT_Tests
\n call System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
\n mov rcx,rax
\n jmp short M00_L00
\n; Total bytes of code 64<\/p>\n

Here the incoming object is always a string and never the Tests class that\u2019s being tested for. The generated code is comparing the incoming object against string, and then, assuming the types match, the JIT knows the object is not a Tests.<\/p>\n

dotnet\/runtime#96311<\/a> also breaks new ground with dynamic PGO, by teaching it how to profile integers and paying attention to their most common values. Then in conjunction with dotnet\/runtime#96571<\/a>, it uses this super power to optimize Buffer.Memmove (which is the workhorse behind methods like Span<T>.CopyTo) and SpanHelpers.SequenceEqual (which is the implementation behind methods like string.Equals). Previously, the JIT was taught how to unroll such operations, where if a constant length was provided, the JIT could generate the exact code sequence to implement the operation for that length. Now with this capability, the JIT can track the most common lengths provided to these methods, and if there\u2019s one length that really stands out, it can special-case it, unrolling and vectorizing the operation when the length matches and falling back to calling the original when it doesn\u2019t. While this is expected to improve in the future, for .NET 9 this set of length-profiling optimizations only kicks in when R2R is disabled, as the JIT is otherwise unable to do the exact profiling required. Disabling R2R is something services can do when startup performance isn\u2019t a big concern and they instead care about maximum throughput at run-time.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Environments;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Running;<\/p>\n

var config = DefaultConfig.Instance
\n .AddJob(Job.Default.WithId(“.NET 8”).WithRuntime(CoreRuntime.Core80).WithEnvironmentVariable(“DOTNET_ReadyToRun”, “0”))
\n .AddJob(Job.Default.WithId(“.NET 9”).WithRuntime(CoreRuntime.Core90).WithEnvironmentVariable(“DOTNET_ReadyToRun”, “0”));
\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, config);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “a”, “b”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(“abcd”, “abcg”)]
\n public bool Equals(string a, string b) => a == b;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nCode Size<\/p>\n

Equals
\n.NET 8.0
\n2.8592 ns
\n78 B<\/p>\n

Equals
\n.NET 9.0
\n0.6754 ns
\n87 B<\/p>\n

Tier 0<\/h3>\n

Tier 0 is all about getting to functioning code quickly, and as such most optimizations are disabled. However, every now and then there\u2019s a reason to do a bit more optimization in tier 0, in situations where the benefits of doing so outweigh the cons. Several of those occurred in .NET 9.<\/p>\n

dotnet\/runtime#104815<\/a> is a simple example. The ArgumentNullException.ThrowIfNull method is now used in thousands upon thousands of places for doing argument validation. It\u2019s a non-generic method, accepting an object argument and checking to see whether it\u2019s null. That non-genericity causes some friction for folks when it\u2019s used with value types. It\u2019s rare for someone to directly call ThrowIfNull with a value type (other than maybe with a Nullable<T>), and in fact if they do, thanks to dotnet\/roslyn-analyzers<\/a> from @CollinAlpert<\/a>, there\u2019s now the CA2264<\/a> analyzer that will warn that what\u2019s being done is nonsensical:<\/p>\n

Instead, the common case is when the argument being validated is an unconstrained generic. In such cases, if the generic argument ends up being a value type, it\u2019ll be boxed in the call to ThrowIfNull. That boxing allocation gets removed in tier 1, because the ThrowIfNull call gets inlined and the JIT can see at the call site that the boxing was unnecessary. But, because inlining doesn\u2019t happen in tier 0, such boxing has remained in tier 0. As the API is so ubiquitous, this caused developers to fret that there was something bad happening, and it caused enough consternation that the JIT now special-cases ArgumentNullException.ThrowIfNull and avoids the boxing, even in tier 0. This is easy to see with a little test console app:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using System.Runtime.CompilerServices;<\/p>\n

while (true)
\n{
\n Test();
\n}<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\nstatic void Test()
\n{
\n long gc = GC.GetAllocatedBytesForCurrentThread();
\n for (int i = 0; i < 100; i++)
\n {
\n ThrowIfNull(i);
\n }
\n gc = GC.GetAllocatedBytesForCurrentThread() – gc;<\/p>\n

Console.WriteLine(gc);
\n Thread.Sleep(1000);
\n}<\/p>\n

static void ThrowIfNull<T>(T value) => ArgumentNullException.ThrowIfNull(value);<\/p>\n

When I run that on .NET 8, I get results like this:<\/p>\n

2400
\n2400
\n2400
\n0
\n0
\n0<\/p>\n

The first few iterations are invoking Test() at tier 0, such that each call to ArgumentNullException.ThrowIfNull boxes the input int. Then when the method gets recompiled at tier 1, the boxing gets elided, and we stabilize at zero allocation. Now on .NET 9, I get results like this:<\/p>\n

0
\n0
\n0
\n0
\n0
\n0<\/p>\n

With these tweaks to tier 0, the boxing is also elided in tier 0, and so starts out without any allocation.<\/p>\n

Another tier 0 boxing example is dotnet\/runtime#90496<\/a>. There\u2019s a hot path method in the async\/await machinery: AsyncTaskMethodBuilder<TResult>.AwaitUnsafeOnCompleted (see How Async\/Await Really Works in C#<\/a> for all the details). It\u2019s really important that this method be optimized well, but it performs various type tests that can end up boxing in tier 0. In a previous release, that boxing was deemed too impactful to startup for async methods invoked early in an application\u2019s lifetime, so [MethodImpl(MethodImplOptions.AggressiveOptimization)] was used to opt the method out of tiering, such that it gets optimized from the get-go. But that itself has downsides, because if it skips tiering up, it also skips dynamic PGO, and thus the optimized code isn\u2019t as good as it possibly could be. So, this PR specifically addresses those type tests patterns that box, removing the boxing in tier 0, enabling removing that AggressiveOptimization from AwaitUnsafeOnCompleted, and thereby enabling better optimized code generation for it.<\/p>\n

Optimizations are avoided in tier 0 because they might slow down compilation. If there are really cheap optimizations, though, and they can have a meaningful impact, they can be worth enabling. That\u2019s especially true if the optimizations can actually help to make compilations and startup faster, such as by minimizing calls to helpers that may take locks, trigger certain kinds of loading, etc. And that\u2019s what dotnet\/runtime#105190<\/a> does, enabling some constant folding in tier 0 at relatively little cost. Even with the low cost, though, there were still concerns about possible impact to JIT throughput, and the PR was fast-followed by dotnet\/runtime#105250<\/a> which optimized some JIT code paths to make up for any impact from the former change.<\/p>\n

Another similar case is dotnet\/runtime#91403<\/a> from @MichalPetryka<\/a>, which allows optimizations around RuntimeHelpers.CreateSpan to kick in for tier 0. Without that, the runtime can end up allocating many field stubs, which themselves add overhead to the startup path.<\/p>\n

Loops<\/h3>\n

Applications spend a lot of time iterating through loops, and finding ways to reduce the overheads of loops has been a key focus for .NET 9. It\u2019s also been quite successful.<\/p>\n

dotnet\/runtime#102261<\/a> and dotnet\/runtime#103181<\/a> help to remove some instructions from even the tightest of loops by converting upward counting loops into downward counting loops. Consider a loop like the following:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public int UpwardCounting()
\n {
\n int count = 0;
\n for (int i = 0; i < 100; i++)
\n {
\n count++;
\n }
\n return count;
\n }
\n}<\/p>\n

Here\u2019s what the generated assembly code for that core loop looks like on .NET 8:<\/p>\n

M00_L00:
\n inc eax
\n inc ecx
\n cmp ecx,64
\n jl short M00_L00<\/p>\n

It\u2019s incrementing eax, which is storing count. And it\u2019s incrementing ecx, which is storing i. It\u2019s then comparing ecx against 100 (0x64) to see if it\u2019s reached the end of the loop, and jumping back up to the beginning of the loop if it hasn\u2019t.<\/p>\n

Now let\u2019s manually rewrite the loop to be downward counting:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public int DownwardCounting()
\n {
\n int count = 0;
\n for (int i = 99; i >= 0; i–)
\n {
\n count++;
\n }
\n return count;
\n }
\n}<\/p>\n

And here\u2019s what the generated assembly code for that core loop looks like there:<\/p>\n

M00_L00:
\n inc eax
\n dec ecx
\n jns short M00_L00<\/p>\n

The key observation here is that by counting down, we can replace a cmp\/jl for comparing against a specific bound to instead just be a jns that jumps if the value isn\u2019t negative. We\u2019ve thus removed an instruction from a tight loop that only had four to begin with. With the aforementioned PRs, the JIT can now do that transformation automatically where it\u2019s applicable and deemed valuable, such that the loop in UpwardCounting now results in the same assembly code on .NET 9 as does the loop in DownwardCounting.<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

UpwardCounting
\n.NET 8.0
\n30.27 ns
\n1.00<\/p>\n

UpwardCounting
\n.NET 9.0
\n26.52 ns
\n0.88<\/p>\n

However, the JIT is only able to do this transformation if the iteration variable (i) isn\u2019t used in the body of the loop, and obviously there are many loops where it is, such as by indexing into an array being iterated over. Thankfully, other optimizations in .NET 9 are able to reduce the actual reliance on the iteration variable, such that this optimization now kicks in frequently.<\/p>\n

One such optimization is strength reduction in loops. In compilers, \u201cstrength reduction\u201d is the simple idea of taking something relatively expensive and replacing it with something cheaper. In the context of loops, that typically means introducing more \u201cinduction variables\u201d (variables whose values change in a predictable pattern on each iteration, such as being incremented by a constant amount). For example, consider a simple loop that sums all of the elements of an array:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _array = Enumerable.Range(0, 1000).ToArray();<\/p>\n

[Benchmark]
\n public int Sum()
\n {
\n int[] array = _array;
\n int sum = 0;<\/p>\n

for (int i = 0; i < array.Length; i++)
\n {
\n sum += array[i];
\n }<\/p>\n

return sum;
\n }
\n}<\/p>\n

We get the following assembly on .NET 8:<\/p>\n

; Tests.Sum()
\n push rbp
\n mov rbp,rsp
\n mov rax,[rdi+8]
\n xor ecx,ecx
\n xor edx,edx
\n mov edi,[rax+8]
\n test edi,edi
\n jle short M00_L01
\nM00_L00:
\n mov esi,edx
\n add ecx,[rax+rsi*4+10]
\n inc edx
\n cmp edi,edx
\n jg short M00_L00
\nM00_L01:
\n mov eax,ecx
\n pop rbp
\n ret
\n; Total bytes of code 35<\/p>\n

The interesting part is the loop starting at M00_L00. i is being stored in edx (though it gets copied into esi), and as part of adding the next element from the array to sum (which is stored in ecx), we\u2019re loading that next value from the array with the address rax+rsi*4+10. A strength reduction view of this would say \u201crather than re-computing the address on each iteration, we can instead have another induction variable and increment it by 4 on each iteration.\u201d A key benefit of that is it then removes a dependency on i from inside of the loop, which then means the iteration variable is no longer used in the loop, enabling the aforementioned downward counting optimization to kick in. That leads to the following assembly on .NET 9:<\/p>\n

; Tests.Sum()
\n push rbp
\n mov rbp,rsp
\n mov rax,[rdi+8]
\n xor ecx,ecx
\n mov edx,[rax+8]
\n test edx,edx
\n jle short M00_L01
\n add rax,10
\nM00_L00:
\n add ecx,[rax]
\n add rax,4
\n dec edx
\n jne short M00_L00
\nM00_L01:
\n mov eax,ecx
\n pop rbp
\n ret
\n; Total bytes of code 35<\/p>\n

Note the loop at M00_L00: it\u2019s now downward counting, reading the next value from the array is simply dereferencing the address in rax, and the address in rax is incremented by 4 each go around.<\/p>\n

A lot of work went into enabling this strength reduction, including providing the basic implementation (dotnet\/runtime#104243<\/a>), enabling it by default (dotnet\/runtime#105131<\/a>), finding more opportunities to apply it (dotnet\/runtime#105169<\/a>), and using it to enable post-indexed addressing (dotnet\/runtime#105181<\/a> and dotnet\/runtime#105185<\/a>), which is an Arm addressing mode where the address stored in the base register is used but then that register is updated to point to the next target memory location. A new phase was also added to the JIT to help with optimizing such induction variables (dotnet\/runtime#97865<\/a>), and in particular, to do induction variable widening where 32-bit induction variables (think of every loop you\u2019ve ever written that starts with for (int i = …)) are widened to 64-bit induction variables. This widening can help to avoid zero extensions that might otherwise occur on every iteration of the loop.<\/p>\n

These optimizations are all new, but of course there are also many loop optimizations already present in the JIT compiler, from loop unrolling to loop cloning to loop hoisting. In order to apply such loop optimizations, though, the JIT first needs to recognize loops, and that can sometimes be more challenging than it would seem (dotnet\/runtime#43713<\/a> describes a case where the JIT was failing to do so). Historically, the JIT\u2019s loop recognition was based on a relatively simplistic lexical analysis. In .NET 8, as part of the work to improve dynamic PGO, a more powerful graph-based loop analyzer was added that was able to recognize many more loops. For .NET 9 with dotnet\/runtime#95251<\/a>, that analyzer was factored out so that it could be used for generalized loop reasoning. And then with PRs like dotnet\/runtime#96756<\/a> for loop alignment, dotnet\/runtime#96754<\/a> and dotnet\/runtime#96553<\/a> for loop cloning, dotnet\/runtime#96752<\/a> for loop unrolling, dotnet\/runtime#96751<\/a> for loop canonicalization, and dotnet\/runtime#96753<\/a> for loop hoisting, many of these loop-related optimizations have now been moved to the better scheme. All of that means that more loops get optimized.<\/p>\n

Bounds Checks<\/h3>\n

.NET code is, by default, \u201cmemory safe.\u201d Unlike in C, where you can iterate through an array and easily walk off the end of it, by default accesses to arrays, strings, and spans are \u201cbounds checked\u201d to ensure you can\u2019t walk off the end or before the beginning. Of course, such bounds checking adds overhead, and so wherever the JIT can prove that adding such checks would be unnecessary, it\u2019ll elide the bounds check, knowing that it\u2019s impossible for the guarded accesses to be problematic. The quintessential example of this is a loop over an array from 0 to array.Length. Let\u2019s look at the same benchmark we just looked at, summing all the elements of an integer array:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _array = new int[1000];<\/p>\n

[Benchmark]
\n public int Test()
\n {
\n int[] array = _array;<\/p>\n

int sum = 0;
\n for (int i = 0; i < array.Length; i++)
\n {
\n sum += array[i];
\n }
\n return sum;
\n }
\n}<\/p>\n

That Test benchmark results in this assembly code on .NET 8:<\/p>\n

; Tests.Test()
\n push rbp
\n mov rbp,rsp
\n mov rax,[rdi+8]
\n xor ecx,ecx
\n xor edx,edx
\n mov edi,[rax+8]
\n test edi,edi
\n jle short M00_L01
\nM00_L00:
\n mov esi,edx
\n add ecx,[rax+rsi*4+10]
\n inc edx
\n cmp edi,edx
\n jg short M00_L00
\nM00_L01:
\n mov eax,ecx
\n pop rbp
\n ret
\n; Total bytes of code 35<\/p>\n

The key part to pay attention to is the loop at M00_L00, for which the only branch is the one comparing edx (which is tracking i) to edi (which was earlier on initialized to the length of the array, [rax+8]) as part of knowing when it\u2019s done iterating. There\u2019s no additional check required to make this safe, as the JIT knows the loop started at 0 (and thus isn\u2019t walking off the beginning of the array) and the JIT knows iteration ends at the array length, which the JIT is already checking for, so it\u2019s safe to index into the array without additional checks.<\/p>\n

Now, let\u2019s tweak the benchmark ever so slightly. In the above, I was copying the _array field to a local array and then doing all accesses against that array; this is critical, because there\u2019s nothing else that could be changing that local out from under the loop. But if we instead change the code to refer to the field directly:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _array = new int[1000];<\/p>\n

[Benchmark]
\n public int Test()
\n {
\n int sum = 0;
\n for (int i = 0; i < _array.Length; i++)
\n {
\n sum += _array[i];
\n }
\n return sum;
\n }
\n}<\/p>\n

now we get this on .NET 8:<\/p>\n

; Tests.Test()
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\n xor ecx,ecx
\n mov rdx,[rdi+8]
\n cmp dword ptr [rdx+8],0
\n jle short M00_L01
\n nop dword ptr [rax]
\n nop dword ptr [rax]
\nM00_L00:
\n mov rdi,rdx
\n cmp ecx,[rdi+8]
\n jae short M00_L02
\n mov esi,ecx
\n add eax,[rdi+rsi*4+10]
\n inc ecx
\n cmp [rdx+8],ecx
\n jg short M00_L00
\nM00_L01:
\n pop rbp
\n ret
\nM00_L02:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 61<\/p>\n

That\u2019s a whole lot worse. Note how much that loop starting at M00_L00 has grown, and in particular note that instead of just having the one cmp\/jg pair at the end, there\u2019s another cmp\/jae pair in the middle, just before it accesses the array element. Since the code is reading from the field on every access, the JIT needs to accommodate the fact that the reference could change between any two accesses; thus, even though the JIT is comparing against _array.Length as part of the loop bounds, it also needs to ensure that the subsequent reference to _array[i] is still in bounds, since by then _array may be an entirely different object. That\u2019s a \u201cbounds check,\u201d which is obvious from the tell-tale sign that immediately after the cmp, there\u2019s a conditional jump to code that unconditionally calls CORINFO_HELP_RNGCHKFAIL; that\u2019s the helper function that\u2019s called to throw an IndexOutOfRangeException when you try to walk off the end of one of these data structures.<\/p>\n

Every release the JIT gets better at removing more and more bounds checks where it can prove they\u2019re superfluous. One of my favorite such improvements in .NET 9 is there on my favorites list because I\u2019ve historically expected the optimization to \u201cjust work\u201d, for various reasons it didn\u2019t, and now it does (it also shows up in a fair amount of real code, which is why I\u2019ve bumped up against it). In this benchmark, the function is handed an offset and a span, and its job is to sum all of the numbers from that offset to the end of the span.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(3)]
\n public int Test() => M(0, “1234567890abcdefghijklmnopqrstuvwxyz”);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n public static int M(int i, ReadOnlySpan<char> src)
\n {
\n int sum = 0;<\/p>\n

while (true)
\n {
\n if ((uint)i >= src.Length)
\n {
\n break;
\n }<\/p>\n

sum += src[i++];
\n }<\/p>\n

return sum;
\n }
\n}<\/p>\n

By casting i to uint as part of the comparison to src.Length, the JIT knows that i is in bounds of src by the time i is used to index into src, because if i were negative, the cast to uint would have made it larger than int.MaxValue and thus also larger than src.Length (which can\u2019t possibly be larger than int.MaxValue). The .NET 8 assembly shows the bounds check has been elided (note the lack of CORINFO_HELP_RNGCHKFAIL):<\/p>\n

; Tests.M(Int32, System.ReadOnlySpan`1<Char>)
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\nM01_L00:
\n cmp edi,edx
\n jae short M01_L01
\n lea ecx,[rdi+1]
\n mov edi,edi
\n movzx edi,word ptr [rsi+rdi*2]
\n add eax,edi
\n mov edi,ecx
\n jmp short M01_L00
\nM01_L01:
\n pop rbp
\n ret
\n; Total bytes of code 27<\/p>\n

But, this is a fairly awkward way to write such a condition. A more natural way would be to have that check as part of the loop condition:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(3)]
\n public int Test() => M(0, “1234567890abcdefghijklmnopqrstuvwxyz”);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n public static int M(int i, ReadOnlySpan<char> src)
\n {
\n int sum = 0;<\/p>\n

for (; (uint)i < src.Length; i++)
\n {
\n sum += src[i];
\n }<\/p>\n

return sum;
\n }
\n}<\/p>\n

Unfortunately, as a result of my code cleanup here to make the code more canonical, the JIT in .NET 8 fails to see that the bounds check can be elided\u2026 note the CORINFO_HELP_RNGCHKFAIL at the end:<\/p>\n

; Tests.M(Int32, System.ReadOnlySpan`1<Char>)
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\n cmp edi,edx
\n jae short M01_L01
\nM01_L00:
\n cmp edi,edx
\n jae short M01_L02
\n mov ecx,edi
\n movzx ecx,word ptr [rsi+rcx*2]
\n add eax,ecx
\n inc edi
\n cmp edi,edx
\n jb short M01_L00
\nM01_L01:
\n pop rbp
\n ret
\nM01_L02:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 36<\/p>\n

But in .NET 9, thanks to dotnet\/runtime#100777<\/a>, the JIT is better able to track the knowledge about guarantees made by the loop condition and is able to elide the bounds check on this variation as well.<\/p>\n

; Tests.M(Int32, System.ReadOnlySpan`1<Char>)
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\n cmp edi,edx
\n jae short M01_L01
\n mov ecx,edi
\nM01_L00:
\n movzx edi,word ptr [rsi+rcx*2]
\n add eax,edi
\n inc ecx
\n cmp ecx,edx
\n jb short M01_L00
\nM01_L01:
\n pop rbp
\n ret
\n; Total bytes of code 26<\/p>\n

Yay!<\/p>\n

Now consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(3)]
\n public int Test(int i)
\n {
\n ReadOnlySpan<byte> rva = [1, 2, 3, 5, 8, 13, 21, 34];
\n return rva[7 – (i & 7)];
\n }
\n}<\/p>\n

The test method here has a span of data initialized in a way where the JIT is able to see how long it is. It\u2019s then indexing into the span, using the supplied index to read not from the start but from the end (the (i & 7) is there to ensure the JIT can see that the value will always be in range); if it were reading from the start, this was already optimized, but from the end, the JIT hadn\u2019t previously been taught how to reason about the bounds checks. On .NET 8, it can\u2019t prove the access is always in-bounds, and we can see the bounds check in place:<\/p>\n

; Tests.Test(Int32)
\n push rax
\n and esi,7
\n mov eax,esi
\n neg eax
\n add eax,7
\n cmp eax,8
\n jae short M00_L00
\n mov rcx,7FC98A741EC8
\n movzx eax,byte ptr [rax+rcx]
\n add rsp,8
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 41<\/p>\n

But, now on .NET 9, thanks to dotnet\/runtime#96123<\/a>, the bounds check gets elided.<\/p>\n

; Tests.Test(Int32)
\n and esi,7
\n mov eax,esi
\n neg eax
\n add eax,7
\n mov rcx,7F39B8724EC8
\n movzx eax,byte ptr [rax+rcx]
\n ret
\n; Total bytes of code 25<\/p>\n

Here\u2019s another case. We\u2019re special-casing spans of lengths less than or equal to 1, returning string.Empty if the span is of length 0 or returning the first string if the span is of length 1:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(3)]
\n public string? Test() => M([“123”]);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static string? M(ReadOnlySpan<string> values)
\n {
\n if (values.Length <= 1)
\n {
\n return values.Length == 0 ?
\n string.Empty :
\n values[0];
\n }<\/p>\n

return null;
\n }
\n}<\/p>\n

You and I can see that the access to values[0] will always succeed, but on .NET 8 we get this:<\/p>\n

; Tests.M(System.ReadOnlySpan`1<System.String>)
\n push rbp
\n mov rbp,rsp
\n cmp esi,1
\n jg short M01_L01
\n test esi,esi
\n je short M01_L00
\n test esi,esi
\n je short M01_L02
\n mov rax,[rdi]
\n pop rbp
\n ret
\nM01_L00:
\n mov rax,7FB62147C008
\n pop rbp
\n ret
\nM01_L01:
\n xor eax,eax
\n pop rbp
\n ret
\nM01_L02:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 44<\/p>\n

The JIT keeps track of what it knows about the lengths of various things, what conditions it\u2019s proved, but here it\u2019s lost track of the fact that, for the else branch of the ternary, values is guaranteed to be of length 1, and thus indexing at index 0 is safe. dotnet\/runtime#101323<\/a> improves the JIT\u2019s range tracking ability, such that on .NET 9, the bounds check is successfully elided:<\/p>\n

; Tests.M(System.ReadOnlySpan`1<System.String>)
\n push rbp
\n mov rbp,rsp
\n cmp esi,1
\n jg short M01_L01
\n test esi,esi
\n je short M01_L00
\n mov rax,[rdi]
\n pop rbp
\n ret
\nM01_L00:
\n mov rax,7F5700FB1008
\n pop rbp
\n ret
\nM01_L01:
\n xor eax,eax
\n pop rbp
\n ret
\n; Total bytes of code 34<\/p>\n

Most if not all of these bounds check elimination improvements come about because someone is optimizing something and sees a bounds check that could have been eliminated but wasn\u2019t. In the case that inspired the improvement in dotnet\/runtime#101352<\/a>, that someone was me, while working on improving Enum for .NET 8. Enums can be backed by various numerical types, including ulong, and there\u2019s a code path in Enum.GetName that\u2019s effectively this:<\/p>\n

if (ulongValue < (ulong)names.Length)
\n{
\n return names[(uint)ulongValue];
\n}<\/p>\n

That bounds check wasn\u2019t previously being removed, but now in .NET 9, it is:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private readonly string[] _names = Enum.GetNames<MyEnum>();<\/p>\n

[Benchmark]
\n [Arguments(2)]
\n public string? GetNameOrNull(ulong ulValue)
\n {
\n string[] names = _names;
\n return ulValue < (ulong)names.Length ?
\n names[(uint)ulValue] :
\n null;
\n }<\/p>\n

public enum MyEnum : ulong { A, B, C, D }
\n}
\n\/\/ .NET 8
\n; Tests.GetNameOrNull(UInt64)
\n push rbp
\n mov rbp,rsp
\n mov rax,[rdi+8]
\n mov ecx,[rax+8]
\n mov edx,ecx
\n cmp rdx,rsi
\n jbe short M00_L00
\n cmp esi,ecx
\n jae short M00_L01
\n mov ecx,esi
\n mov rax,[rax+rcx*8+10]
\n pop rbp
\n ret
\nM00_L00:
\n xor eax,eax
\n pop rbp
\n ret
\nM00_L01:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 41<\/p>\n

\/\/ .NET 9
\n; Tests.GetNameOrNull(UInt64)
\n mov rax,[rdi+8]
\n mov ecx,[rax+8]
\n cmp rcx,rsi
\n jbe short M00_L00
\n mov ecx,esi
\n mov rax,[rax+rcx*8+10]
\n ret
\nM00_L00:
\n xor eax,eax
\n ret
\n; Total bytes of code 23<\/p>\n

Sometimes eliding bounds checks is about learning new tricks; other times, it\u2019s about fixing old ones. Consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static ReadOnlySpan<int> Lookup => [1, 2, 3, 5, 8, 13, 21];<\/p>\n

[Benchmark]
\n [Arguments(3)]
\n public int Test1(int i) => (uint)i < 7 ? Lookup[i] : -1;<\/p>\n

[Benchmark]
\n [Arguments(3)]
\n public int Test2(int i) => (uint)i <= 6 ? Lookup[i] : -1;
\n}<\/p>\n

Test1 and Test2 are effectively the same thing, both guarding a lookup table by a known length and only accessing the table if we know the index to be in bounds. The bounds check will then be elided by the JIT in both cases, right? Wrong. On .NET 8, we get this:<\/p>\n

; Tests.Test1(Int32)
\n cmp esi,7
\n jae short M00_L00
\n mov eax,esi
\n mov rcx,7F6D40064030
\n mov eax,[rcx+rax*4]
\n ret
\nM00_L00:
\n mov eax,0FFFFFFFF
\n ret
\n; Total bytes of code 27<\/p>\n

; Tests.Test2(Int32)
\n push rbp
\n mov rbp,rsp
\n cmp esi,6
\n ja short M00_L00
\n cmp esi,7
\n jae short M00_L01
\n mov eax,esi
\n mov rcx,7F8D11621030
\n mov eax,[rcx+rax*4]
\n pop rbp
\n ret
\nM00_L00:
\n mov eax,0FFFFFFFF
\n pop rbp
\n ret
\nM00_L01:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 44<\/p>\n

Note the bounds check in Test2. dotnet\/runtime#97908<\/a> fixes this, such that on .NET 9 Test2 now successfully elides the bounds check as well:<\/p>\n

; Tests.Test1(Int32)
\n cmp esi,7
\n jae short M00_L00
\n mov eax,esi
\n mov rcx,7F5B9DC5E030
\n mov eax,[rcx+rax*4]
\n ret
\nM00_L00:
\n mov eax,0FFFFFFFF
\n ret
\n; Total bytes of code 27<\/p>\n

; Tests.Test2(Int32)
\n cmp esi,6
\n ja short M00_L00
\n mov eax,esi
\n mov rcx,7F7FDE2C9030
\n mov eax,[rcx+rax*4]
\n ret
\nM00_L00:
\n mov eax,0FFFFFFFF
\n ret
\n; Total bytes of code 27<\/p>\n

Interestingly, sometimes even if we can\u2019t elide a bounds check, we can learn things from the fact that one occurred, and then use that knowledge to optimize subsequent things. Consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private readonly int[] _x = new int[10];<\/p>\n

[Benchmark]
\n [Arguments(2)]
\n public int Add(int y) => _x[y] + (y % 8);
\n}<\/p>\n

There\u2019s nothing the JIT can do here to elide the bounds check on _x[y]; it has no information about the value of y or the length of _x. As such, as shown in the .NET 8 assembly here, we see a bounds check:<\/p>\n

; Tests.Add(Int32)
\n push rax
\n mov rax,[rdi+8]
\n cmp esi,[rax+8]
\n jae short M00_L00
\n mov ecx,esi
\n mov edx,esi
\n sar edx,1F
\n and edx,7
\n add edx,esi
\n and edx,0FFFFFFF8
\n mov edi,esi
\n sub edi,edx
\n add edi,[rax+rcx*4+10]
\n mov eax,edi
\n add rsp,8
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 46<\/p>\n

However, all is not lost. After indexing into the array, we proceed to use y as the numerator of a % operation. C#\u2019s % operator supports both int and uint numerators, but it has to do a little more work for int in case the value is negative. However, by the time we get to that % operation, we know<\/em> that y is not negative, as if it were negative, the _x[y] would have thrown and we\u2019d never end up here. dotnet\/runtime#102089<\/a> teaches the JIT how to learn such non-negative information from such bounds checks, such that in .NET 9, we get code generation equivalent to if we\u2019d explicitly cast y to uint.<\/p>\n

; Tests.Add(Int32)
\n push rax
\n mov rax,[rdi+8]
\n cmp esi,[rax+8]
\n jae short M00_L00
\n mov ecx,esi
\n and esi,7
\n add esi,[rax+rcx*4+10]
\n mov eax,esi
\n add rsp,8
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 32<\/p>\n

Arm64<\/h3>\n

Making .NET on Arm an awesome and fast experience has been a critical, multi-year investment. You can read about it in Arm64 Performance Improvements in .NET 5<\/a>, Arm64 Performance Improvements in .NET 7<\/a>, and Arm64 Performance Improvements in .NET 8<\/a>. And things continue to improve even further in .NET 9. Here are some examples:<\/p>\n

Better barriers.<\/strong> dotnet\/runtime#91553<\/a> implements volatile writes via using the stlur (Store-Release Register) instruction rather than a dmb (Data Memory Barrier) \/ str (Store) pair of instructions (stlur is generally cheaper). Similarly, dotnet\/runtime#101359<\/a> eliminates full memory barriers when dealing with volatile reads and writes on floats. For example, code that would previously have produced a ldr (Load Register) \/ dmb pair may now produce a ldar (Load-Acquire Register) \/ fmov (Floating-point Move) pair.
\nBetter switches.<\/strong> Depending on the shape of a switch statement, the C# compiler may generate a variety of IL patterns, one of which is to use a switch IL instruction. Normally for a switch IL instruction, the JIT will generate a jump table, but for some forms, it has an optimization to instead rely on a bit test. Thus far this optimization only existed for x86\/64, with the bt (Bit Test) instruction. Now with dotnet\/runtime#91811<\/a>, it also exists for Arm, with the tbz (Test bit and Branch if Zero) instruction.
\nBetter conditionals.<\/strong> Arm has conditional instructions that logically contain a branch albeit without any branching, e.g. Performance Improvements in .NET 8<\/a> talked about the csel (Conditional Select) instruction that \u201cconditionally selects\u201d a value from one of two registers based on some condition. Another such instruction is csinc (Conditional Select Increment), which conditionally selects either the value from one register or the value from another register incremented by one. dotnet\/runtime#91262<\/a> from @c272<\/a> enables the JIT to utilize csinc, so that a statement like x = condition ? x + 1 : y; will be able to compile down to a csinc rather than to a branching construct. dotnet\/runtime#92810<\/a> also improves the custom comparison operation the JIT emits for some SequenceEqual operations (e.g. “hello, there”u8.SequenceEqual(spanOfBytes)) to be able to use ccmp (Conditional Compare).
\nBetter multiplies.<\/strong> Arm has single instructions that represent doing a multiply followed by an addition, subtraction, or negation. dotnet\/runtime#91886<\/a> from @c272<\/a> finds such sequences of multiplies followed by one of those operations and consolidates them to use the single combined instruction.
\nBetter loads.<\/strong> Arm has instructions for loading a value from memory into a single register, but it also has instructions for loading multiple values into multiple registers. When the JIT emits a customized memory copy (such as for byteArray.AsSpan(0, 32).SequenceEqual(otherByteArray)), it may emit multiple ldr instructions for loading a value into a register. dotnet\/runtime#92704<\/a> enables consolidating pairs of those into ldp (Load Pair of Registers) instructions, which load two values into two registers.<\/p>\n

ARM SVE<\/h3>\n

Bringing up a new instruction set is a huge deal and a huge undertaking. I\u2019ve mentioned in the past my process for gearing up to write one of these \u201cPerformance Improvements in .NET X\u201d posts, including that throughout the year I keep a running list of the PRs I might want to talk about when it comes time to actually put pen to paper. Just for \u201cSVE\u201d, I found myself with over 200 links. I\u2019m not going to bore you with such a laundry list; if you\u2019re interested, you can search for SVE PRs<\/a>, which includes PRs from @a74nh<\/a>, from @ebepho<\/a>, from @mikabl-arm<\/a>, from @snickolls-arm<\/a>, and from @SwapnilGaikwad<\/a>. But, we can still talk a bit about what it is and what it means for .NET.<\/p>\n

Single instruction, multiple data (SIMD) is a kind of parallel processing where one instruction performs the same operation on multiple pieces of data at the same time, rather than one instruction manipulating just a single piece of data. For example, the add instruction on x86\/64 can add together one pair of 32-bit integers, whereas the paddd (Add Packed Doubleword Integers) instruction that\u2019s part of Intel\u2019s SSE2 (Streaming SIMD Extensions 2) instruction set operates on a pair of xmm registers that can each store four 32-bit integer values at once. Many such instructions have been added to many different hardware platforms over the years, coming in groups referred to as instruction set architectures (ISA), where an ISA defines what the instructions are, what registers they interact with, how memory is accessed, and so on. Even if you\u2019re not steeped in this stuff, you\u2019ve likely heard names of these ISAs mentioned, like Intel\u2019s SSE (Streaming SIMD Extensions) and AVX (Advanced Vector Extensions), or Arm\u2019s Advanced SIMD (also known as Neon). In general, the instructions in all of these ISAs operate on a fixed number of values of a fixed size, e.g. the paddd previously mentioned only works with 128-bits at a time, no more, no less. Different instructions exist for 256 bits at a time or 512 bits at a time.<\/p>\n

SVE, or \u201cScalable Vector Extensions,\u201d is an ISA from Arm that\u2019s a bit different. The instructions in SVE don\u2019t operate on a fixed size. Rather, the specification allows for them to operate on sizes from 128 bits up to 2048 bits, and the specific hardware can choose which size to use (allowed sizes are multiples of 128, and with SVE 2 further constrained to be powers of 2). The same assembly code using these instructions might operate on 128 bits at a time on one piece of hardware and 256 bits at a time on another piece of hardware.<\/p>\n

There are multiple ways such an ISA impacts .NET, and in particular the JIT. The JIT needs to be able to be able to work with the ISA, understand the associated registers and be able to do register allocation, be taught about encoding and emitting the instructions, and so on. The JIT needs to be taught when and where it\u2019s appropriate to use these instructions, so that as part of compiling IL down to assembly, if operating on a machine that supports SVE, the JIT might be able to pick SVE instructions for use in the generated assembly. And the JIT needs to be taught how to represent this data, these vectors, to user code. All of that is a huge amount of work, especially when you consider that there are thousands of operations represented. What makes it even more work is hardware intrinsics.<\/p>\n

Hardware intrinsics<\/a> are a feature of .NET where, effectively, each of these instructions shows up as its own dedicated .NET method, such as Sse2.Add<\/a>, and the JIT emits use of that method as the underlying instruction to which it maps. If you look at Sve.cs<\/a> in dotnet\/runtime, you\u2019ll see the System.Runtime.Intrinsics.Arm.Sve type, which already exposes more than 1400 public methods (that number is not a typo).<\/p>\n

Two interesting things to notice if you open that file (beyond its sheer length):<\/p>\n

The use of Vector<T>.<\/strong> .NET\u2019s foray into SIMD started in 2014<\/a> and was accompanied by the Vector<T> type. Vector<T> represents a single vector (list) of the T numeric type. To provide a platform-agnostic representation, since different platforms were capable of different vector widths, Vector<T> was defined to be variable in size, so for example on x86\/x64 hardware that supported AVX2, Vector<T> might be 256 bits wide, whereas on an Arm machine that supported Neon, Vector<T> might be 128 bits wide. If the hardware supported both 128 bits and 256 bits, Vector<T> would map to the larger. Since the introduction of Vector<T>, various fixed-width vector types have been introduced, like Vector64<T>, Vector128<T>, Vector256<T>, and Vector512<T>, and the hardware intrinsics for most of the other ISAs are all in terms of those fixed-width vector sizes, since the instructions themselves are fixed width. But SVE is not; its instructions might be 128 bits here and 512 bits there, thus it\u2019s not possible to use those same fixed-width vector types in the Sve definition\u2026 but it makes a lot of sense to use the variable with Vector<T>. What\u2019s old is new again.
\nThe Sve class is tagged as [Experimental].<\/strong> The [Experimental]<\/a> attribute was introduced in .NET 8 and C# 12. The intent is it can be used to indicate that some functionality in an otherwise stable assembly is not yet stable and may change in the future. If code tries to use such a member, by default the C# compiler will issue an error telling the developer they\u2019re using something that could break in the future. As long as the developer is willing to accept such breaking change risk, they can then suppress the error. Designing and enabling the SVE support is a monstrous, multi-year effort, and while the support is functional and folks are encouraged to take it for a spin, it\u2019s not yet baked enough for us to be 100% confident the shape won\u2019t need to evolve (for .NET 9, it\u2019s also restricted to hardware with a vector width of 128 bits, but that restriction will be removed subsequently). Hence, [Experimental].<\/p>\n

AVX10.1<\/h3>\n

Even with the size of the SVE effort, it\u2019s not the only new ISA available in .NET 9. Thanks in large part to dotnet\/runtime#99784<\/a> from @Ruihan-Yin<\/a> and dotnet\/runtime#101938<\/a> from @khushal1996<\/a>, .NET 9 now also supports AVX10.1 (AVX10 version 1). AVX10.1 provides everything AVX512 provides, all of the base support, the updated encodings, support for embedded broadcasts, masking, and so on, but it only requires 256-bit support in the hardware (with 512-bits being optional, whereas AVX512 requires 512-bit support), and it does so in a much less incremental manner (AVX512 has multiple instruction sets like \u201cF\u201d, \u201cDQ\u201d, \u201cVbmi\u201d, etc.). That\u2019s modeled in the .NET APIs as well, where you can check Avx10v1.IsSupported as well as Avx10v1.V512.IsSupported, both of which govern more than 500 new APIs available for consumption. (Note that at the time of this writing, there aren\u2019t actually any chips on the market that support AVX10.1, but they\u2019re expected in the foreseeable future.)<\/p>\n

AVX512<\/h3>\n

On the subject of ISAs, it\u2019s worth mentioning AVX512. .NET 8 added broad support for AVX512, including support in the JIT and employment of it throughout the libraries. Both of those improve further in .NET 9. We\u2019ll talk more about places it\u2019s better used in the libraries later. For now, here are some JIT-specific improvements.<\/p>\n

One of the things the JIT needs to generate code for is zeroing, e.g. by default all locals in a method need to be set to zero, and even if [SkipLocalsInit] is employed, references still need to be zeroed (otherwise, when the GC does a pass through all of the locals looking for references to objects to see what\u2019s no longer referenced, it could see the references as being whatever garbage happened to be in that location in memory and end up making bad choices). Such zeroing of locals is overhead that occurs on every invocation of that method, so obviously it\u2019s valuable for that to be as efficient as possible. Rather than zeroing out each word with a single instruction, if the current hardware supports the appropriate SIMD instructions, the JIT can instead emit code to use those instructions, so that it can zero out more per instruction. With dotnet\/runtime#91166<\/a>, it\u2019s now able to use AVX512 instructions if available to zero out 512 bits per instruction, rather than \u201conly\u201d 256 bits or 128 bits using other ISAs. As an example, here\u2019s a benchmark that needs to zero out 256 bytes:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;
\nusing System.Runtime.InteropServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic unsafe class Tests
\n{
\n [Benchmark]
\n public void Sum()
\n {
\n Bytes values;
\n Nop(&values);
\n }<\/p>\n

[SkipLocalsInit]
\n [Benchmark]
\n public void SumSkipLocalsInit()
\n {
\n Bytes values;
\n Nop(&values);
\n }<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static void Nop(Bytes* value) { }<\/p>\n

[StructLayout(LayoutKind.Sequential, Size = 256)]
\n private struct Bytes { }
\n}<\/p>\n

Here\u2019s the assembly for Sum on .NET 8:<\/p>\n

; Tests.Sum()
\n sub rsp,108
\n xor eax,eax
\n mov [rsp+8],rax
\n vxorps xmm8,xmm8,xmm8
\n mov rax,0FFFFFFFFFFFFFF10
\nM00_L00:
\n vmovdqa xmmword ptr [rsp+rax+100],xmm8
\n vmovdqa xmmword ptr [rsp+rax+110],xmm8
\n vmovdqa xmmword ptr [rsp+rax+120],xmm8
\n add rax,30
\n jne short M00_L00
\n mov [rsp+100],rax
\n lea rdi,[rsp+8]
\n call qword ptr [7F6B56B85CB0]; Tests.Nop(Bytes*)
\n nop
\n add rsp,108
\n ret
\n; Total bytes of code 90<\/p>\n

This is on a machine with AVX512 hardware support, but we can see the zero\u2019ing is happening using a loop (M00_L00 through to the jne that jumps back to it), as with only 256-bit instructions, this was deemed by the JIT\u2019s heuristics too large to unroll completely. Now, here\u2019s .NET 9:<\/p>\n

; Tests.Sum()
\n sub rsp,108
\n xor eax,eax
\n mov [rsp+8],rax
\n vxorps xmm8,xmm8,xmm8
\n vmovdqu32 [rsp+10],zmm8
\n vmovdqu32 [rsp+50],zmm8
\n vmovdqu32 [rsp+90],zmm8
\n vmovdqa xmmword ptr [rsp+0D0],xmm8
\n vmovdqa xmmword ptr [rsp+0E0],xmm8
\n vmovdqa xmmword ptr [rsp+0F0],xmm8
\n mov [rsp+100],rax
\n lea rdi,[rsp+8]
\n call qword ptr [7F4D3D3A44C8]; Tests.Nop(Bytes*)
\n nop
\n add rsp,108
\n ret
\n; Total bytes of code 107<\/p>\n

Now there\u2019s no loop, because vmovdqu32 (Move unaligned packed doubleword integer values) can be used to zero twice as much at a time (64 bytes) as vmovdqa (Move aligned packed integer values), and thus the zeroing can be done in fewer instructions that\u2019s still considered a reasonable number.<\/p>\n

Zeroing also shows up elsewhere, such as when initializing structs. Those have also previously employed SIMD instructions where relevant, e.g. this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public MyStruct Init() => new();<\/p>\n

public struct MyStruct
\n {
\n public Int128 A, B, C, D;
\n }
\n}<\/p>\n

produces this assembly today on .NET 8:<\/p>\n

; Tests.Init()
\n vzeroupper
\n vxorps ymm0,ymm0,ymm0
\n vmovdqu32 [rsi],zmm0
\n mov rax,rsi
\n ret
\n; Total bytes of code 17<\/p>\n

But, if we tweak MyStruct to add a field of a reference type anywhere in the struct (e.g. add public string Oops; as the first line of the struct above), it knocks the initialization off this optimized path, and we end up with initialization like this on .NET 8:<\/p>\n

; Tests.Init()
\n xor eax,eax
\n mov [rsi],rax
\n mov [rsi+8],rax
\n mov [rsi+10],rax
\n mov [rsi+18],rax
\n mov [rsi+20],rax
\n mov [rsi+28],rax
\n mov [rsi+30],rax
\n mov [rsi+38],rax
\n mov [rsi+40],rax
\n mov [rsi+48],rax
\n mov rax,rsi
\n ret
\n; Total bytes of code 45<\/p>\n

This is due to alignment requirements in order to provide necessary atomicity guarantees. But rather than giving up wholesale, dotnet\/runtime#102132<\/a> allows the SIMD zeroing to be used for the contiguous portions that don\u2019t contain GC references, so now on .NET 9 we get this:<\/p>\n

; Tests.Init()
\n xor eax,eax
\n mov [rsi],rax
\n vxorps xmm0,xmm0,xmm0
\n vmovdqu32 [rsi+8],zmm0
\n mov [rsi+48],rax
\n mov rax,rsi
\n ret
\n; Total bytes of code 27<\/p>\n

This optimization isn\u2019t specific to AVX512, but it includes the ability to use AVX512 instructions when available. (dotnet\/runtime#99140<\/a> provides similar support for Arm64.)<\/p>\n

Other optimizations improve the JIT\u2019s ability to select AVX512 instructions as part of generating code. One neat example of this is dotnet\/runtime#91227<\/a> from @Ruihan-Yin<\/a>, which utilizes the cool vpternlog (Bitwise Ternary Logic) instruction. Imagine you have three bools (a, b, and c), and you want to perform a series of Boolean operations on them, e.g. a ? (b ^ c) : (b & c). If you were to naively compile that down, you\u2019d end up with branches. We could make it branchless by distributing the a to both sides of the ternary, e.g. (a & (b ^ c)) | (!a & (b & c)), but now we\u2019ve gone from one branch and one Boolean operation to six Boolean operations. What if instead we could do all of that in a single instruction and<\/em> do it for all of the lanes in a vector at the same time so it could be applied to multiple values as part of a SIMD operation? That\u2019d be cool, right? That\u2019s what vpternlog enables. Try running this:<\/p>\n

\/\/ dotnet run -c Release -f net9.0<\/p>\n

internal class Program
\n{
\n private static bool Exp(bool a, bool b, bool c) => (a & (b ^ c)) | (!a & b & c);<\/p>\n

private static void Main()
\n {
\n Console.WriteLine(“a b c result”);
\n Console.WriteLine(“————“);
\n int control = 0;
\n foreach (var (a, b, c, result) in from a in new[] { true, false }
\n from b in new[] { true, false }
\n from c in new[] { true, false }
\n select (a, b, c, Exp(a, b, c)))
\n {
\n Console.WriteLine($”{Convert.ToInt32(a)} {Convert.ToInt32(b)} {Convert.ToInt32(c)} {Convert.ToInt32(result)}”);
\n control = control << 1 | Convert.ToInt32(result);
\n }
\n Console.WriteLine(“————“);
\n Console.WriteLine($”Control: {control:b8} == 0x{control:X2}”);
\n }
\n}<\/p>\n

Here we\u2019ve put our Boolean operation into an Exp function, which is then being invoked for all 8 possible combinations of inputs (each of the three bools each having two possible values). We\u2019re then printing out the resulting \u201ctruth table,\u201d that details the Boolean output for each possible input. With this particular Boolean expression, that yields this truth table being output:<\/p>\n

a b c result
\n————
\n1 1 1 0
\n1 1 0 1
\n1 0 1 1
\n1 0 0 0
\n0 1 1 1
\n0 1 0 0
\n0 0 1 0
\n0 0 0 0
\n————<\/p>\n

We then take that last result column and we treat it as a binary number:<\/p>\n

Control: 01101000 == 0x68<\/p>\n

So the values are 0 1 1 0 1 0 0 0, which we read as the binary 0b01101000, which is 0x68. That byte is used as a \u201ccontrol code\u201d to the vpternlog instruction to encode which of the 256 possible truth tables that exist for any possible (deterministic) Boolean combination of those inputs is being chosen. This PR then teaches the JIT how to analyze the tree structures produced by the JIT to recognize such sequences of Boolean operations, compute the control code, and substitute in the use of the better instruction. Of course, the JIT isn\u2019t going to do the enumeration I did above; turns out there\u2019s a more efficient way to compute the control code, performing the same sequence of operations but on specific byte values instead of Booleans, e.g. this:<\/p>\n

\/\/ dotnet run -c Release -f net9.0<\/p>\n

Console.WriteLine($”0x{Exp(0xF0, 0xCC, 0xAA):X2}”);
\nstatic int Exp(int a, int b, int c) => (a & (b ^ c)) | (~a & b & c);<\/p>\n

also yields:<\/p>\n

0x68<\/p>\n

Why those specific three values of 0xF0, 0xCC, and 0xAA? Let\u2019s expand them from hex to binary: 0b11110000, 0b11001100, 0b10101010. Look familiar? They\u2019re the columns for a, b, and c in the earlier truth table, so we\u2019re really just running this expression over each of the 8 rows in the table at the same time. Fun.<\/p>\n

Another neat example is in dotnet\/runtime#92017<\/a> from @Ruihan-Yin<\/a>, which optimizes 512-bit vector constants via broadcast. \u201cbroadcast\u201d is a fancy way of saying \u201creplicate,\u201d or \u201ccopy to each.\u201d The instruction is used to take a single value and duplicate it to be used for each element of a vector. If, for example, I write:<\/p>\n

Vector512<int> vector = Vector512.Create(42);<\/p>\n

that\u2019s broadcasting the single value 42, replicating it 16 times to fill up the 512-bit vector. Now imagine I have the following C# code, which is creating a Vector512<byte> composed of the byte sequence for the hex digits, but manually replicated four times, to fill up the 64 bytes that compose a 512-bit vector.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.Intrinsics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public Vector512<byte> HexLookupTable() =>
\n Vector512.Create(“0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef”u8);
\n}<\/p>\n

This would result in that whole byte sequence being stored in the assembly data section, and then the JIT would emit the code to load that data into the appropriate registers; no broadcasting. But instead, the JIT should be able to recognize that this is actually the same 16-byte sequence repeated four times, store the sequence once, and then use a broadcast to load and replicate that value to fill out the vector. With this PR, that\u2019s exactly what happens.<\/p>\n

\/\/ .NET 8
\n; Tests.HexLookupTable()
\n push rax
\n vzeroupper
\n vmovups zmm0,[7FB205399700]
\n vmovups [rsi],zmm0
\n mov rax,rsi
\n vzeroupper
\n add rsp,8
\n ret
\n; Total bytes of code 31<\/p>\n

\/\/ .NET 9
\n; Tests.HexLookupTable()
\n push rax
\n vbroadcasti32x4 zmm0,xmmword ptr [7F78F75290F0]
\n vmovups [rsi],zmm0
\n mov rax,rsi
\n vzeroupper
\n add rsp,8
\n ret
\n; Total bytes of code 28<\/p>\n

This is beneficial for a variety of reasons, including less data to store, less data to load, and if the register containing this state needed to be spilled (meaning something else needs to be put into the register, so the value currently in the register is temporarily stored in memory), reloading it is similarly cheaper.<\/p>\n

Two of the more far-reaching changes related to AVX512, though, come from dotnet\/runtime#97675<\/a> and dotnet\/runtime#101886<\/a>, which do the work to enable the JIT to utilize AVX512 \u201cembedded masking.\u201d Masking is a commonly needed solution when writing SIMD code; anywhere you see a ConditionalSelect, that\u2019s masking. Consider again a ternary operation, e.g. a ? (b + c) : (b – c). Here, a would be considered the \u201cmask\u201d: anywhere it\u2019s true, the value of b + c is used, and anywhere it\u2019s false, the value of b – c is used. If each of these were Vector512<byte>, for example, it would look like this in C#:<\/p>\n

public static Vector512<byte> Exp(Vector512<byte> a, Vector512<byte> b, Vector512<byte> c) =>
\n Vector512.ConditionalSelect(a, b + c, b – c);<\/p>\n

And guess what I\u2019d get for assembly? You guessed it, our good friend vpternlogd:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;
\nusing System.Runtime.Intrinsics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public Vector512<byte> Test() => Exp(default, default, default);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n public static Vector512<byte> Exp(Vector512<byte> a, Vector512<byte> b, Vector512<byte> c) =>
\n Vector512.ConditionalSelect(a, b + c, b – c);
\n}
\n; Tests.Exp(System.Runtime.Intrinsics.Vector512`1<Byte>, System.Runtime.Intrinsics.Vector512`1<Byte>, System.Runtime.Intrinsics.Vector512`1<Byte>)
\n vzeroupper
\n vmovups zmm0,[rsp+48]
\n vmovups zmm1,[rsp+88]
\n vpaddb zmm2,zmm0,zmm1
\n vpsubb zmm0,zmm0,zmm1
\n vmovups zmm1,[rsp+8]
\n vpternlogd zmm1,zmm2,zmm0,0CA
\n vmovups [rdi],zmm1
\n mov rax,rdi
\n vzeroupper
\n ret
\n; Total bytes of code 68<\/p>\n

We can see it\u2019s computing both the b + c (vpaddb zmm2,zmm0,zmm1) and the b – c (vpsubb zmm0,zmm0,zmm1), and it\u2019s then choosing between them based on the mask ([rsp+8], aka the a parameter). In this example, the mask a was being passed in and computed in a manner unknown to the ConditionalSelect. A more common scheme, however, is that the mask is computed as an argument to the ConditionalSelect. Let\u2019s say for example that instead of passing in a as a mask, we pass in Vector512.LessThan(b, c) as the mask:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;
\nusing System.Runtime.Intrinsics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public Vector512<byte> Test() => Exp(default, default);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n public static Vector512<byte> Exp(Vector512<byte> b, Vector512<byte> c) =>
\n Vector512.ConditionalSelect(Vector512.LessThan(b, c), b + c, b – c);
\n}<\/p>\n

AVX512 supports this implicitly via embedded masking, which means that instructions can include the masking operation as part of them rather than performing the operation separately and then doing the masking via vpternlogd. Instructions like the comparison operation employed by LessThan can target storing the result into a new kind of register defined by AVX512, a mask register, and then that mask register can be used as part of other compound operations to incorporate the mask into them. .NET developers don\u2019t need to do anything to take advantage of this support, though: the JIT just uses the specialized masking instructions where it sees an opportunity to do so. For the previous example, on .NET 8, we\u2019d get this:<\/p>\n

; Tests.Exp(System.Runtime.Intrinsics.Vector512`1<Byte>, System.Runtime.Intrinsics.Vector512`1<Byte>)
\n vzeroupper
\n vmovups zmm0,[rsp+8]
\n vmovups zmm1,[rsp+48]
\n vpcmpltub k1,zmm0,zmm1
\n vpmovm2b zmm2,k1
\n vpaddb zmm3,zmm0,zmm1
\n vpsubb zmm0,zmm0,zmm1
\n vpternlogd zmm2,zmm3,zmm0,0CA
\n vmovups [rdi],zmm2
\n mov rax,rdi
\n vzeroupper
\n ret
\n; Total bytes of code 70<\/p>\n

Here we still have a vpternlogd. But, with the aforementioned PRs, now here\u2019s what we get on .NET 9:<\/p>\n

; Tests.Exp(System.Runtime.Intrinsics.Vector512`1<Byte>, System.Runtime.Intrinsics.Vector512`1<Byte>)
\n vmovups zmm0,[rsp+8]
\n vmovups zmm1,[rsp+48]
\n vpcmpltub k1,zmm0,zmm1
\n vpsubb zmm2,zmm0,zmm1
\n vpaddb zmm2{k1},zmm0,zmm1
\n vmovups [rdi],zmm2
\n mov rax,rdi
\n vzeroupper
\n ret
\n; Total bytes of code 54<\/p>\n

That vpcmpltub instruction is doing the LessThan between b and c and storing the result as a mask in the k1 masking register. The vpsubbfor the b – c still happens as it did before. But now the b + c operation is significantly different, and note there\u2019s no vpternlogd anymore. The vpternlogd and the vpaddb we previously saw have now effectively been folded into a single vpaddb instruction with<\/em> the mask. The result of the b – c is sitting in the zmm2 register. The vpaddb instruction then performs the addition between zmm0 (b) and zmm1 (c), and uses the mask k1 to decide whether to use that addition result or the existing subtraction result in zmm2. (dotnet\/runtime#97468<\/a> also enables some such usage of vpternlogd to instead use vblendmps. vblendmps is similar to vpternlogd except that it\u2019s specific to floating-point and works with one of the dedicated mask registers.)<\/p>\n

dotnet\/runtime#97529<\/a> also improved casting from double and float to integer types, in particular when AVX512 is available such that it can benefit from dedicated AVX512 instructions for the purpose, e.g. the VCVTTSD2USI (Convert With Truncation Scalar Double Precision Floating-Point Value to Unsigned Integer) instruction.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Linq;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private double[] _doubles = Enumerable.Range(0, 1024).Select(i => (double)i).ToArray();
\n private ulong[] _ulongs = new ulong[1024];<\/p>\n

[Benchmark]
\n public void DoubleToUlong()
\n {
\n ReadOnlySpan doubles = _doubles;
\n Span ulongs = _ulongs;
\n for (int i = 0; i < doubles.Length; i++)
\n {
\n ulongs[i] = (ulong)doubles[i];
\n }
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size<\/p>\n

DoubleToUlong
\n.NET 8.0
\n1,386.5 ns
\n1.00
\n135 B<\/p>\n

DoubleToUlong
\n.NET 9.0
\n461.4 ns
\n0.33
\n102 B<\/p>\n

Vectorization<\/h3>\n

In addition to improvements that teach the JIT about entirely new architectures, there have also been a plethora of improvements that simply help the JIT to better employ SIMD in general.<\/p>\n

One of my favorites is dotnet\/runtime#92852<\/a>, which merges consecutive stores into a single operation. Consider wanting to implement a method like bool.TryFormat:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private bool _value;
\n private char[] _destination = new char[10];<\/p>\n

[Benchmark]
\n public bool TryFormat() => TryFormat(_destination, out _);<\/p>\n

private bool TryFormat(char[] destination, out int charsWritten)
\n {
\n if (_value)
\n {
\n if (destination.Length >= 4)
\n {
\n destination[0] = ‘T’;
\n destination[1] = ‘r’;
\n destination[2] = ‘u’;
\n destination[3] = ‘e’;
\n charsWritten = 4;
\n return true;<\/p>\n

}
\n }
\n else
\n {
\n if (destination.Length >= 5)
\n {
\n destination[0] = ‘F’;
\n destination[1] = ‘a’;
\n destination[2] = ‘l’;
\n destination[3] = ‘s’;
\n destination[4] = ‘e’;
\n charsWritten = 5;
\n return true;
\n }
\n }<\/p>\n

charsWritten = 0;
\n return false;
\n }
\n}<\/p>\n

Pretty simple: we\u2019re writing out each individual value. That\u2019s a bit unfortunate, though, in that we\u2019re naively then spending several movs to write each character individually, when instead we could pack all of those values into a single value to write. In fact, that\u2019s exactly what the real bool.TryFormat does. Here is its handling of the true case today:<\/p>\n

if (destination.Length > 3)
\n{
\n ulong true_val = BitConverter.IsLittleEndian ? 0x65007500720054ul : 0x54007200750065ul; \/\/ “True”
\n MemoryMarshal.Write(MemoryMarshal.AsBytes(destination), in true_val);
\n charsWritten = 4;
\n return true;
\n}<\/p>\n

The developer has manually done the work of computing the value of the merged writes, e.g.<\/p>\n

ulong true_val = (((ulong)’e’ << 48) | ((ulong)’u’ << 32) | ((ulong)’r’ << 16) | (ulong)’T’)
\nAssert.Equal(0x65007500720054ul, true_val);<\/p>\n

in order to be able to perform a single write rather than doing four individual ones. For this particular case, now in .NET 9, the JIT can automatically do this merging so the developer doesn\u2019t have to. The developer just writes the code that\u2019s natural to write, and the JIT does the heavy lifting of optimizing its output (note below the mov rax, 65007500720054 instruction, loading the same value we manually computed above).<\/p>\n

\/\/ .NET 8
\n; Tests.TryFormat(Char[], Int32 ByRef)
\n push rbp
\n mov rbp,rsp
\n cmp byte ptr [rdi+10],0
\n jne short M01_L01
\n mov ecx,[rsi+8]
\n cmp ecx,5
\n jl short M01_L00
\n mov word ptr [rsi+10],46
\n mov word ptr [rsi+12],61
\n mov word ptr [rsi+14],6C
\n mov word ptr [rsi+16],73
\n mov word ptr [rsi+18],65
\n mov dword ptr [rdx],5
\n mov eax,1
\n pop rbp
\n ret
\nM01_L00:
\n xor eax,eax
\n mov [rdx],eax
\n pop rbp
\n ret
\nM01_L01:
\n mov ecx,[rsi+8]
\n cmp ecx,4
\n jl short M01_L00
\n mov word ptr [rsi+10],54
\n mov word ptr [rsi+12],72
\n mov word ptr [rsi+14],75
\n mov word ptr [rsi+16],65
\n mov dword ptr [rdx],4
\n mov eax,1
\n pop rbp
\n ret
\n; Total bytes of code 112<\/p>\n

\/\/ .NET 9
\n; Tests.TryFormat(Char[], Int32 ByRef)
\n push rbp
\n mov rbp,rsp
\n cmp byte ptr [rdi+10],0
\n jne short M01_L00
\n mov ecx,[rsi+8]
\n cmp ecx,5
\n jl short M01_L01
\n mov rax,73006C00610046
\n mov [rsi+10],rax
\n mov word ptr [rsi+18],65
\n mov dword ptr [rdx],5
\n mov eax,1
\n pop rbp
\n ret
\nM01_L00:
\n mov ecx,[rsi+8]
\n cmp ecx,4
\n jl short M01_L01
\n mov rax,65007500720054
\n mov [rsi+10],rax
\n mov dword ptr [rdx],4
\n mov eax,1
\n pop rbp
\n ret
\nM01_L01:
\n xor eax,eax
\n mov [rdx],eax
\n pop rbp
\n ret
\n; Total bytes of code 92<\/p>\n

dotnet\/runtime#92939<\/a> improves this further by enabling longer sequences to similarly be merged using SIMD instructions.<\/p>\n

Of course, you may then wonder, why wasn\u2019t bool.TryFormat reverted to use the simpler code? The unfortunate answer is that this optimization only currently applies to array targets rather than span targets. That\u2019s because there are alignment requirements for performing these kinds of writes, and whereas the JIT can make certain assumptions about the alignment of arrays, it can\u2019t make those same assumptions about spans, which can represent slices of something else at unaligned boundaries. This is now one of the few cases where arrays are better than spans; typically span is as good or better. But I\u2019m hopeful it will be improved in the future.<\/p>\n

Another nice improvement is dotnet\/runtime#86811<\/a> from @BladeWise<\/a>, which adds SIMD support for multiplying two vectors of bytes or sbytes. Previously this would end up falling back to a software implementation, which is very slow compared to true SIMD operations. Now, the code is much faster and much more compact.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.Intrinsics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Vector128<byte> _v1 = Vector128.Create((byte)0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15);<\/p>\n

[Benchmark]
\n public Vector128<byte> Square() => _v1 * _v1;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Square
\n.NET 8.0
\n15.4731 ns
\n1.000<\/p>\n

Square
\n.NET 9.0
\n0.0284 ns
\n0.002<\/p>\n

dotnet\/runtime#103555<\/a> (x64, when AVX512 isn\u2019t available) and dotnet\/runtime#104177<\/a> (Arm64) also improve vector multiplication, this time for long\/ulong. This can be seen with a simple micro-benchmark (and because I\u2019m running on a machine that supports AVX512, the benchmark is explicitly disabling it):<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Environments;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.Intrinsics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, DefaultConfig.Instance
\n .AddJob(Job.Default.WithId(“.NET 8”).WithRuntime(CoreRuntime.Core80).WithEnvironmentVariable(“DOTNET_EnableAVX512F”, “0”).AsBaseline())
\n .AddJob(Job.Default.WithId(“.NET 9”).WithRuntime(CoreRuntime.Core90).WithEnvironmentVariable(“DOTNET_EnableAVX512F”, “0”)));<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Vector256<long> _a = Vector256.Create(1, 2, 3, 4);
\n private Vector256<long> _b = Vector256.Create(5, 6, 7, 8);<\/p>\n

[Benchmark]
\n public Vector256<long> Multiply() => Vector256.Multiply(_a, _b);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Multiply
\n.NET 8.0
\n9.5448 ns
\n1.00<\/p>\n

Multiply
\n.NET 9.0
\n0.3868 ns
\n0.04<\/p>\n

It\u2019s also evident, however, on higher-level benchmarks, for example on this benchmark for XxHash128, an implementation that makes heavy use of multiplication of such vectors.<\/p>\n

\/\/ Add a <PackageReference Include=”System.IO.Hashing” Version=”8.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Environments;
\nusing System.IO.Hashing;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, DefaultConfig.Instance
\n .AddJob(Job.Default.WithId(“.NET 8”).WithRuntime(CoreRuntime.Core80).WithEnvironmentVariable(“DOTNET_EnableAVX512F”, “0”).AsBaseline())
\n .AddJob(Job.Default.WithId(“.NET 9”).WithRuntime(CoreRuntime.Core90).WithEnvironmentVariable(“DOTNET_EnableAVX512F”, “0”)));<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _data;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _data = new byte[1024 * 1024];
\n new Random(42).NextBytes(_data);
\n }<\/p>\n

[Benchmark]
\n public UInt128 Hash() => XxHash128.HashToUInt128(_data);
\n}<\/p>\n

This benchmark references the System.IO.Hashing nuget package. Note that we\u2019re explicitly adding in a reference to the 8.0.0 version; that means that even when running on .NET 9, we\u2019re using the .NET 8 version of the hashing code, yet it\u2019s still significantly faster, because of these runtime improvements.<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Hash
\n.NET 8.0
\n40.49 us
\n1.00<\/p>\n

Hash
\n.NET 9.0
\n26.40 us
\n0.65<\/p>\n

Some other notable examples:<\/p>\n

Improved SIMD comparisons.<\/strong> dotnet\/runtime#104944<\/a> and dotnet\/runtime#104215<\/a> improve how vector comparisons are handled.
\nImproved ConditionalSelects.<\/strong> dotnet\/runtime#104092<\/a> from @ezhevita<\/a> improves the generated code for ConditionalSelects when the condition is a set of constants.
\nBetter Const Handling.<\/strong> Certain operations are only optimized when one of their arguments is a constant, otherwise falling back to a much slower software emulation implementation. dotnet\/runtime#102827<\/a> enables such instructions (like for shuffling) to continue to be treated as optimized operations if the non-const argument becomes a constant as part of other optimizations (like inlining).
\nUnblocking other optimizations.<\/strong> Some changes don\u2019t themselves introduce optimizations, but instead make tweaks that enable other optimizations to do a better job. dotnet\/runtime#104517<\/a> decomposes some bitwise operations (e.g. replacing a unified \u201cand not\u201d operation with an \u201cand\u201d and a \u201cnot\u201d), which in turn enables other existing optimizations like common sub-expression elimination (CSE) to kick in more often. And dotnet\/runtime#104214<\/a> normalized various negation patterns, which similarly enables other optimizations to apply in more places.<\/p>\n

Branching<\/h3>\n

Just like the JIT tries to elide redundant bounds checking, where it can prove the bounds check is unnecessary, it similarly does so for branching.<\/p>\n

The ability to handle the relationships between branches is improved in .NET 9. Consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(50)]
\n public void Test(int x)
\n {
\n if (x > 100)
\n {
\n Helper(x);
\n }
\n }<\/p>\n

private void Helper(int x)
\n {
\n if (x > 10)
\n {
\n Console.WriteLine(“Hello!”);
\n }
\n }
\n}<\/p>\n

The Helper function is simple enough to be inlined, and in .NET 8 we end up with this assembly:<\/p>\n

; Tests.Test(Int32)
\n push rbp
\n mov rbp,rsp
\n cmp esi,64
\n jg short M00_L01
\nM00_L00:
\n pop rbp
\n ret
\nM00_L01:
\n cmp esi,0A
\n jle short M00_L00
\n mov rdi,7F35E44C7E18
\n pop rbp
\n jmp qword ptr [7F35E914C7C8]
\n; Total bytes of code 33<\/p>\n

We can see in the original code that the branch within the inlined Helper is entirely unnecessary: we\u2019re only there if x is greater than 100, so it\u2019s definitely greater than 10, yet in the assembly code, we have both comparisons happening (notice the two cmps). Now in .NET 9, thanks to dotnet\/runtime#95234<\/a> which improves the JIT\u2019s ability to reason about the relationship between two ranges and whether one is implied by the other, we get this instead:<\/p>\n

; Tests.Test(Int32)
\n cmp esi,64
\n jg short M00_L00
\n ret
\nM00_L00:
\n mov rdi,7F81C120EE20
\n jmp qword ptr [7F8148626628]
\n; Total bytes of code 22<\/p>\n

Just the one outer cmp. The same thing happens for the negative case: if we tweak the x > 10 to instead be x < 10, we end up with this:<\/p>\n

\/\/ .NET 8
\n; Tests.Test(Int32)
\n push rbp
\n mov rbp,rsp
\n cmp esi,64
\n jg short M00_L01
\nM00_L00:
\n pop rbp
\n ret
\nM00_L01:
\n cmp esi,0A
\n jge short M00_L00
\n mov rdi,7F6138428DE0
\n pop rbp
\n jmp qword ptr [7FA1DDD4C7C8]
\n; Total bytes of code 33<\/p>\n

\/\/ .NET 9
\n; Tests.Test(Int32)
\n ret
\n; Total bytes of code 1<\/p>\n

Similar to the x > 10 case, on .NET 8 the JIT retained both branches. But on .NET 9, it recognized that not only was the inner conditional redundant, it was redundant in a way that would make it always false, which then allowed it to dead-code eliminate the body of that if, leaving the whole method a nop. dotnet\/runtime#94689<\/a> makes this kind of information flow by enabling the JIT\u2019s support for \u201ccross-block local assertion prop\u201d.<\/p>\n

Another PR that eliminated some redundant branches is dotnet\/runtime#94563<\/a>, which feeds information from value numbering (a technique used to eliminate redundant expressions by giving every unique expression its own unique identifier) into the building of PHIs (a kind of node in the JIT\u2019s intermediate representation of the code that aids in determining a variable\u2019s value based on control flow). Consider this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic unsafe class Tests
\n{
\n [Benchmark]
\n [Arguments(50)]
\n public void Test(int x)
\n {
\n byte[] data = new byte[128];
\n fixed (byte* ptr = data)
\n {
\n Nop(ptr);
\n }
\n }<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static void Nop(byte* ptr) { }
\n}<\/p>\n

This is allocating a byte[] and then pinning it in order to use it with a method that requires a pointer. The C# specification for fixed with arrays states \u201cIf the array expression is null or if the array has zero elements, the initializer computes an address equal to zero,\u201d and as such if you look at the IL for this code, you\u2019ll see that it\u2019s checking the length and setting the pointer equal to 0 if the length is 0. You can see this same behavior explicitly implemented as well for spans if you look at Span<T>\u2018s GetPinnableReference implementation:<\/p>\n

public ref T GetPinnableReference()
\n{
\n ref T ret = ref Unsafe.NullRef<T>();
\n if (_length != 0) ret = ref _reference;
\n return ref ret;
\n}<\/p>\n

As such, there\u2019s actually an extra branch not visible in the Tests.Test test. But, in this particular case, that branch is also redundant, because we can very clearly see (and the JIT should be able to as well) that the length of the array is non-0. On .NET 8, we still get that branch:<\/p>\n

; Tests.Test(Int32)
\n push rbp
\n sub rsp,10
\n lea rbp,[rsp+10]
\n xor eax,eax
\n mov [rbp-8],rax
\n mov rdi,offset MT_System.Byte[]
\n mov esi,80
\n call CORINFO_HELP_NEWARR_1_VC
\n mov [rbp-8],rax
\n mov rdi,[rbp-8]
\n cmp dword ptr [rdi+8],0
\n je short M00_L01
\n mov rdi,[rbp-8]
\n cmp dword ptr [rdi+8],0
\n jbe short M00_L02
\n mov rdi,[rbp-8]
\n add rdi,10
\nM00_L00:
\n call qword ptr [7F3F99B45C98]; Tests.Nop(Byte*)
\n xor eax,eax
\n mov [rbp-8],rax
\n add rsp,10
\n pop rbp
\n ret
\nM00_L01:
\n xor edi,edi
\n jmp short M00_L00
\nM00_L02:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 96<\/p>\n

But now on .NET 9, that branch (in fact, multiple redundant branches) is removed:<\/p>\n

; Tests.Test(Int32)
\n push rax
\n xor eax,eax
\n mov [rsp],rax
\n mov rdi,offset MT_System.Byte[]
\n mov esi,80
\n call CORINFO_HELP_NEWARR_1_VC
\n mov [rsp],rax
\n add rax,10
\n mov rdi,rax
\n call qword ptr [7F22DAC844C8]; Tests.Nop(Byte*)
\n xor eax,eax
\n mov [rsp],rax
\n add rsp,8
\n ret
\n; Total bytes of code 55<\/p>\n

dotnet\/runtime#87656<\/a> is another nice example and addition to the JIT\u2019s optimization repertoire. As was discussed earlier, branches have costs associated with them. A hardware\u2019s branch predictor can often do a very good job of mitigating the bulk of those costs, but there\u2019s still some, and even if it were fully mitigated in the common case, a branch prediction failure can be relatively very costly. As such, minimizing branches can be very helpful, and if nothing else, turning branch-based operations into branchless ones leads to more consistent and predictable throughput, as it\u2019s then less subject to the nature of the data being processed. Consider the following function that\u2019s used to determine whether a character is a particular subset of whitespace characters:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(‘s’)]
\n public bool IsJsonWhitespace(int c)
\n {
\n if (c == ‘ ‘ || c == ‘t’ || c == ‘r’ || c == ‘n’)
\n {
\n return true;
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

On .NET 8, we get what you\u2019d probably expect, a series of cmps followed by conditional jumps, one for each character:<\/p>\n

; Tests.IsJsonWhitespace(Int32)
\n push rbp
\n mov rbp,rsp
\n cmp esi,20
\n je short M00_L00
\n cmp esi,9
\n je short M00_L00
\n cmp esi,0D
\n je short M00_L00
\n cmp esi,0A
\n je short M00_L00
\n xor eax,eax
\n pop rbp
\n ret
\nM00_L00:
\n mov eax,1
\n pop rbp
\n ret
\n; Total bytes of code 35<\/p>\n

On .NET 9, though, we now get this:<\/p>\n

; Tests.IsJsonWhitespace(Int32)
\n push rbp
\n mov rbp,rsp
\n cmp esi,20
\n ja short M00_L00
\n mov eax,0FFFFD9FF
\n bt rax,rsi
\n jae short M00_L01
\nM00_L00:
\n xor eax,eax
\n pop rbp
\n ret
\nM00_L01:
\n mov eax,1
\n pop rbp
\n ret
\n; Total bytes of code 31<\/p>\n

It\u2019s now using a bt instruction (a bit test) against a pattern where there\u2019s a bit set for each of the characters being tested against, consolidating most of the branches down to just this one.<\/p>\n

Unfortunately, this also highlights that such optimizations, which are looking for a particular pattern, can get knocked off their golden path, at which point the optimization won\u2019t kick in. In this case, there are several ways it can get knocked off. The most obvious is if there are too many values or if they\u2019re too spread out, such that they can\u2019t fit into the 32-bit or 64-bit bit mask. More interesting, if you switch it to instead use C# pattern matching (e.g. c is ‘ ‘ or ‘t’ or ‘r’ or ‘n’), it also doesn\u2019t kick in. Why? Because the C# compiler itself is trying to optimize, and the pattern it ends up generating in the IL is different from what this optimization is expecting. I expect this\u2019ll get better in the future, but it\u2019s a good reminder that these kinds of optimizations are useful when they make arbitrary code better, but if you\u2019re coding to the exact nature of the optimization and relying on it happening, you really need to be paying attention.<\/p>\n

A related optimization was added in dotnet\/runtime#93521<\/a>. Consider a function like the following, which is checking to see whether a character is a lower-case hexadecimal char:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(‘s’)]
\n public bool IsHexLower(char c)
\n {
\n if ((c >= ‘0’ && c <= ‘9’) || (c >= ‘a’ && c <= ‘f’))
\n {
\n return true;
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

On .NET 8, we get a comparison against ‘0’, a comparison again ‘9’, a comparison against ‘a’, and a comparison against ‘f’, with a conditional branch for each:<\/p>\n

; Tests.IsHexLower(Char)
\n push rbp
\n mov rbp,rsp
\n movzx eax,si
\n cmp eax,30
\n jl short M00_L00
\n cmp eax,39
\n jle short M00_L02
\nM00_L00:
\n cmp eax,61
\n jl short M00_L01
\n cmp eax,66
\n jle short M00_L02
\nM00_L01:
\n xor eax,eax
\n pop rbp
\n ret
\nM00_L02:
\n mov eax,1
\n pop rbp
\n ret
\n; Total bytes of code 38<\/p>\n

But on .NET 9, we instead get this:<\/p>\n

; Tests.IsHexLower(Char)
\n push rbp
\n mov rbp,rsp
\n movzx eax,si
\n mov ecx,eax
\n sub ecx,30
\n cmp ecx,9
\n jbe short M00_L00
\n sub eax,61
\n cmp eax,5
\n jbe short M00_L00
\n xor eax,eax
\n pop rbp
\n ret
\nM00_L00:
\n mov eax,1
\n pop rbp
\n ret
\n; Total bytes of code 36<\/p>\n

Effectively the JIT has rewritten the condition as if I\u2019d written it like this:<\/p>\n

(((uint)c – ‘0’) <= (‘9’ – ‘0’)) || (((uint)c – ‘a’) <= (‘f’ – ‘a’))<\/p>\n

which is nice, because it\u2019s replaced two of the conditional branches with two (cheaper) subtractions.<\/p>\n

Write Barriers<\/h3>\n

The .NET garbage collector (GC) is a generational collector. That means it divides the heap up logically by object age, where \u201cgeneration 0\u201d (or \u201cgen0\u201d) are the newest objects that haven\u2019t been around for very long, \u201cgen2\u201d are the objects that have been around for a while, and \u201cgen1\u201d are in the middle. This approach is based on the theory (that also generally plays out in practice) that most objects end up being very short-lived, created for some task and then quickly dropped, and conversely that if an object has been around for a while, there\u2019s a really good chance it\u2019ll continue to be around for a while. By partitioning up objects like this, the GC can reduce the amount of work it needs to do when it scans for objects to be collected. It can do a scan focused only on gen0 objects, allowing it to ignore anything in gen1 or gen2 and thereby make its scan much faster. Or at least, that\u2019s the goal. If it were to only scan gen0 objects, though, it could easily think a gen0 object wasn\u2019t referenced because it couldn\u2019t find any references to one from other gen0 objects\u2026 but there may have been a reference from a gen1 or gen2 object. That would be bad. How does the GC deal with this then, having its cake and eating it, too? It colludes with the rest of the runtime to track any time its generational assumptions might be violated. The GC maintains a table (called the \u201ccard table\u201d) that indicates whether an object in a higher generation might<\/em> contain a reference to a lower generation object, and any time a reference is written such that there could end up being a reference from a higher generation to a lower one, this table is updated. Then when the GC does its scan, it only needs to examine higher generation objects if the relevant bit in the table is set (the table doesn\u2019t track individual objects, just ranges of them, so it\u2019s similar to a \u201cBloom filter\u201d<\/a>, where the lack of a bit means there\u2019s definitely not a reference but the presence of a bit only means there might<\/em> be a reference).<\/p>\n

The code that\u2019s executed to track the reference write and possibly update the card table is referred to as a GC write barrier. And, obviously, if that code is happening every time a reference is written to an object, you really, really, really want that code to be efficient. There are actually multiple different forms of GC write barriers, all specialized for slightly different purposes.<\/p>\n

The standard GC write barrier is CORINFO_HELP_ASSIGN_REF. However, there\u2019s another one called CORINFO_HELP_CHECKED_ASSIGN_REF that needs to do a bit more work. The JIT is the one deciding which of these to use, and it uses the latter when it\u2019s possible the target isn\u2019t on the heap, in which case the barrier needs to do a little more work to figure that out.<\/p>\n

dotnet\/runtime#98166<\/a> helps the JIT do better in a certain case. If you have a static field of a value type:<\/p>\n

static SomeStruct s_someField;
\n…
\nstruct SomeStruct
\n{
\n public object Obj;
\n}<\/p>\n

the runtime implements that by having a box associated with that field for storing that struct. Such static boxes are always on the heap, so if you then do:<\/p>\n

static void Store(object o) => s_someField.Obj = o;<\/p>\n

the JIT can prove that the cheaper unchecked write barrier may be used, and this PR teaches it that. Previously sometimes the JIT would be able to figure it out, but this effectively ensures it.<\/p>\n

Another similar improvement comes from dotnet\/runtime#97953<\/a>. Here\u2019s an example based on ConcurrentQueue<T>, which maintains arrays of elements, each of which is the actual item tagged with a sequence number that\u2019s used by the implementation to maintain correctness in the face of concurrency.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Slot<object>[] _arr = new Slot<object>[1];
\n private object _obj = new object();<\/p>\n

[Benchmark]
\n public void Test() => Store(_arr, _obj);<\/p>\n

private static void Store<T>(Slot<T>[] arr, T o)
\n {
\n arr[0].Item = o;
\n arr[0].SequenceNumber = 1;
\n }<\/p>\n

private struct Slot<T>
\n {
\n public T Item;
\n public int SequenceNumber;
\n }
\n}<\/p>\n

Here as well we can see on .NET 8 it\u2019s using the more expensive checked write barrier, but on .NET 9 the JIT has recognized it can use the cheaper unchecked write barrier:<\/p>\n

\/\/ .NET 8
\n; Tests.Test()
\n push rbx
\n mov rbx,[rdi+8]
\n mov rsi,[rdi+10]
\n cmp dword ptr [rbx+8],0
\n jbe short M00_L00
\n add rbx,10
\n mov rdi,rbx
\n call CORINFO_HELP_CHECKED_ASSIGN_REF
\n mov dword ptr [rbx+8],1
\n pop rbx
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 42<\/p>\n

\/\/ .NET 9
\n; Tests.Test()
\n push rbx
\n mov rbx,[rdi+8]
\n mov rsi,[rdi+10]
\n cmp dword ptr [rbx+8],0
\n jbe short M00_L00
\n add rbx,10
\n mov rdi,rbx
\n call CORINFO_HELP_ASSIGN_REF
\n mov dword ptr [rbx+8],1
\n pop rbx
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 42<\/p>\n

dotnet\/runtime#101761<\/a> actually introduces a new form of write barrier. Consider this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private MyStruct _value;
\n private Wrapper _wrapper = new();<\/p>\n

[Benchmark]
\n public void Store() => _wrapper.Value = _value;<\/p>\n

private record struct MyStruct(string a1, string a2, string a3, string a4);<\/p>\n

private class Wrapper
\n {
\n public MyStruct Value;
\n }
\n}<\/p>\n

Previously as part of copying that struct, each of those fields (represented by a1 through a4) would individually incur a write barrier:<\/p>\n

; Tests.Store()
\n push rax
\n mov [rsp],rdi
\n mov rax,[rdi+8]
\n lea rdi,[rax+8]
\n mov rsi,[rsp]
\n add rsi,10
\n call CORINFO_HELP_ASSIGN_BYREF
\n call CORINFO_HELP_ASSIGN_BYREF
\n call CORINFO_HELP_ASSIGN_BYREF
\n call CORINFO_HELP_ASSIGN_BYREF
\n nop
\n add rsp,8
\n ret
\n; Total bytes of code 47<\/p>\n

Now in .NET 9, this PR added a new bulk write barrier, which can implement the operation more efficiently.<\/p>\n

; Tests.Store()
\n push rax
\n mov rsi,[rdi+8]
\n add rsi,8
\n cmp [rsi],sil
\n add rdi,10
\n mov [rsp],rdi
\n cmp [rdi],dil
\n mov rdi,rsi
\n mov rsi,[rsp]
\n mov edx,20
\n call qword ptr [7F5831BC5740]; System.Buffer.BulkMoveWithWriteBarrier(Byte ByRef, Byte ByRef, UIntPtr)
\n nop
\n add rsp,8
\n ret
\n; Total bytes of code 47<\/p>\n

Making GC write barriers faster is good; after all, they\u2019re used a lot<\/em>. However, switching from the checked write barrier to the non-checked write barrier is a very micro optimization; the extra overhead of the checked variant is often just a couple of comparisons. A better optimization is avoiding the need for a barrier entirely! dotnet\/runtime#103503<\/a> recognizes that ref structs can\u2019t possibly be on the GC heap by their very nature, and as such, write barriers can be entirely elided when writing into the fields of a ref struct.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public void Store()
\n {
\n MyRefStruct s = default;
\n Test(ref s, new object(), new object());
\n }<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private void Test(ref MyRefStruct s, object o1, object o2)
\n {
\n s.Obj1 = o1;
\n s.Obj2 = o2;
\n }<\/p>\n

private ref struct MyRefStruct
\n {
\n public object Obj1;
\n public object Obj2;
\n }
\n}<\/p>\n

On .NET 8, we have two barriers; on .NET 9, zero:<\/p>\n

\/\/ .NET 8
\n; Tests.Test(MyRefStruct ByRef, System.Object, System.Object)
\n push r15
\n push rbx
\n mov rbx,rsi
\n mov r15,rcx
\n mov rdi,rbx
\n mov rsi,rdx
\n call CORINFO_HELP_CHECKED_ASSIGN_REF
\n lea rdi,[rbx+8]
\n mov rsi,r15
\n call CORINFO_HELP_CHECKED_ASSIGN_REF
\n nop
\n pop rbx
\n pop r15
\n ret
\n; Total bytes of code 37<\/p>\n

\/\/ .NET 9
\n; Tests.Test(MyRefStruct ByRef, System.Object, System.Object)
\n mov [rsi],rdx
\n mov [rsi+8],rcx
\n ret
\n; Total bytes of code 8<\/p>\n

Similarly, dotnet\/runtime#102084<\/a> is able to remove some barriers on Arm64 as part of ref struct copies.<\/p>\n

Object Stack Allocation<\/h3>\n

For years, .NET has explored the possibility of stack-allocating managed objects. It\u2019s something that other managed languages like Java are already capable of doing, but it\u2019s also more critical in Java, which lacks the equivalent of value types (e.g. if you want a list of integers, that\u2019d most likely be List<Integer>, which will box each integer value added to the list, similar to if List<object> were used in .NET). In .NET 9, object stack allocation starts to happen. Before you get too excited, it\u2019s limited in scope right now, but in the future it\u2019s likely to expand out further.<\/p>\n

The hardest part of stack allocating objects is ensuring that it\u2019s safe. If a reference to the object were to escape and end up being stored somewhere that outlived the stack frame containing the stack-allocated object, that would be very bad; when the method returned, those outstanding references would be pointing to garbage. So, the JIT needs to perform escape analysis to ensure that never happens, and doing that well is extremely challenging. For .NET 9, the support was introduced in dotnet\/runtime#103361<\/a> (and brought to Native AOT in dotnet\/runtime#104411<\/a>), and it doesn\u2019t do any interprocedural analysis, which means it\u2019s limited to only handling cases where it can easily prove the object reference doesn\u2019t leave the current frame. Even so, there are plenty of situations where this will help to eliminate allocations, and I expect it\u2019ll be expanded to handle more and more cases in the future. When the JIT does choose to allocate an object on the stack, it effectively promotes the fields of the object to be individual variables in the stack frame.<\/p>\n

Here\u2019s a very simple example of the mechanism in action:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public int GetValue() => new MyObj(42).Value;<\/p>\n

private class MyObj
\n {
\n public MyObj(int value) => Value = value;
\n public int Value { get; }
\n }
\n}<\/p>\n

On .NET 8, the generated code for GetValue looks like this:<\/p>\n

; Tests.GetValue()
\n push rax
\n mov rdi,offset MT_Tests+MyObj
\n call CORINFO_HELP_NEWSFAST
\n mov dword ptr [rax+8],2A
\n mov eax,[rax+8]
\n add rsp,8
\n ret
\n; Total bytes of code 31<\/p>\n

The generated code is allocating a new object, populating that object\u2019s Value, and then reading that Value as the value to return. On .NET 9, we instead end up with this picture of simplicity:<\/p>\n

; Tests.GetValue()
\n mov eax,2A
\n ret
\n; Total bytes of code 6<\/p>\n

The JIT has inlined the constructor, inlined accesses to the Value property, promoted the field backing that property to be a variable, and in effect optimized the entire operation simply to be return 42;.<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size
\nAllocated
\nAlloc Ratio<\/p>\n

GetValue
\n.NET 8.0
\n3.6037 ns
\n1.00
\n31 B
\n24 B
\n1.00<\/p>\n

GetValue
\n.NET 9.0
\n0.0519 ns
\n0.01
\n6 B
\n\u2013
\n0.00<\/p>\n

Here\u2019s another more impactful example. When it comes to performance optimization, it\u2019s really nice when the right things just happen; otherwise, developers need to learn the minute differences between performing an operation this way or that way. Every programming language and platform has non-trivial amounts of such things, but we really want to drive the number of them down. One interesting case for .NET has had to do with structs and casting. Consider these two Dispose1 and Dispose2 methods:<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public void Test()
\n {
\n Dispose1<MyStruct>(default);
\n Dispose2<MyStruct>(default);
\n }<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private bool Dispose1<T>(T o)
\n {
\n bool disposed = false;
\n if (o is IDisposable disposable)
\n {
\n disposable.Dispose();
\n disposed = true;
\n }
\n return disposed;
\n }<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private bool Dispose2<T>(T o)
\n {
\n bool disposed = false;
\n if (o is IDisposable)
\n {
\n ((IDisposable)o).Dispose();
\n disposed = true;
\n }
\n return disposed;
\n }<\/p>\n

private struct MyStruct : IDisposable
\n {
\n public void Dispose() { }
\n }
\n}<\/p>\n

Ideally, if you call them with a value type T, there wouldn\u2019t be any allocation, but unfortunately, in Dispose1 because of how things line up here, the JIT would end up needing to box o to produce the IDisposable. Interestingly, due to optimizations several years ago, in Dispose2 the JIT is in fact able to elide the boxing. On .NET 8, we get this:<\/p>\n

; Tests.Dispose1[[Tests+MyStruct, benchmarks]](MyStruct)
\n push rbx
\n mov rdi,offset MT_Tests+MyStruct
\n call CORINFO_HELP_NEWSFAST
\n add rax,8
\n mov ebx,[rsp+10]
\n mov [rax],bl
\n mov eax,1
\n pop rbx
\n ret
\n; Total bytes of code 33<\/p>\n

; Tests.Dispose2[[Tests+MyStruct, benchmarks]](MyStruct)
\n mov eax,1
\n ret
\n; Total bytes of code 6<\/p>\n

This is one of those things that a developer would have to \u201cjust know,\u201d and also fight against tooling like IDE0038<\/a> that pushes developers to write this code like in my first version, whereas for structs the latter ends up being more efficient. This work on stack allocation makes that difference go away, because the boxing that occurs as part of the first version is a quintessential example of the allocation the compiler is now able to stack allocate. On .NET 9, we now end up with this:<\/p>\n

; Tests.Dispose1[[Tests+MyStruct, benchmarks]](MyStruct)
\n mov eax,1
\n ret
\n; Total bytes of code 6<\/p>\n

; Tests.Dispose2[[Tests+MyStruct, benchmarks]](MyStruct)
\n mov eax,1
\n ret
\n; Total bytes of code 6<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size
\nAllocated
\nAlloc Ratio<\/p>\n

Test
\n.NET 8.0
\n5.726 ns
\n1.00
\n94 B
\n24 B
\n1.00<\/p>\n

Test
\n.NET 9.0
\n2.095 ns
\n0.37
\n45 B
\n\u2013
\n0.00<\/p>\n

Inlining<\/h3>\n

Improvements in inlining were a major focus of previous releases, and will likely be a major focus again in the future. For .NET 9, there weren\u2019t a ton of changes, but there was one particularly impactful improvement.<\/p>\n

As a motivating example, consider ArgumentNullException.ThrowIfNull again. It is defined like this:<\/p>\n

public static void ThrowIfNull(object? arg, [CallerArgumentExpression(nameof(arg))] string? paramName = null);<\/p>\n

Notably, it\u2019s non-generic, and that\u2019s a question we get asked about at some relevant frequency. We chose not to make it generic for three reasons:<\/p>\n

The main benefit of making it generic would be to avoid boxing structs, but the JIT already eliminated said boxing in tier 1, and as was highlighted earlier in this post, it\u2019s possible for it to eliminate it in tier 0 as well (and now does).
\nEvery generic instantiation (using a generic with a different type) adds runtime overhead. We didn\u2019t want to bloat a process with such additional metadata and runtime data structures just to support argument validation that should rarely if ever fail in production.
\nWhen used with reference types (which is its raison d\u2019etre), it would not play well with inlining, but inlining of such a \u201cthrow helper\u201d is critical for performance. Generic methods with coreclr and Native AOT work in one of two ways. For value types, every time a generic is used with a different value type, an entire copy of the generic method is made and specialized for that parameter type; it\u2019s as if you wrote a dedicated version of that generic code that wasn\u2019t generic and was instead customized specifically for that type. For reference types, there\u2019s only one copy of the code that\u2019s then shared across all reference types, and it\u2019s parameterized at run-time based on the actual type being used. When you access such a shared generic, at run-time it ends up looking up in a dictionary the information about the generic argument and using the discovered information to inform the rest of the method. Historically, this has not been conducive to inlining.<\/p>\n

So, ThrowIfNull is non-generic. But, there are other throw helpers, many of them are generic. That\u2019s because a) they\u2019re primarily expected to work with value types, and b) we had no choice, given the nature of the methods. So, for example, ArgumentOutOfRangeException.ThrowIfEqual is generic on T, accepting two values of T to compare and throw if they\u2019re the same. And if T is a reference type, on .NET 8 it may not successfully inline if the caller is a shared generic as well. With this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

namespace Benchmarks;<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic unsafe class Tests
\n{
\n private static void Main(string[] args) =>
\n BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[Benchmark]
\n public void Test() => ThrowOrDispose(new Version(1, 0), new Version(1, 1));<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static void ThrowOrDispose<T>(T value, T invalid) where T : IEquatable<T>
\n {
\n ArgumentOutOfRangeException.ThrowIfEqual(value, invalid);
\n if (value is IDisposable disposable)
\n {
\n disposable.Dispose();
\n }
\n }
\n}<\/p>\n

on .NET 8 we get this for the ThrowOrDispose method (this example benchmark has a slightly different shape from previous examples and this output is from Windows, for reasons to be made clearer shortly):<\/p>\n

; Benchmarks.Tests.ThrowOrDispose[[System.__Canon, System.Private.CoreLib]](System.__Canon, System.__Canon)
\n push rsi
\n push rbx
\n sub rsp,28
\n mov [rsp+20],rcx
\n mov rbx,rdx
\n mov rsi,r8
\n mov rdx,[rcx+10]
\n mov rax,[rdx+10]
\n test rax,rax
\n je short M01_L00
\n mov rcx,rax
\n jmp short M01_L01
\nM01_L00:
\n mov rdx,7FF996A8B170
\n call CORINFO_HELP_RUNTIMEHANDLE_METHOD
\n mov rcx,rax
\nM01_L01:
\n mov rdx,rbx
\n mov r8,rsi
\n mov r9,1DB81B20390
\n call qword ptr [7FF996AC5BC0]; System.ArgumentOutOfRangeException.ThrowIfEqual[[System.__Canon, System.Private.CoreLib]](System.__Canon, System.__Canon, System.String)
\n mov rdx,rbx
\n mov rcx,offset MT_System.IDisposable
\n call qword ptr [7FF996664348]; System.Runtime.CompilerServices.CastHelpers.IsInstanceOfInterface(Void*, System.Object)
\n test rax,rax
\n jne short M01_L03
\nM01_L02:
\n add rsp,28
\n pop rbx
\n pop rsi
\n ret
\nM01_L03:
\n mov rcx,rax
\n mov r11,7FF9965204F8
\n call qword ptr [r11]
\n jmp short M01_L02
\n; Total bytes of code 124<\/p>\n

Two things in particular to notice here. First, we see there\u2019s a call to CORINFO_HELP_RUNTIMEHANDLE_METHOD; that\u2019s the helper being used to obtain information about the actual type T being used. Second, ThrowIfEqual is not being inlined; if that were being inlined, we wouldn\u2019t see that call to ThrowIfEqual here but instead we\u2019d see the actual code for ThrowIfEqual. We can confirm why it\u2019s not being inlined via another BenchmarkDotNet diagnoser: [InliningDiagnoser]. The JIT is capable of emitting events for much of its activity, including reporting on any successful or failed inlining operations, and [InliningDiagnoser] listens to those events and reports them as part of the benchmarking results. This particular diagnoser is in a separate BenchmarkDotNet.Diagnostics.Windows package and only works today when running on Windows, because it relies on ETW, hence for comparison why I made the previous benchmark also be Windows. When I add:<\/p>\n

[InliningDiagnoser(allowedNamespaces: [“Benchmarks”])]<\/p>\n

to my Tests class, and run the benchmarks for .NET 8:<\/p>\n

\/\/ Add a <PackageReference Include=”BenchmarkDotNet.Diagnostics.Windows” Version=”9.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Diagnostics.Windows.Configs;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

namespace Benchmarks;<\/p>\n

[InliningDiagnoser(allowedNamespaces: [“Benchmarks”])]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static void Main(string[] args) =>
\n BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[Benchmark]
\n public void Test() => ThrowOrDispose(new Version(1, 0), new Version(1, 1));<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static void ThrowOrDispose<T>(T value, T invalid) where T : IEquatable<T>
\n {
\n ArgumentOutOfRangeException.ThrowIfEqual(value, invalid);
\n if (value is IDisposable disposable)
\n {
\n disposable.Dispose();
\n }
\n }
\n}<\/p>\n

I see this as part of the output:<\/p>\n

Inliner: Benchmarks.Tests.ThrowOrDispose – generic void (!!0,!!0)
\nInlinee: System.ArgumentOutOfRangeException.ThrowIfEqual – generic void (!!0,!!0,class System.String)
\nFail Reason: runtime dictionary lookup<\/p>\n

In other words, ThrowOrDispose called ThrowIfEqual but couldn\u2019t inline it because ThrowIfEqual contained a \u201cruntime dictionary lookup;\u201d in other words, it\u2019s a shared generic method.<\/p>\n

Now on .NET 9, thanks to dotnet\/runtime#99265<\/a>, it is inlined! The resulting assembly is too large for me to show here, but we can see the impact in the benchmark results:<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Test
\n.NET 8.0
\n17.54 ns
\n1.00<\/p>\n

Test
\n.NET 9.0
\n12.76 ns
\n0.73<\/p>\n

and we can see it in the inlining report as successfully inlining.<\/p>\n

GC<\/h2>\n

Applications end up having different needs when it comes to memory management. Would you be willing to throw more memory at maximizing throughput, or do you care more about minimizing working set? How important is it that unused memory be returned to the system aggressively? Is your expected workload constant or ebbing and flowing in nature? The GC has long had lots of knobs for configuring behavior based on these kinds of questions, but none more apparent than the choice of whether to use the \u201cworkstation GC\u201d or \u201cserver GC\u201d.<\/p>\n

By default, an application uses the workstation GC, though some environments (like ASP.NET) opt-in to using server GC automatically. You can explicitly opt-in in a variety of ways, including by adding <ServerGarbageCollection>true<\/ServerGarbageCollection> into your project file (as we did in the Benchmarking Setup<\/a> section of this post). Workstation GC optimizes for reduced memory consumption, while server GC optimizes for maximum throughput. Historically, workstation employs a single heap, whereas server employs a heap per core. That typically represents a tradeoff between amount of memory consumed and the overhead of accessing a heap, such as the cost of allocating. If a bunch of threads are all trying to allocate at the same time, with server GC they\u2019re very likely to all be accessing different heaps, thereby reducing contention, whereas with workstation GC, they\u2019re all going to be fighting for access. Conversely, more heaps generally means more memory consumed (even though each heap could be smaller than the single one), especially in lull periods where the system might not be fully loaded, yet is paying in working set for those extra heaps.<\/p>\n

The decision for which to use isn\u2019t always so clear. Especially in the presence of containers, you frequently still care about really good throughput, but also don\u2019t want to be spending memory uselessly. Enter \u201cDATAS,\u201d or \u201cDynamically Adapting To Application Sizes\u201d<\/a>. DATAS was introduced in .NET 8 and serves to narrow the gap between workstation and server GC, bringing server GC closer to workstation memory consumption. It dynamically scales how much memory is being consumed by server GC, such that in times of less load, less memory is being used. While DATAS shipped in .NET 8, it was only on by default for Native AOT-based projects, and even there it still had some issues to be sorted. Those issues have now been sorted (e.g. dotnet\/runtime#98743<\/a>, dotnet\/runtime#100390<\/a>, dotnet\/runtime#102368<\/a>, and dotnet\/runtime#105545<\/a>), such that in .NET 9, as of dotnet\/runtime#103374<\/a>, DATAS is now enabled by default for server GC.<\/p>\n

If you have a workload where absolute best possible throughput is paramount and you\u2019re ok with additional memory being consumed to enable that, you should feel free to disable DATAS, e.g. by adding this to your project file:<\/p>\n

<GarbageCollectionAdaptationMode>0<\/GarbageCollectionAdaptationMode><\/p>\n

While DATAS being enabled by default is a very impactful improvement for .NET 9, there are other GC-related improvements in the release as well. For example, when compacting heaps, the GC may end up sorting objects by addresses. For large numbers of objects, this sort can be relatively expensive, and it behooves the GC to parallelize the sorting operation. For this purpose, several releases ago the GC incorporated a parallelized sorting algorithm called vxsort, which is effectively a quicksort with a parallelized partitioning step. However, it was only enabled for Windows (and only on x64). In .NET 9, it\u2019s enabled for Linux as well as part of dotnet\/runtime#98712<\/a>. This helps to reduce GC pause times.<\/p>\n

VM<\/h2>\n

The .NET runtime provides many services to managed code. There\u2019s the GC, of course, and the JIT compiler, and then there\u2019s a whole bunch of functionality around things like assembly and type loading, exception handling, configuration management, virtual dispatch, interop infrastructure, stub management, and so on. All of that functionality is generally referred to as being part of the coreclr virtual machine (VM).<\/p>\n

Many performance changes in this area are hard to demonstrate, but they\u2019re still worth mentioning. dotnet\/runtime#101580<\/a> lazily-allocates some information related to method entrypoints, resulting in smaller heap sizes and less work on startup. dotnet\/runtime#96857<\/a> also removed some unnecessary allocation happening related to data structures around methods. dotnet\/runtime#96703<\/a> reduced the algorithmic complexity of some key functions involved in building up method tables, while dotnet\/runtime#96466<\/a> streamlined access to those tables, minimizing the number of indirections involved.<\/p>\n

Another set of changes went in to improving various calls from managed code into the VM. When managed code needs to call into the runtime, it has a couple of mechanisms it can employ. One is a \u201cQCALL,\u201d which is effectively just a P\/Invoke \/ DllImport into functions declared in the runtime. The other is an \u201cFCALL,\u201d a much more specialized and complicated mechanism for invoking runtime code that\u2019s capable of accessing managed objects. FCALL used to be the dominant mechanism, but each release more and more such calls are transitioned over to being QCALLs, which helps with both correctness (FCALLs can be hard to \u201cget right\u201d) and in some cases performance (some FCALLS need helper method frames that in turn typically make them more expensive than QCALLs). dotnet\/runtime#96860<\/a> converted over members of Marshal, dotnet\/runtime#96916<\/a> did so for Interlocked, dotnet\/runtime#96926<\/a> handled several more threading-related members, dotnet\/runtime#97432<\/a> converted some of the built-in marshaling support, dotnet\/runtime#97469<\/a> and dotnet\/runtime#100939<\/a> handled methods from GC and throughout reflection, dotnet\/runtime#103211<\/a> from @AustinWise<\/a> converted GC.ReRegisterForFinalize, and dotnet\/runtime#105584<\/a> converted Delegate.GetMulticastInvoke (which is used in APIs like Delegate.Combine and Delegate.Remove). dotnet\/runtime#97590<\/a> did the same for the slow path in ValueType.GetHashCode, while also converting the fast path to managed to avoid the transition entirely.<\/p>\n

But arguably the most impactful change in this area for .NET 9 is around exceptions. Exceptions are expensive and should be avoided where performance matters. But\u2026 just because they\u2019re expensive doesn\u2019t mean it\u2019s not valuable to make them less expensive. And in fact, there are cases where it\u2019s really worthwhile to make them less expensive. One of the things we sporadically observe in the wild are \u201cexception storms.\u201d Some failure happens, which causes another failure, which causes another. Each of those incurs exceptions. CPU consumption starts to spike as the overhead of those exceptions is incurred. Now other things start to time out because they\u2019re getting starved, and they throw exceptions, which in turn causes more failures. You get the idea.<\/p>\n

In Performance Improvements in .NET 8<\/a>, I highlighted that in my opinion the single most important performance improvement in the release was a single character change, enabling dynamic PGO by default. Now in .NET 9, dotnet\/runtime#98570<\/a> is similar, a super small and simple PR that belies all the work that came before it. Earlier on, dotnet\/runtime#88034<\/a> had ported the Native AOT exception handling implementation over to coreclr, but it was disabled by default due to still needing bake time. It\u2019s now had that bake time, and the new implementation is now on by default in .NET 9. And it\u2019s much faster. Things get better still with dotnet\/runtime#103076<\/a>, which removes a global spinlock involved in the handling of exceptions.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public async Task ExceptionThrowCatch()
\n {
\n for (int i = 0; i < 1000; i++)
\n {
\n try { await Recur(10); } catch { }
\n }
\n }<\/p>\n

private async Task Recur(int depth)
\n {
\n if (depth <= 0)
\n {
\n await Task.Yield();
\n throw new Exception();
\n }<\/p>\n

await Recur(depth – 1);
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ExceptionThrowCatch
\n.NET 8.0
\n123.03 ms
\n1.00<\/p>\n

ExceptionThrowCatch
\n.NET 9.0
\n54.68 ms
\n0.44<\/p>\n

Mono<\/h2>\n

We frequently say \u201cthe runtime,\u201d but in reality there are currently multiple runtime implementations in .NET. \u201ccoreclr\u201d is the runtime thus far referred to, which is the default runtime used on Windows, Linux, and macOS, and for services and desktop applications, but there\u2019s also \u201cmono,\u201d which is mainly used when the form factor of the target application requires a small runtime: by default, it\u2019s the runtime that\u2019s used when building mobile apps for Android and iOS today, as well as the runtime used for Blazor WASM apps. mono has also seen a multitude of performance improvements in .NET 9:<\/p>\n

Save\/restoring of profile data.<\/strong> One of the features provided by mono is an interpreter, which enables .NET code to execute in environments where JIT\u2019ing isn\u2019t permitted, as well as to enable faster startup. Specifically for when targeting WASM, the interpreter has a form of PGO where after methods have been invoked some number of times and are deemed important, it\u2019ll generate WASM on-the-fly to optimize those methods. This tiering gets better in .NET 9 with dotnet\/runtime#92981<\/a>, which enables keeping track of which methods tiered up, and if the code is running in a browser, storing that information in the browser\u2019s cache for subsequent runs. When the code then runs subsequently, it can incorporate the previous learnings to tier up better and more quickly.
\nSSA-based Optimization.<\/strong> The compiler that generates that WASM applied optimizations primarily at the basic block level. dotnet\/runtime#96315<\/a> overhauls the implementation to employ Static Single Assignment (SSA) form, which is commonly used by optimizing compilers and which ensures that every variable is assigned in exactly one location. That form simplifies many resulting analyses and thus helps to better optimize the code.
\nVector improvements.<\/strong> More and more vectorization is being done by the core libraries, utilizing hardware intrinsics and the various Vector types. To enable such library code to execute well on mono, the various mono backends need to also handle those operations efficiently. One of the most impactful changes here is dotnet\/runtime#105299<\/a>, which updated mono to accelerate Shuffle for types other than byte and sbyte (which were already handled). This is very impactful to functionality in the core libraries, many of which use Shuffle as part of core algorithms, like throughout IndexOfAny, hex encoding and decoding, Base64 encoding and decoding, Guid, and more. dotnet\/runtime#92714<\/a> and dotnet\/runtime#98037<\/a> also improved vector construction, such as by enabling the mono JIT to utilize the Arm64 ins (Insert) instruction when creating one float or double vector from the values in another.
\nMore intrinsics.<\/strong> dotnet\/runtime#98077<\/a>, dotnet\/runtime#98514<\/a>, and dotnet\/runtime#98710<\/a> implemented various AdvSimd.Load* and AdvSimd.Store* APIs. dotnet\/runtime#99115<\/a> and dotnet\/runtime#101622<\/a> intrinsified several clearing and filling methods that back Span<T>.Clear\/Fill. And dotnet\/runtime#105150<\/a> and dotnet\/runtime#104698<\/a> optimized various Unsafe methods, such as BitCast. dotnet\/runtime#91813<\/a> also significantly improved unaligned access on a variety of CPUs by not forcing the implementation down a slow path if the CPU is able to handle such reads and writes.
\nStartup<\/strong>. dotnet\/runtime#100146<\/a> is a fun one, as it had accidentally positive benefits for mono startup. The change updated dotnet\/runtime\u2019s configuration to enable more static analysis, and in particular enforcing CA1865, CA1866, and CA1867<\/a>, which we hadn\u2019t yet gotten around to enabling for the repo. The change included fixing all of the violations of the rules, which mostly meant fixing call sites like IndexOf(“!”) (IndexOf taking a single-character string) and replacing it with IndexOf(‘!’). The intent of the rule was that doing so is a little bit faster and the call site ends up being a little bit cleaner. But IndexOf(string) is cultural-aware, which means using it can force the globalization library ICU to be loaded and initialized. As it turns out, some of these uses were on mono\u2019s startup path and were forcing ICU to be loaded when it wasn\u2019t actually necessary. Fixing those meant the loading could be delayed, and startup performance improved as a result. dotnet\/runtime#101312<\/a> also improved startup with the interpreter by adding a cache to the code that does vtable setups. This uses a custom hash table implementation added in dotnet\/runtime#100386<\/a>, which is then also used elsewhere, such as in dotnet\/runtime#101460<\/a> and dotnet\/runtime#102476<\/a>. That hash table is itself interesting, as it\u2019s lookups are vectorized for x64, Arm, and WASM and it\u2019s generally optimized for cache locality.
\nVariance check removal.<\/strong> When storing objects into an array, that operation needs to be validated to ensure compatibility between the type being stored and the concrete type of the array. Given a base type B and two derived types D1 : B and D2 : B, you could have an array B[] array = new D1[42];, and then the code array[0] = new D2() would successfully compile, because D2 is a B, but at run-time this must fail, as D2 is not a D1, and so the runtime needs a check to ensure correctness. If the array\u2019s type is sealed, though, this check can be avoided, since then you can\u2019t end up with this discrepancy. coreclr already does that optimization; now as part of dotnet\/runtime#99829<\/a>, the mono interpreter does so as well.<\/p>\n

Native AOT<\/h2>\n

Native AOT is a solution for generating native executables directly from .NET applications. The resulting binary doesn\u2019t require .NET to be installed and does not require JIT\u2019ing; instead it contains in it all of the assembly code for the whole app, inclusive of the code for any core library functionality accessed, the assembly for the garbage collector, and so on. Native AOT first shipped in .NET 7 and was then significantly improved for .NET 8, in particular around reducing the size of the resulting applications. Now in .NET 9, investment continues in Native AOT, with some very nice fruits of the labor on it. (Note that the Native AOT tool chain uses the JIT to generate assembly code, so most of the code generation improvements discussed in the JIT<\/a> section and elsewhere in this post accrue to Native AOT as well.)<\/p>\n

One of the biggest concerns for Native AOT is size and trimming. Native AOT-based applications and libraries compile everything, all user code, all the library code, the runtime, everything, into the single native binary. It\u2019s thus imperative that the tool chain goes to extra lengths to get rid of as much as possible in order to keep that size down. This can include being more clever about how the runtime stores the state necessary for execution. It can include being more thoughtful about generics in order to reduce the possible code size explosion that can result from lots of generic instantiations (effectively multiple copies of the exact same code all specialized for different type arguments). And it can include being very diligent about avoiding dependencies that can bring in lots of code unexpectedly and that the trimming tools are unable to reason about enough to remove. Here are some examples of all of these in .NET 9:<\/p>\n

Refactoring choke points.<\/strong> Think through your code: how many times have you written a method that takes some input and then dispatches to one of many different kinds of things based on the input provided? That\u2019s reasonably common. Unfortunately, it can also be problematic for Native AOT code size. A good example is fixed by dotnet\/runtime#91185<\/a> in System.Security.Cryptography. There are a bunch of hashing related types, like SHA256 or SHA3_384, that all offer a HashData method. Then there are places where the exact hashing algorithm to be used is specified via a HashAlgorithmName. You can likely envision the large switch statement that results (or don\u2019t imagine and instead just look at the code<\/a>), where based on the exact HashAlgorithmName specified, the implementation selects the right type\u2019s HashData method to call. That is what\u2019s often referred to as a \u201cchoke point,\u201d where all callers end up coming through this one method, which then fans out to the relevant implementations, but that also then causes this size problem for Native AOT: if that choke point is referenced, it typically ends up needing to generate the code for all of the referenced methods, even if only a subset are actually used. Some of these cases are really challenging to solve. In this particular case, though, thankfully all of those HashData methods turned around and called to a parameterized, shared implementation. So the fix was to just skip the middle tier and have the HashAlgorithmName layer go directly to the workhorse implementation, without naming the intermediate layer methods.
\nLess LINQ.<\/strong> LINQ is an amazing productivity tool. We love LINQ and invest in it every release of .NET (see the later section in this post on tons of performance improvements in LINQ in .NET 9<\/a>). With Native AOT, however, significant use of LINQ can also measurably increase code size, in particular when value types are involved. As will be discussed later when talking about LINQ optimizations, one of the optimizations LINQ employs is to special-case based on the inputs what kind of IEnumerable<T> its methods give back. So, for example, if you call Select with an array input, the IEnumerable<T> you get back might actually be an instance of the internal ArraySelectIterator<T>, and if you call Select with a List<T>, the IEnumerable<T> you get back might actually be an instance of the internal ListSelectIterator<T>. The Native AOT trimmer can\u2019t readily determine which of those paths might be used, so the Native AOT compiler needs to generate code for all such types when you call Select<T>. If the T is a reference type, there will just be a single copy of the generated code shared for all reference types. But if the T is a value type, there will be a custom stamp of the code generated for and optimized for each unique T. That means if such LINQ APIs (and other similar APIs) are used a lot, they can disproportionately increase the size of a Native AOT binary. dotnet\/runtime#98109<\/a> is an example of a PR that replaced a bit of LINQ code in order to measurably reduce the size of ASP.NET applications compiled with Native AOT. But you can also see that PR being thoughtful about which LINQ usage was removed, citing these few specific instances making a measurable difference and leaving the rest of the LINQ usage in the library intact.
\nAvoiding unnecessary array types.<\/strong> The SharedArrayPool<T> that backs ArrayPool<T>.Shared was storing lots of state, including several fields with types along the lines of T[][]. This makes sense; it\u2019s pooling arrays, so it needs an array of arrays. From a Native AOT perspective, though, if T is a value type (as is very common with ArrayPool<T>), T[][] as its own unique array type needs its own code generated for it, distinct from the code for, for example, T[]. As it turns out, ArrayPool<T> doesn\u2019t actually need to work with the array instances in these cases, so it doesn\u2019t actually need the strongly-typed nature of the arrays; this could just as well be object[] or Array[]. And that\u2019s one of the main things that dotnet\/runtime#97058<\/a> does: with that, the compiled binary can carry the code generated for just Array[] rather than needing code for byte[][] and char[][] and object[][] and for whatever other type Ts are used with ArrayPool<T> in the application.
\nAvoiding unnecessarily generic code.<\/strong> The Native AOT compiler doesn\u2019t do any kind of \u201coutlining\u201d today (the opposite of inlining, where, rather than moving code from a called method into the caller, the compiler would extract code from a method out into a separate method). If you have a large method, the compiler will need to generate code for the whole method, and if that method is generic and multiple generic specializations are compiled, the whole method will be compiled and optimized for each. But, if you have any meaningful amounts of code in that method that don\u2019t actually depend on the generic types in question, you can avoid that duplication by refactoring the code into separate non-generic methods that are invoked by the generic. That\u2019s what dotnet\/runtime#101474<\/a> does in some of the types in Microsoft.Extensions.Logging.Console, like SimpleConsoleFormatter and JsonConsoleFormatter. There\u2019s a generic Write<TState> method, but the TState is only used in the very first line of the method, which formats the arguments into a string. After that, there\u2019s a lot of logic about doing the actual writing, but all of it only needs the output of that formatting operation, not the input. So, this PR simply refactors that Write<TState> to do the formatting and then delegates to the bulk of the work in a separate non-generic method.
\nCutting out unnecessary dependencies.<\/strong> There are many small but meaningful dependencies one doesn\u2019t think about until they start focusing on generated code size and zooming in on exactly where all that code size is coming from. dotnet\/runtime#95710<\/a> is a good example. The AppContext.OnProcessExit method is rooted (never trimmed) by the runtime because it\u2019s invoked when the process is exiting. That OnProcessExit was accessing AppDomain.CurrentDomain, which returns an AppDomain. AppDomain\u2018s ToString override depends on a bunch of stuff. And ToString on a type that\u2019s not trimmed away is itself basically never trimmed because if anything anywhere calls to the base object.ToString, the system needs to know that any possible derived type that might find its way to that call site will be invokable. That all means that all of that stuff used by AppDomain.ToString was never being trimmed. This small refactoring made it so that all that stuff would only need to be kept if AppDomain.CurrentDomain is ever actually accessed by user code. Another example of this comes in dotnet\/runtime#101858<\/a>, which removes a dependency on some of the Convert methods.
\nUsing a better tool for the job.<\/strong> Sometimes there\u2019s just a better, simpler answer. dotnet\/runtime#100916<\/a> highlights one such case. Some code in Microsoft.Extensions.DependencyInjection needed a MethodInfo for a particular method, and it was using System.Linq.Expressions to extract one, when it could instead just use a delegate. That\u2019s not only cheaper in terms of allocation and overhead, it removes a dependency on the Expressions library.
\nCompile time instead of run time.<\/strong> Source generators are a great boon for Native AOT, as they enable computing things at build time and baking the results into the assembly rather than computing those same things at run time (which, in the relevant situations, typically is done once and then cached). That\u2019s useful for startup performance, as you\u2019re not having to do that work just to get going. It\u2019s useful for steady-state throughput, as you can often take more time to do a better job when the work is being done at build time. But it\u2019s also useful for size, because it removes a dependency on anything that might have been used as part of the computation. And often that dependency is reflection, which brings with it a lot of size. As it turns out, System.Private.CoreLib has its own source generator that\u2019s used when building CoreLib, and dotnet\/runtime#102164<\/a> augmented that source generator to generate a dedicated implementation of Environment.Version and RuntimeInformation.FrameworkDescription. Previously, both of these methods that are implemented in CoreLib would use reflection to look up attributes also in CoreLib, but that\u2019s something the source generator can instead do at a build time, and just bake the answer into the implementation of these methods.
\nAvoiding duplication.<\/strong> It\u2019s not uncommon to have two methods somewhere in your app that have the same implementations, especially for small helper methods, like property accessors. dotnet\/runtime#101969<\/a> teaches the Native AOT tool chain to deduplicate those, such that the code is only stored once.
\nInterfaces be gone.<\/strong> Previously, unused interface methods could be trimmed away (effectively removing them from the interface type and removing all implementations of that method), but the compiler wasn\u2019t able to fully remove the actual interface types themselves. Now with dotnet\/runtime#100000<\/a>, it can.
\nUnnecessary static constructors.<\/strong> The trimmer was keeping the static constructor of a type if any field was accessed. This is unnecessarily broad: those static constructors only need to be kept if a static<\/em> field was accessed. dotnet\/runtime#96656<\/a> improves that.<\/p>\n

Previous releases saw a considerable amount of time spent on driving down binary sizes, but these kinds of changes chip away at them even further. Let\u2019s create a new ASP.NET minimal APIs application using Native AOT. This command uses the webapiaot template and creates the new project in a new myapp directory:<\/p>\n

dotnet new webapiaot -o myapp<\/p>\n

Replace the contents of the generated myapp.csproj with this:<\/p>\n

<Project Sdk=”Microsoft.NET.Sdk.Web”><\/p>\n

<PropertyGroup>
\n <TargetFrameworks>net9.0;net8.0<\/TargetFrameworks>
\n <Nullable>enable<\/Nullable>
\n <ImplicitUsings>enable<\/ImplicitUsings>
\n <InvariantGlobalization>true<\/InvariantGlobalization>
\n <PublishAot>true<\/PublishAot><\/p>\n

<OptimizationPreference>Size<\/OptimizationPreference>
\n <StackTraceSupport>false<\/StackTraceSupport>
\n <\/PropertyGroup><\/p>\n

<\/Project><\/p>\n

All I\u2019ve done on top of the template\u2019s defaults is have both net9.0 and net8.0 as target frameworks and then add a couple of settings (at the bottom) focused on driving down the size of Native AOT apps. The app is a simple site that exposes a \/todos list as JSON.\n<\/p>\n

We can publish this app with Native AOT:<\/p>\n

dotnet publish -f net8.0 -r linux-x64 -c Release
\nls -hs bin\/Release\/net8.0\/linux-x64\/publish\/myapp<\/p>\n

which yields:<\/p>\n

9.4M bin\/Release\/net8.0\/linux-x64\/publish\/myapp<\/p>\n

We can see here that the whole site, web server, garbage collector, everything, are contained in myapp app, which on .NET 8 is weighing in at 9.4 megabytes. Now, let\u2019s do the same thing for .NET 9:<\/p>\n

dotnet publish -f net9.0 -r linux-x64 -c Release
\nls -hs bin\/Release\/net9.0\/linux-x64\/publish\/myapp<\/p>\n

which results in:<\/p>\n

8.5M bin\/Release\/net9.0\/linux-x64\/publish\/myapp<\/p>\n

Now, just by moving to the new version, that same myapp has shrunk to 8.5 megabytes, an ~10% reduction in binary size.<\/p>\n

Beyond a focus on size, ahead-of-time compilation also differs from just-in-time compilation in that each has their own opportunities for unique optimizations. The JIT can see the exact details of the current machine and employ the best possible instructions based on what\u2019s available (e.g. using AVX512 instructions on hardware that supports it), and the JIT can use dynamic PGO to evolve the code based on execution characteristic. But, Native AOT is capable of doing whole program optimization, where it can look at everything in the program and optimize based on the totality of everything involved (in contrast, a JIT\u2019d .NET application may load additional .NET libraries at any point). For example, dotnet\/runtime#92923<\/a> enables automatically making fields readonly based on looking at the whole program to see if anything could possibly write to the field from outside of the constructor; this can in turn help things like improving pre-initialization.<\/p>\n

dotnet\/runtime#99761<\/a> provides a nice example where, based on whole program analysis, the compiler can see that a particular type is never instantiated. If it\u2019s never instantiated, then type checks for that type will never succeed. And thus if a program has a check like if (variable is SomethingNeverInstantiated), that can be turned into a constant false, and all of the code associated with that if block then eliminated. dotnet\/runtime#102248<\/a> is similar, but for types; if code is doing if (someType == typeof(X)) and the compiler never had to construct a method table for X, it can similarly turn this into a constant result.<\/p>\n

Whole program analysis is also applicable to devirtualization in really cool ways. With dotnet\/runtime#92440<\/a>, the compiler can now devirtualize all calls to a virtual method C.M if the compiler doesn\u2019t see any instantiations of types that derive from C. And with dotnet\/runtime#97812<\/a> and dotnet\/runtime#97867<\/a>, the compiler can now treat virtual methods as instead being non-virtual and sealed when there are no overrides of those methods anywhere in the program.<\/p>\n

NativeAOT also has a super power in its ability to do pre-initialization. The compiler contains an interpreter that\u2019s able to evaluate code at build time and replace that code with just the result; for some objects, it\u2019s then also able to blit the object\u2019s data into the binary in a way that it can be cheaply dehydrated at execution time. The interpreter is limited in what it\u2019s able and allowed to do, but over time its capabilities are improving. dotnet\/runtime#92470<\/a> extends it to support more type checks, static interface method calls, constrained method calls, and various operations on spans, while dotnet\/runtime#92666<\/a> expands it to have some support for hardware intrinsics and the various IsSupported methods. dotnet\/runtime#92739<\/a> further rounds it out with support for stackalloc\u2018ing spans, IntPtr\/nint math, and Unsafe.Add.<\/p>\n

Threading<\/h2>\n

Since the beginning of .NET, general wisdom has been that the vast majority of code that needs to synchronize access to shared state should just use Monitor, either directly or more likely via the the C# language syntax for it with lock(…). There are a plethora of other synchronization primitives available, at various levels of complexity and with varying purposes, but lock(…) is the workhorse and the thing that everyone should reach for by default.<\/p>\n

Over 20 years after the introduction of .NET, that\u2019s evolving, just a bit. lock(…) is still the go-to syntax, but in .NET 9 as of dotnet\/runtime#87672<\/a> and dotnet\/runtime#102222<\/a>, there is now a dedicated System.Threading.Lock type. Anywhere you were previously allocating an object just to use that object with lock(…), you should consider using the new Lock type. You can absolutely still just use object, and you\u2019ll still need to do so in certain situations, like if you\u2019re using the \u201ccondition variable\u201d aspects of Monitor (such as Signal and Wait), and you\u2019ll still want to in others (such as if you\u2019re trying to reduce managed allocation and you have another existing object that can serve double-duty as the monitor). But locking a Lock can be a more efficient answer. It can also help to be self-documenting, making the code cleaner and more maintainable.<\/p>\n

As is evident from this benchmark, the syntax for using both can be identical.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private readonly object _monitor = new();
\n private readonly Lock _lock = new();
\n private int _value;<\/p>\n

[Benchmark]
\n public void WithMonitor()
\n {
\n lock (_monitor)
\n {
\n _value++;
\n }
\n }<\/p>\n

[Benchmark]
\n public void WithLock()
\n {
\n lock (_lock)
\n {
\n _value++;
\n }
\n }
\n}<\/p>\n

Lock, however, will generally be a tad cheaper (and in the future, as most locking shifts to use the new type, we may be able to make most objects lighter weight by not optimizing for direct locking on arbitrary objects):<\/p>\n

Method
\nMean<\/p>\n

WithMonitor
\n14.30 ns<\/p>\n

WithLock
\n13.86 ns<\/p>\n

Note that C# 13 has special-recognition of System.Threading.Lock. If you look at the code that\u2019s generated for WithMonitor above, it\u2019s equivalent to this:<\/p>\n

public void WithMonitor()
\n{
\n object monitor = _monitor;
\n bool lockTaken = false;
\n try
\n {
\n Monitor.Enter(monitor, ref lockTaken);
\n _value++;
\n }
\n finally
\n {
\n if (lockTaken)
\n {
\n Monitor.Exit(monitor);
\n }
\n }
\n}<\/p>\n

but even though the syntax is identical, here\u2019s an equivalent of what\u2019s generated for WithLock:<\/p>\n

Lock.Scope scope = _lock.EnterScope();
\ntry
\n{
\n _value++;
\n}
\nfinally
\n{
\n scope.Dispose();
\n}<\/p>\n

We\u2019ve also started using Lock internally. dotnet\/runtime#103085<\/a> and dotnet\/runtime#103104<\/a> used it instead of object locks in Timer, ThreadLocal, and RegisteredWaitHandle. In time, I expect to see more and more use switched over.<\/p>\n

Of course, while locks are the default recommendation for synchronization, there\u2019s still a lot of code that demands the higher throughput and scalability that comes from more lock-free programming, and the workhorse for such implementations is Interlocked. In .NET 9, Interlocked.Exchange and Interlocked.CompareExchange gain some very welcome capabilities. First, dotnet\/runtime#92974<\/a> from @MichalPetryka<\/a>, dotnet\/runtime#97588<\/a> from @filipnavara<\/a>, and dotnet\/runtime#106660<\/a> grant Interlocked some new powers, the ability to operate over types smaller than int. It introduces new overloads of Exchange and CompareExchange that can work on byte, sbyte, ushort, and short. These overloads are public and available for anyone to call, but they\u2019re also then consumed by dotnet\/runtime#97528<\/a> from @MichalPetryka<\/a> to improve Parallel.ForAsync<T>. ForAsync is given a range of T to be processed, and schedules multiple workers that all need to repeatedly get the next item from the range, until the range is exhausted. For arbitrary types, that means ForAsync needs to lock to protect the increment while iterating through the range. But for types where an Interlocked operation is available, we can use that with low-lock techniques to avoid the lock entirely (both the need to access it and the need to allocate it).<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public async Task ParallelForAsync()
\n {
\n await Parallel.ForAsync(”, ‘uFFFF’, async (c, _) =>
\n {
\n });
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ParallelForAsync
\n.NET 8.0
\n42.807 ms
\n1.00<\/p>\n

ParallelForAsync
\n.NET 9.0
\n7.184 ms
\n0.17<\/p>\n

Even with those new overloads, though, there are still places it\u2019s desirable to use Interlocked.Exchange or Interlocked.CompareExchange where they can\u2019t be used easily. Consider the aforementioned Parallel.ForAsync. It\u2019d be really nice if we could just call Interlocked.CompareExchange<T>, but CompareExchange<T> only works with reference types. So we\u2019re instead left with unsafe code:<\/p>\n

static unsafe bool CompareExchange(ref T location, T value, T comparand) =>
\n sizeof(T) == sizeof(byte) ? Interlocked.CompareExchange(ref Unsafe.As<T, byte>(ref location), Unsafe.As<T, byte>(ref value), Unsafe.As<T, byte>(ref comparand)) == Unsafe.As<T, byte>(ref comparand) :
\n sizeof(T) == sizeof(ushort) ? Interlocked.CompareExchange(ref Unsafe.As<T, ushort>(ref location), Unsafe.As<T, ushort>(ref value), Unsafe.As<T, ushort>(ref comparand)) == Unsafe.As<T, ushort>(ref comparand) :
\n sizeof(T) == sizeof(uint) ? Interlocked.CompareExchange(ref Unsafe.As<T, uint>(ref location), Unsafe.As<T, uint>(ref value), Unsafe.As<T, uint>(ref comparand)) == Unsafe.As<T, uint>(ref comparand) :
\n sizeof(T) == sizeof(ulong) ? Interlocked.CompareExchange(ref Unsafe.As<T, ulong>(ref location), Unsafe.As<T, ulong>(ref value), Unsafe.As<T, ulong>(ref comparand)) == Unsafe.As<T, ulong>(ref comparand) :
\n throw new UnreachableException();<\/p>\n

Another place it\u2019d be really nice to use Interlocked.Exchange and Interlocked.CompareExchange is with enums. It\u2019s reasonable common to use these APIs to transition between states in some algorithm, and often the ideal is for those states to be represented as an enum. However, there are no overloads of {Compare}Exchange that have worked with enums, so developers have been forced to use integers instead, often with comments stating something along the lines of \u201cThis should be an enum, but enums can\u2019t work with CompareExchange.\u201d Or, at least, they couldn\u2019t, until .NET 9.<\/p>\n

Now in .NET 9, as of dotnet\/runtime#104558<\/a> the generic Exchange and CompareExchange have had their class constraint removed. This means use of Exchange<T> and CompareExchange<T> will compile for any T. Then at runtime, the T is checked to ensure it\u2019s a reference type, a primitive type, or an enum type; anything else, and it\u2019ll throw. When it is one of those, it delegates to the correspondingly-sized overload. For example, this now compiles and runs successfully:<\/p>\n

static DayOfWeek UpdateIfEqual(ref DayOfWeek location, DayOfWeek newValue, DayOfWeek expectedValue) =>
\n Interlocked.CompareExchange(ref location, newValue, expectedValue);<\/p>\n

This is not only good for usability, it\u2019s good for performance in a few ways. First, it enables performance improvements like the Parallel.ForAsync one described without needing to resort to Unsafe tricks. But second, it enables smaller objects. The previously listed change not only updated CompareExchange to remove the constraint but also then employed the overload in dozens of places. In Http3Connection, for example, the object previously had these three fields which were updated with Interlocked.Exchange:<\/p>\n

private int _haveServerControlStream;
\nprivate int _haveServerQpackDecodeStream;
\nprivate int _haveServerQpackEncodeStream;<\/p>\n

but these are really just bools masquerading as ints, exactly because they needed to be updated atomically. Now with Interlocked.Exchange<T> and Interlocked.CompareExchange<T> supporting bool, these have been updated to just be:<\/p>\n

private bool _haveServerControlStream;
\nprivate bool _haveServerQpackDecodeStream;
\nprivate bool _haveServerQpackEncodeStream;<\/p>\n

Any additional padding aside, that reduces 12 bytes down to 3 bytes on the object.<\/p>\n

Also related to Interlocked, dotnet\/runtime#96258<\/a> intrinsifies the Interlocked.And and Interlocked.Or methods for additional platforms; previously they were specially handled on Arm, but now they\u2019re also specially handled on x86\/64. As an example, the implementation in the And method is a fairly typical CompareExchange loop:<\/p>\n

public static int And(ref int location1, int value)
\n{
\n int current = location1;
\n while (true)
\n {
\n int newValue = current & value;
\n int oldValue = CompareExchange(ref location1, newValue, current);
\n if (oldValue == current)
\n {
\n return oldValue;
\n }
\n current = oldValue;
\n }
\n}<\/p>\n

You\u2019ll see a very similar loop any time you want to use optimistic concurrency to create a new value and substitute it for the original in an atomic manner. The actual & operation is just one line here, and to highlight that this is broadly applicable, you could create a generalized version of this method for any operation using a delegate, like this:<\/p>\n

public static int CompareExchange(ref int location1, int value, Func<int, int, int> update)
\n{
\n int current = location1;
\n while (true)
\n {
\n int newValue = update(current, value);
\n int oldValue = CompareExchange(ref location1, newValue, current);
\n if (oldValue == current)
\n {
\n return oldValue;
\n }
\n current = oldValue;
\n }
\n}<\/p>\n

such that And could be implemented then like:<\/p>\n

public static int And(ref int location1, int value) =>
\n CompareExchange(ref location1, value, static (current, value) => current & value)<\/p>\n

The approach employed by And is reasonable when there\u2019s nothing better you can do, but as it turns out, modern hardware platforms have single instructions capable of performing such an interlocked and and or in a much more efficient manner. The JIT already handled this for Arm because the instructions on Arm have semantics that very closely align with the semantics of Interlocked.And and Interlocked.Or. On x86\/64, however, the relevant instruction sequence (lock and or lock or) doesn\u2019t enable accessing the original value atomically replaced, whereas And\/Or require that as part of their definition. Luckily, most uses of Interlocked.And\/Interlocked.Or don\u2019t actually need the return value. For example, SafeHandle.SetHandleAsInvalid simply wants to atomically OR an additional flag into some bit flags, ignoring the result of Or:<\/p>\n

public void SetHandleAsInvalid()
\n{
\n Interlocked.Or(ref _state, StateBits.Closed);
\n GC.SuppressFinalize(this);
\n}<\/p>\n

And luckily, the JIT can see that it\u2019s ignoring the result. As such, on x86\/64, the JIT can use the optimal sequence when it can see that the result isn\u2019t being used, and even if it is being used, it can still emit a slightly more concise instruction sequence than would have naturally resulted from our open-coded implementation:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int _location;<\/p>\n

[Benchmark] public void Test_ResultNotUsed() => Interlocked.And(ref _location, 42);
\n [Benchmark] public int Test_ResultUsed() => Interlocked.And(ref _location, 42);
\n}
\n\/\/ .NET 8
\n; Tests.Test_ResultNotUsed()
\n push rbp
\n sub rsp,10
\n lea rbp,[rsp+10]
\n add rdi,8
\n mov eax,[rdi]
\nM00_L00:
\n mov ecx,eax
\n and ecx,2A
\n mov [rbp-4],eax
\n lock cmpxchg [rdi],ecx
\n mov ecx,[rbp-4]
\n cmp eax,ecx
\n je short M00_L01
\n mov ecx,eax
\n mov eax,ecx
\n jmp short M00_L00
\nM00_L01:
\n add rsp,10
\n pop rbp
\n ret
\n; Total bytes of code 47<\/p>\n

; Tests.Test_ResultUsed()
\n push rbp
\n sub rsp,10
\n lea rbp,[rsp+10]
\n add rdi,8
\n mov eax,[rdi]
\nM00_L00:
\n mov ecx,eax
\n and ecx,2A
\n mov [rbp-4],eax
\n lock cmpxchg [rdi],ecx
\n mov ecx,[rbp-4]
\n cmp eax,ecx
\n je short M00_L01
\n mov ecx,eax
\n mov eax,ecx
\n jmp short M00_L00
\nM00_L01:
\n add rsp,10
\n pop rbp
\n ret
\n; Total bytes of code 47<\/p>\n

\/\/ .NET 9
\n; Tests.Test_ResultNotUsed()
\n add rdi,8
\n mov eax,2A
\n lock and [rdi],eax
\n ret
\n; Total bytes of code 13<\/p>\n

; Tests.Test_ResultUsed()
\n add rdi,8
\n mov ecx,2A
\n mov eax,[rdi]
\nM00_L00:
\n mov edx,eax
\n and edx,ecx
\n lock cmpxchg [rdi],edx
\n jne short M00_L00
\n ret
\n; Total bytes of code 22<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size<\/p>\n

Test_ResultNotUsed
\n.NET 8.0
\n6.630 ns
\n1.00
\n47 B<\/p>\n

Test_ResultNotUsed
\n.NET 9.0
\n3.132 ns
\n0.47
\n13 B<\/p>\n

Test_ResultUsed
\n.NET 8.0
\n6.853 ns
\n1.00
\n47 B<\/p>\n

Test_ResultUsed
\n.NET 9.0
\n6.435 ns
\n0.94
\n22 B<\/p>\n

Locks and interlocked operations are about coordinating between operations, at a relatively low level. There are higher level coordination constructs as well; that\u2019s effectively what Task is, providing a representation for a piece of work with which you can later join. Such joining can be accomplished with await along with a myriad of APIs that faciliate joining with tasks in various ways. In that regard, one of my favorite new APIs in .NET 9 is on Task: Task.WhenEach. I like it because it utilizes newer language features to cleanly solve a problem that we wanted to solve over a decade ago when Task was originally introduced, and the lack of it has led to folks writing code with known pits of failure.<\/p>\n

Task.WhenAll is fairly easy to understand: you give it a collection of tasks, and the task it returns will complete only when all of the constituent tasks have completed:<\/p>\n

await Task.WhenAll([t1, t2, t3]);
\n… \/\/ only get here when t1, t2, and t3 have all completed successfully<\/p>\n

Task.WhenAny is a bit more complex, in that it returns when any of the constituent tasks has completed, and it gives you back a reference to that task:<\/p>\n

Task tCompleted = await Task.WhenAny([t1, t2, t3]);
\n… \/\/ tCompleted is either t1, t2, or t3, and will be completed here<\/p>\n

and you can then explicitly join with that returned task to observe any exceptions it may have incurred or consume its result value if it has one. But what do you then do to join with the remaining two tasks? You might end up writing code something like this:<\/p>\n

List<Task> tasks = new() { t1, t2, t3 };
\nwhile (tasks.Count > 0)
\n{
\n Task completed = await Task.WhenAny(tasks);
\n Handle(completed);
\n tasks.Remove(completed);
\n}<\/p>\n

That\u2019s not terribly hard, but it\u2019s also not terribly efficient. Or, rather, for larger number of tasks, it\u2019s terribly inefficient, as it\u2019s an O(N^2) algorithm. Some of the complexity is likely obvious: you\u2019ve got a loop and inside that loop you\u2019ve got a List<T>.Remove call, which will do an O(N) walk of the list looking for the target element to remove: there\u2019s the O(N^2). But, there\u2019s actually another less obvious O(N) operation in the loop: the WhenAny itself. Every call to WhenAny needs to hook a continuation up to each of the constituent Task objects. (There are of course cheaper ways to implement this functionality than using such a WhenAny, but they\u2019re all more complicated and thus not the answers towards which folks have gravitated.)<\/p>\n

Enter Task.WhenEach. WhenEach\u2018s purpose is to make consuming tasks as they complete both simple and efficient. To do so, rather than returning a Task<Task> as does WhenAny, it returns an IAsyncEnumerable<Task>, so one can simply iterate through the completing tasks as they complete.<\/p>\n

await foreach (Task completed in Task.WhenEach([t1, t2, t3]))
\n{
\n Debug.Assert(completed.IsCompleted);
\n Handle(completed);
\n}<\/p>\n

It\u2019s a little hard to get a good applies-to-apples comparison of the overhead here, but this benchmark is a reasonable approximation:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Params(10, 1_000)]
\n public int Count { get; set; }<\/p>\n

[Benchmark]
\n public async Task WithWhenAny()
\n {
\n var tcs = Enumerable.Range(0, Count).Select(_ => new TaskCompletionSource()).ToList();<\/p>\n

List<Task> tasks = tcs.Select(t => t.Task).ToList();
\n tcs[^1].SetResult();
\n while (tasks.Count > 0)
\n {
\n Task completed = await Task.WhenAny(tasks);
\n tasks.Remove(completed);
\n tcs.RemoveAt(tcs.Count – 1);<\/p>\n

if (tasks.Count == 0) break;
\n tcs[^1].SetResult();
\n }
\n }<\/p>\n

[Benchmark]
\n public async Task WithWhenEach()
\n {
\n var tcs = Enumerable.Range(0, Count).Select(_ => new TaskCompletionSource()).ToList();<\/p>\n

int remaining = tcs.Count – 1;
\n tcs[remaining].SetResult();
\n await foreach (Task completed in Task.WhenEach(tcs.Select(t => t.Task)))
\n {
\n if (remaining == 0) break;
\n tcs[–remaining].SetResult();
\n }
\n }
\n}<\/p>\n

Method
\nCount
\nMean
\nAllocated<\/p>\n

WithWhenAny
\n10
\n3.232 us
\n3.47 KB<\/p>\n

WithWhenEach
\n10
\n1.223 us
\n1.43 KB<\/p>\n

WithWhenAny
\n1000
\n20,082.683 us
\n4207.12 KB<\/p>\n

WithWhenEach
\n1000
\n102.759 us
\n94.24 KB<\/p>\n

WhenAll also gets a bit cheaper, in a couple of ways. dotnet\/runtime#93953<\/a> utilizes a trick employed elsewhere in Task in .NET 8, which is to use its used m_stateObject field (unused because there\u2019s no way to set it with WhenAll) to store some of the state that previously had a dedicated field (a field for storing information about constituent tasks that failed or were canceled). That means the Task object WhenAll returns gets 8 bytes smaller (on 64-bit). On top of that, dotnet\/runtime#101308<\/a> adds new ReadOnlySpan<T>-based overloads to a bunch of methods, including Task.WhenAll. This enables passing in any number of tasks without needing to allocate.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public async Task WhenAll()
\n {
\n var atmb1 = new AsyncTaskMethodBuilder();
\n var atmb2 = new AsyncTaskMethodBuilder();
\n Task whenAll = Task.WhenAll([atmb1.Task, atmb2.Task]);
\n atmb1.SetResult();
\n atmb2.SetResult();
\n await whenAll;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

WhenAll
\n.NET 8.0
\n123.8 ns
\n1.00
\n264 B
\n1.00<\/p>\n

WhenAll
\n.NET 9.0
\n103.8 ns
\n0.86
\n216 B
\n0.82<\/p>\n

There are some other interesting performance improvements in threading in .NET 9 as well.<\/p>\n

Debugger.NotifyOfCrossThreadDependency.<\/strong> This is a big deal. When you\u2019re debugging a .NET process and you break in the debugger, it pauses all threads in the debuggee process so that nothing is making forward progress while you examine state. However, .NET debuggers, like the one in Visual Studio, support invoking properties and methods in the debuggee while debugging. That can be a big problem if the functionality being invoked relies on one of those paused threads to do something, e.g. if the property you access tries to take a lock that\u2019s held by another thread or tries to Wait on a Task. To mitigate problems here, the Debugger.NotifyOfCrossThreadDependency method exists. Functionality that relies on another thread to do something can call NotifyOfCrossThreadDependency; if there\u2019s no debugger attached, it\u2019s a nop, but if there is a debugger attached, this signals the problem to the debugger, which can then react accordingly. The Visual Studio debugger reacts by stopping the evaluation but then by offering an opt-in option of \u201cslipping\u201d all threads, unpausing all threads until the evaluated operation completes, at which point all threads will be paused again, thereby again trying to mitigate any problems that might occur from the cross-thread dependency. NotifyOfCrossThreadDependency is generally not used by application code, but it\u2019s used in a few critical choke points in the core libraries, in particular throughout System.Threading and the infrastructure for async\/await. That means, for example, that this method is being called any time you await a Task that\u2019s not yet completed. And, unfortunately, while the method is a cheap nop when the debugger isn\u2019t attached, historically it\u2019s been fairly expensive when the debugger is attached, to the point where it can meaningfully impact a developer\u2019s experience in the tool. Thankfully, .NET 9 addresses this with dotnet\/runtime#101864<\/a>, which significantly improves the performance of NotifyOfCrossThreadDependency when a debugger is attached. We can see this with a low-tech benchmark. Replace the contents of your Program.cs with this:
\nusing System.Diagnostics;<\/p>\n

const int Iters = 100_000;
\nStopwatch sw = new();
\nwhile (true)
\n{
\n sw.Restart();
\n for (int i = 0; i < Iters; i++)
\n {
\n Debugger.NotifyOfCrossThreadDependency();
\n }
\n sw.Stop();
\n Console.WriteLine($”{sw.Elapsed.TotalMicroseconds \/ Iters:N3}us”);
\n}<\/p>\n

open the project in Visual Studio, ensure that .NET 8 is selected as the target framework and that you\u2019re targeting Release:\n<\/p>\n

and run with the debugger attached (F5, not ctrl-F5). When I do that, I see numbers like this (on Windows):<\/p>\n

48.360us
\n45.281us
\n46.714us
\n46.945us
\n46.525us<\/p>\n

Then change the target framework to be .NET 9, and run with the debugger attached again. I then see numbers like this:<\/p>\n

1.973us
\n1.713us
\n1.714us
\n1.871us
\n1.963us<\/p>\n

While such an improvement shouldn\u2019t impact your production workloads, it can make an impactful difference to your productivity as a developer.<\/p>\n

Volatile.<\/strong> A \u201cmemory model\u201d is a description of how threads interact with memory and what guarantees are made about how different threads produce and consume changes in shared memory. Reads and writes to memory from a single thread are guaranteed to be observed by that thread in the order they occurred, but once multiple threads enter the picture, it\u2019s up to the memory model to define what behaviors can be relied on and which can\u2019t. For example, if there are two fields, _a and _b, both of which start as 0, and if one thread does:
\n_a = 1;
\n_b = 2;<\/p>\n

and then another does:<\/p>\n

while (_b != 2);
\nAssert(_a == 1);<\/p>\n

is that assert guaranteed to always pass? It depends on the memory model, and whether the writes from thread 1 might get reordered (by any of the involved compilers or even by the hardware) such that the write to _b became visible to thread 2 before the write to _a. For the longest time, the only official memory model for .NET was the one defined by the ECMA 335<\/a> specification, but real implementations, including coreclr, generally had stronger guarantees than what ECMA detailed. Thankfully, the official .NET memory model<\/a> has now been documented. However, some of the practices that were being employed in the core libraries (due to defensive coding or uncertainty of the memory model or out-of-date requirements) are no longer necessary. One of the main tools available for folks coding at a level where memory model is relevant is the volatile keyword \/ the Volatile class. Marking a field as volatile causes any reads or writes of that field to be considered \u201cvolatile,\u201d just as does using Volatile.Read\/Volatile.Write to perform that read or write. Making the read or write volatile means it prevents certain kinds of \u201cmovement,\u201d e.g. if both _a and _b in the previous example were marked as volatile, the assert would always pass. Marking fields or operations as volatile can come with an expense, depending on the circumstance and the target platform. For example, it can restrict the C# compiler and the JIT compiler from performing certain optimizations. Let\u2019s take a simple example. This code:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private volatile int _volatile;
\n private int _nonVolatile;<\/p>\n

[Benchmark]
\n public int UsingVolatile() => _volatile + _volatile;<\/p>\n

[Benchmark]
\n public int UsingNonVolatile() => _nonVolatile + _nonVolatile;
\n}<\/p>\n

results in this assembly on .NET 9:<\/p>\n

; Tests.UsingVolatile()
\n mov eax,[rdi+8]
\n add eax,[rdi+8]
\n ret
\n; Total bytes of code 7<\/p>\n

; Tests.UsingNonVolatile()
\n mov eax,[rdi+0C]
\n add eax,eax
\n ret
\n; Total bytes of code 6<\/p>\n

The important difference between the two assembly blocks is in the add instruction. In the UsingVolatile method, the first instruction is loading the value from memory stored at address rcx+8, and then re-reading that same rcx+8 memory location again to add whatever is there with what it just read. In UsingNonVolatile, it starts the same way, reading the value stored at rcx+0xc, but then the add isn\u2019t doing another memory read and is instead just doubling the value stored in the register. One of the effects of volatile requiring that reads can\u2019t be moved is also that they can\u2019t be removed<\/em>, which means both reads in the code are required to stay. Here\u2019s another example:<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private volatile bool _volatile;
\n private bool _nonVolatile;<\/p>\n

[Benchmark]
\n public int CountVolatile()
\n {
\n int count = 0;
\n while (_volatile) count++;
\n return count;
\n }<\/p>\n

[Benchmark]
\n public int CountNonVolatile()
\n {
\n int count = 0;
\n while (_nonVolatile) count++;
\n return count;
\n }
\n}<\/p>\n

which on .NET 9 produces this assembly:<\/p>\n

; Tests.CountVolatile()
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\n cmp byte ptr [rdi+8],0
\n jne short M00_L01
\nM00_L00:
\n pop rbp
\n ret
\nM00_L01:
\n inc eax
\n cmp byte ptr [rdi+8],0
\n jne short M00_L01
\n jmp short M00_L00
\n; Total bytes of code 24<\/p>\n

; Tests.CountNonVolatile()
\n push rbp
\n mov rbp,rsp
\n xor eax,eax
\n cmp byte ptr [rdi+9],0
\n jne short M00_L00
\n pop rbp
\n ret
\nM00_L00:
\n jmp short M00_L00
\n; Total bytes of code 16<\/p>\n

They look somewhat similar, in fact the first five instructions are almost identical, but there\u2019s a critical difference. In both cases, the bool value is being loaded and checked to see if it\u2019s false (the cmp against 0 followed by a conditional jump), in which case the implementations both fall through to the ending ret to exit out of the method. The compiler is rewriting the while (cond) { … } loop to instead be more like an if (cond) { do { … } while(cond); }, so this initial test is the one for that if (cond). But then things diverge meaningfully. CountVolatile then proceeds to do the do while equivalent, incrementing the count (stored in eax), reading _volatile and comparing it to 0 (false), and if it\u2019s still true, jumping back up to loop again. So basically what you\u2019d expect. But now look at CountNonVolatile. The loop is just this:<\/p>\n

M00_L00:
\n jmp short M00_L00<\/p>\n

It\u2019s now sitting in an infinite loop, with an unconditional jump back to the same jmp instruction, looping forever. That\u2019s because the JIT was able to hoist the read of _nonVolatile out of the loop. It then also sees that no one will ever observe count\u2018s changed value, so it also elides the increment. At which point it\u2019s more like if I\u2019d written this C#:<\/p>\n

public int CountNonVolatile()
\n{
\n int count = 0;
\n if (_nonVolatile)
\n {
\n while (true);
\n }<\/p>\n

return count;
\n}<\/p>\n

That hoisting can\u2019t be done when the field is volatile, because it can\u2019t reorder or remove reads associated with the field. But with _nonVolatile, nothing prevents that. On multiple occasions I\u2019ve seen folks trying to engage in low-lock programming experience the ramifications of this latter example: they\u2019ll be using some bool to signal to a consumer that it should break out of the loop, but the bool isn\u2019t volatile, and the consumer then never notices when the producer eventually sets it.<\/p>\n

Those are examples of the ramifications of volatile in terms of what the C# or JIT compiler are constrained from doing. But there are also things the JIT needs<\/em> to do (rather than avoid) in order to ensure the hardware can respect the requirements put in place by the developer. On some hardware, like x64, the memory model of the hardware is relatively \u201cstrong,\u201d meaning it doesn\u2019t do most of the kinds of reorderings that volatile inhibits, and therefore you won\u2019t see anything emitted into the assembly code by the JIT to help the hardware enforce the constraints. On other hardware, like Arm64, though, the hardware has a relatively \u201cweaker\u201d model, meaning it allows more of these kinds of reorderings, and as a result, the JIT needs to actively inhibit such reorderings by inserting appropriate \u201cmemory barriers\u201d into the code. On Arm, this shows up with instructions like dmb (\u201cdata memory barrier\u201d). Such barriers have some overhead associated with them.<\/p>\n

For all of these reasons, fewer volatiles is good for performance, but of course you need to ensure you have enough volatiles to actually achieve a correct application (with the best answer being avoid writing lock-lock code in the first place, and then you never need to know or think about volatile). It\u2019s a balance. Luckily, and bringing us full circle to why we\u2019re talking about this, there are a set of common cases where volatile used to be recommended but now that we have a well-defined memory model, those uses are obsolete. Removing them can help to avoid a layer of thin cost across the code. So dotnet\/runtime#100969<\/a> and dotnet\/runtime#101346<\/a> removed a bunch of volatile usage where it was no longer necessary. Almost all of these uses were as part of lazy initialization of reference types, e.g.<\/p>\n

private volatile MyReferenceType? _instance;<\/p>\n

public MyReferenceType Instance => _instance ??= new MyReferenceType();<\/p>\n

which if we expand that out to not use ??= looks something like this:<\/p>\n

private MyReferenceType? _instance;<\/p>\n

public MyReferenceType Instance
\n{
\n get
\n {
\n MyReferenceType? instance = _instance;
\n if (instance is null)
\n {
\n _instance = instance = new MyReferenceType();
\n }<\/p>\n

return instance;
\n }
\n}<\/p>\n

The reason for the volatile here would be two-fold, one for the part of the operation that reads and one for the part of the operation that writes. Without the volatile, the concern would be that one of the compilers or the hardware could actually \u201cintroduce a read,\u201d effectively making the code equivalent to this:<\/p>\n

private MyReferenceType? _instance;<\/p>\n

public MyReferenceType Instance
\n{
\n get
\n {
\n MyReferenceType? instance = _instance;
\n if (_instance is null)
\n {
\n _instance = instance = new MyReferenceType();
\n }<\/p>\n

return instance;
\n }
\n}<\/p>\n

If that were to happen, there\u2019s a problem that between the two reads, the value of _instance could go from null to non-null, in which case instance could be assigned null, _instance is null might then be false, and return instance would return null. However, the .NET memory model explicitly states \u201cReads cannot be introduced.\u201d Then there\u2019s the concern about the write. The concern there that leads to volatile being used is the initialization that happens inside of MyReferenceType. Imagine if MyReferenceType were defined like this:<\/p>\n

class MyReferenceType()
\n{
\n internal int _value;
\n public MyReferenceType() => _value = 42;
\n}<\/p>\n

The question then becomes \u201cis it possible for the write to _value inside of the constructor to be viewed by another thread after<\/em> the write of the instance to _instance\u201c? In other words, could the code logically become the equivalent of this:<\/p>\n

private MyReferenceType? _instance;<\/p>\n

public MyReferenceType Instance
\n{
\n get
\n {
\n MyReferenceType? instance = _instance;
\n if (_instance is null)
\n {
\n _instance = instance = RuntimeHelpers.GetUninitializedObject(typeof(MyReferenceType));
\n instance._value = 42;
\n }<\/p>\n

return instance;
\n }
\n}<\/p>\n

If that could happen, then two threads could be racing to access Instance, one of them could get as far as setting _instance (but _value hasn\u2019t been set yet), then another thread could access Instance, see _instance as non-null, and start using it, even though _value hasn\u2019t yet been initialized. Thankfully here as well, the .NET memory model doesn\u2019t allow such transformations, explicitly covering this point:<\/p>\n

\u201cObject assignment to a location potentially accessible by other threads is a release with respect to accesses to the instance\u2019s fields\/elements and metadata. An optimizing compiler must preserve the order of object assignment and data-dependent memory accesses. The motivation is to ensure that storing an object reference to shared memory acts as a \u201ccommitting point\u201d to all modifications that are reachable through the instance reference.\u201d<\/p>\n

Phew!<\/p>\n

ManagedThreadId.<\/strong> dotnet\/runtime#91232<\/a> is fun, in a \u201cwhy didn\u2019t we already do this\u201d sort of way. Thread.ManagedThreadId is implemented as an internal call (an FCALL) into the runtime, resulting in a call to ThreadNative::GetManagedThreadId, which in turn reads the thread object\u2019s m_ManagedThreadId field. At least, that\u2019s the field in the C definition of the object. The managed Thread object has corresponding fields at the exact location that are available for the C# code to use, in this case _managedThreadId. So what did this PR do? It removed those complicated gymnastics and just made the whole implementation be public int ManagedThreadId => _managedThreadId. (It\u2019s worth noting, though, that Thread.CurrentThread.ManagedThreadId was already previously recognized specially by the JIT, so this is only relevant when accessing the ManagedThreadId from some other Thread instance.) The main benefit of this is avoiding the extra function call, as the FCALL can\u2019t be inlined.
\nusing BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Thread _thread = Thread.CurrentThread;<\/p>\n

[Benchmark]
\n public int GetID() => _thread.ManagedThreadId;
\n}
\n\/\/ .NET 8
\n; Tests.GetID()
\n mov rdi,[rdi+8]
\n cmp [rdi],edi
\n jmp near ptr System.Threading.Thread.get_ManagedThreadId()
\n; Total bytes of code 11
\n**Extern method**
\nSystem.Threading.Thread.get_ManagedThreadId()<\/p>\n

\/\/ .NET 9
\n; Tests.GetID()
\n mov rax,[rdi+8]
\n mov eax,[rax+34]
\n ret
\n; Total bytes of code 8<\/p>\n

Ports to NativeAOT.<\/strong> Previous releases of .NET enabled inlining the fast path of thread-local state (TLS) access on coreclr. With dotnet\/runtime#104282<\/a>, dotnet\/runtime#89472<\/a>, and dotnet\/runtime#97910<\/a>, this improvement comes to NativeAOT as well. Similarly, dotnet\/runtime#103675<\/a> ports coreclr\u2019s \u201cyield normalization\u201d implementation to NativeAOT; this is in support of enabling the runtime to measure the cost of various pause instructions, which can then be used as part of tuning spinning and spin waiting.
\nStartup time.<\/strong> Performance improvements related to threading are generally about steady-state throughput improvements, e.g. reducing synchronization costs while processing requests. That\u2019s what makes dotnet\/runtime#106724<\/a> from @harisokanovic<\/a> so interesting, in that it\u2019s instead about reducing startup overheads of a process using .NET on Linux. The GC uses the equivalent of a process-wide memory barrier (also exposed publicly as Interlocked.MemoryBarrierProcessorWide) to ensure that all threads involved in a collection see a consistent state. On Linux, implementing this method efficiently involves using the membarrier system call with MEMBARRIER_CMD_PRIVATE_EXPEDITED, and using that requires the same syscall to have been made earlier on with MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED, which really means doing so at startup. However, the Linux kernel has some optimizations that make this registration a lot cheaper to use when there\u2019s only one thread in the process. The way it was being used in .NET previously guaranteed there would be multiple. This PR changed where this initialization was performed in order to maximize the possibility of there only being the single thread in the process, which in turn makes startup faster. The improvement was upwards of 10ms on various systems on which it was measured, which is a large percentage of a .NET process\u2019 startup overhead on Linux.<\/p>\n

Reflection<\/h2>\n

Reflection is a very powerful (though sometimes overused) capability of .NET that enables code to load and introspect .NET assemblies and invoke their functionality. It is used in all manner of library and application, including by the core .NET libraries themselves, and it\u2019s important that we continue to find ways to reduce the overheads associated with reflection.<\/p>\n

Several PRs in .NET 9 whittle away at some of the allocation overheads in reflection. dotnet\/runtime#92310<\/a> and dotnet\/runtime#93115<\/a> avoid some defensive array copies by instead handing around ReadOnlySpan<T> instances, while dotnet\/runtime#95952<\/a> removed a use of string.Split that turned out to only be used with constants and thus could be replaced by just manually splitting those constants. But a more interesting and impactful addition comes from dotnet\/runtime#97683<\/a>, which added an allocation-free way to get the invocation list from a delegate. Delegates in .NET are \u201cmulticast,\u201d meaning a single delegate instance might actually represent multiple methods to be invoked; this is how .NET events are implemented. If I invoke a delegate, the delegate implementation handles invoking each constituent method, sequentially, in turn. But what if I want to customize the invocation logic? Maybe I want to wrap each individual method in a try\/catch, or maybe I want to track the return values from all of the methods rather than just the last, or some such behavior. To achieve that, delegates expose a way to get an array of delegates, one for each method that\u2019s part of the original. So, if I have:<\/p>\n

Action action = () => Console.Write(“A “);
\naction += () => Console.Write(“B “);
\naction += () => Console.Write(“C “);
\naction();<\/p>\n

that\u2019ll print out “A B C “, and if I have:<\/p>\n

Action action = () => Console.Write(“A “);
\naction += () => Console.Write(“B “);
\naction += () => Console.Write(“C “);<\/p>\n

Delegate[] actions = action.GetInvocationList();
\nfor (int i = 0; i < actions.Length; ++i)
\n{
\n Console.Write($”{i}: “);
\n ((Action)actions[i])();
\n Console.WriteLine();
\n}<\/p>\n

that\u2019ll print out:<\/p>\n

0: A
\n1: B
\n2: C<\/p>\n

However, that GetInvocationList needs to allocate. Now in .NET 9, there\u2019s the new Delegate.EnumerateInvocationList<TDelegate> method, which returns a struct-based enumerable for iterating through the delegates rather than needing to allocate a new array to store them all.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Action _action;
\n private int _count;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _action = () => _count++;
\n _action += () => _count += 2;
\n _action += () => _count += 3;
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public void GetInvocationList()
\n {
\n foreach (Action action in _action.GetInvocationList())
\n {
\n action();
\n }
\n }<\/p>\n

[Benchmark]
\n public void EnumerateInvocationList()
\n {
\n foreach (Action action in Delegate.EnumerateInvocationList(_action))
\n {
\n action();
\n }
\n }
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

GetInvocationList
\n32.11 ns
\n1.00
\n48 B
\n1.00<\/p>\n

EnumerateInvocationList
\n11.07 ns
\n0.34
\n\u2013
\n0.00<\/p>\n

Reflection is particularly important with libraries involved in dependency injection, where object construction is frequently done in a more dynamic fashion. ActivatorUtilities.CreateInstance plays a key role there, and has also seen improvements from allocation reduction. dotnet\/runtime#99383<\/a>, in particular, helped to significantly reduce allocation by employing the ConstructorInvoker type introduced in .NET 8, and by piggybacking on the changes from dotnet\/runtime#99175<\/a> to cut down on the number of constructors it needs to examine.<\/p>\n

\/\/ Add a <PackageReference Include=”Microsoft.Extensions.DependencyInjection” Version=”8.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Environments;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Running;
\nusing Microsoft.Extensions.DependencyInjection;<\/p>\n

var config = DefaultConfig.Instance
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core80)
\n .WithNuGet(“Microsoft.Extensions.DependencyInjection”, “8.0.0”)
\n .WithNuGet(“Microsoft.Extensions.DependencyInjection.Abstractions”, “8.0.1”).AsBaseline())
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core90)
\n .WithNuGet(“Microsoft.Extensions.DependencyInjection”, “9.0.0-rc.1.24431.7”)
\n .WithNuGet(“Microsoft.Extensions.DependencyInjection.Abstractions”, “9.0.0-rc.1.24431.7”));
\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, config);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n private IServiceProvider _serviceProvider = new ServiceCollection().BuildServiceProvider();<\/p>\n

[Benchmark]
\n public MyClass Create() => ActivatorUtilities.CreateInstance<MyClass>(_serviceProvider, 1, 2, 3);<\/p>\n

public class MyClass
\n {
\n public MyClass() { }
\n public MyClass(int a) { }
\n public MyClass(int a, int b) { }
\n [ActivatorUtilitiesConstructor]
\n public MyClass(int a, int b, int c) { }
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Create
\n.NET 8.0
\n163.60 ns
\n1.00
\n288 B
\n1.00<\/p>\n

Create
\n.NET 9.0
\n83.46 ns
\n0.51
\n144 B
\n0.50<\/p>\n

The aforementioned ConstructorInvoker, along with a MethodInvoker, was introduced in .NET 8 as a way to cache first-use information to enable all subsequent operations to be much faster. Without introducing a new public FieldInvoker, dotnet\/runtime#98199<\/a> is able to achieve similar levels of speedup for field access via a FieldInfo by employing an internal FieldAccessor that\u2019s cached onto the FieldInfo object (dotnet\/runtime#92512<\/a> also helped here by moving some native runtime implementations back up into C#). Varying levels of large speedups are achieved depending on the exact nature of the field being accessed.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Reflection;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n private static object s_staticReferenceField = new object();
\n private object _instanceReferenceField = new object();
\n private static int s_staticValueField = 1;
\n private int _instanceValueField = 2;
\n private object _obj = new();<\/p>\n

private FieldInfo _staticReferenceFieldInfo = typeof(Tests).GetField(nameof(s_staticReferenceField), BindingFlags.NonPublic | BindingFlags.Static)!;
\n private FieldInfo _instanceReferenceFieldInfo = typeof(Tests).GetField(nameof(_instanceReferenceField), BindingFlags.NonPublic | BindingFlags.Instance)!;
\n private FieldInfo _staticValueFieldInfo = typeof(Tests).GetField(nameof(s_staticValueField), BindingFlags.NonPublic | BindingFlags.Static)!;
\n private FieldInfo _instanceValueFieldInfo = typeof(Tests).GetField(nameof(_instanceValueField), BindingFlags.NonPublic | BindingFlags.Instance)!;<\/p>\n

[Benchmark] public object? GetStaticReferenceField() => _staticReferenceFieldInfo.GetValue(null);
\n [Benchmark] public void SetStaticReferenceField() => _staticReferenceFieldInfo.SetValue(null, _obj);<\/p>\n

[Benchmark] public object? GetInstanceReferenceField() => _instanceReferenceFieldInfo.GetValue(this);
\n [Benchmark] public void SetInstanceReferenceField() => _instanceReferenceFieldInfo.SetValue(this, _obj);<\/p>\n

[Benchmark] public int GetStaticValueField() => (int)_staticValueFieldInfo.GetValue(null)!;
\n [Benchmark] public void SetStaticValueField() => _staticValueFieldInfo.SetValue(null, 3);<\/p>\n

[Benchmark] public int GetInstanceValueField() => (int)_instanceValueFieldInfo.GetValue(this)!;
\n [Benchmark] public void SetInstanceValueField() => _instanceValueFieldInfo.SetValue(this, 4);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

GetStaticReferenceField
\n.NET 8.0
\n24.839 ns
\n1.00<\/p>\n

GetStaticReferenceField
\n.NET 9.0
\n1.720 ns
\n0.07<\/p>\n

SetStaticReferenceField
\n.NET 8.0
\n41.025 ns
\n1.00<\/p>\n

SetStaticReferenceField
\n.NET 9.0
\n6.964 ns
\n0.17<\/p>\n

GetInstanceReferenceField
\n.NET 8.0
\n29.595 ns
\n1.00<\/p>\n

GetInstanceReferenceField
\n.NET 9.0
\n5.960 ns
\n0.20<\/p>\n

SetInstanceReferenceField
\n.NET 8.0
\n31.753 ns
\n1.00<\/p>\n

SetInstanceReferenceField
\n.NET 9.0
\n9.577 ns
\n0.30<\/p>\n

GetStaticValueField
\n.NET 8.0
\n43.847 ns
\n1.00<\/p>\n

GetStaticValueField
\n.NET 9.0
\n36.011 ns
\n0.82<\/p>\n

SetStaticValueField
\n.NET 8.0
\n39.462 ns
\n1.00<\/p>\n

SetStaticValueField
\n.NET 9.0
\n10.396 ns
\n0.26<\/p>\n

GetInstanceValueField
\n.NET 8.0
\n45.125 ns
\n1.00<\/p>\n

GetInstanceValueField
\n.NET 9.0
\n39.104 ns
\n0.87<\/p>\n

SetInstanceValueField
\n.NET 8.0
\n36.664 ns
\n1.00<\/p>\n

SetInstanceValueField
\n.NET 9.0
\n13.571 ns
\n0.37<\/p>\n

Of course, if you can avoid using these more expensive reflection approaches in the first place, that\u2019s very desirable. One reason for using reflection is to access private members of other types, and while that\u2019s a scary thing to do and generally something to be avoided, there are valid cases for it where having an efficient solution is highly desirable. .NET 8 added such a mechanism in [UnsafeAccessor], which enables a type to declare a method that effectively serves as direct access to a member of another type. So, for example, with this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Reflection;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n private MyClass _myClass = new MyClass(new List<int>() { 1, 2, 3 });
\n private FieldInfo _fieldInfo = typeof(MyClass).GetField(“_list”, BindingFlags.NonPublic | BindingFlags.Instance)!;<\/p>\n

private static class Accessors
\n {
\n [UnsafeAccessor(UnsafeAccessorKind.Field, Name = “_list”)]
\n public static extern ref object GetList(MyClass myClass);
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public object WithFieldInfo() => _fieldInfo.GetValue(_myClass)!;<\/p>\n

[Benchmark]
\n public object WithUnsafeAccessor() => Accessors.GetList(_myClass);
\n}<\/p>\n

public class MyClass(object list)
\n{
\n private object _list = list;
\n}<\/p>\n

I get this:<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

WithFieldInfo
\n.NET 8.0
\n27.5299 ns
\n1.00<\/p>\n

WithFieldInfo
\n.NET 9.0
\n4.0789 ns
\n0.15<\/p>\n

WithUnsafeAccessor
\n.NET 8.0
\n0.5005 ns
\n0.02<\/p>\n

WithUnsafeAccessor
\n.NET 9.0
\n0.5499 ns
\n0.02<\/p>\n

However, in .NET 8, this mechanism could only be used with non-generic members. Now in .NET 9, thanks to dotnet\/runtime#99468<\/a> and dotnet\/runtime#99830<\/a>, this capability now extends to generics, as well.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Reflection;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n private MyClass<int> _myClass = new MyClass<int>(new List<int>() { 1, 2, 3 });
\n private FieldInfo _fieldInfo = typeof(MyClass<int>).GetField(“_list”, BindingFlags.NonPublic | BindingFlags.Instance)!;
\n private static class Accessors<T>
\n {
\n [UnsafeAccessor(UnsafeAccessorKind.Field, Name = “_list”)]
\n public static extern ref List<T> GetList(MyClass<T> myClass);
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public List<int> WithFieldInfo() => (List<int>)_fieldInfo.GetValue(_myClass)!;<\/p>\n

[Benchmark]
\n public List<int> WithUnsafeAccessor() => Accessors<int>.GetList(_myClass);
\n}<\/p>\n

public class MyClass<T>(List<T> list)
\n{
\n private List<T> _list = list;
\n}<\/p>\n

Method
\nMean
\nRatio<\/p>\n

WithFieldInfo
\n4.4251 ns
\n1.00<\/p>\n

WithUnsafeAccessor
\n0.5147 ns
\n0.12<\/p>\n

Parsing that occurs as part of reflection, and in particular as part of type names, was also improved as part of some work to consolidate type name parsing into a reusable component. dotnet\/runtime#100094<\/a>\u2018s primary purpose wasn\u2019t to improve performance, but it ended up doing so, anyway.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public Type? Parse() =>
\n Type.GetType(“System.Collections.Generic.Dictionary`2[” +
\n “[System.Collections.Generic.List`1[” +
\n “[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ” +
\n “System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],” +
\n “[System.Collections.Generic.List`1[” +
\n “[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ” +
\n “System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ” +
\n “System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e”);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Parse
\n.NET 8.0
\n7.590 us
\n1.00
\n5.03 KB
\n1.00<\/p>\n

Parse
\n.NET 9.0
\n6.361 us
\n0.84
\n4.73 KB
\n0.94<\/p>\n

And then there are more intrinsics. In compiler speak, an \u201cintrinsic\u201d is something the compiler has \u201cintrinsic\u201d knowledge of, a fancy way of saying it\u2019s something the compiler implicitly knows about. This typically manifests as a method whose implementation is provided by the compiler, sometimes always or sometimes based on certain conditions. For example, string.Equals is attributed as [Intrinsic]: it has its own fully-functional implementation, but if the JIT sees that at least one of the inputs is a constant string, the JIT may emit its own optimized implementation for Equals that unrolls and vectorizes the comparison based on the exact value being compared.<\/p>\n

Several new members became intrinsics in .NET 9. dotnet\/runtime#96226<\/a> turns typeof(T).IsPrimitive into an intrinsic, which allows the JIT to supply a constant replacement for the expression, which in turn allows branches to be eliminated and possibly whole swaths of then dead code to follow. For example, as part of its code path for moving to the next value, Parallel.ForAsync has a code path that looks like this:<\/p>\n

if (typeof(T).IsPrimitive)
\n{
\n UseInterlockedCompareExchangeToAdvance();
\n}
\nelse
\n{
\n UseALockAroundAReadIncrementStoreToAdvance();
\n}<\/p>\n

With IsPrimitive as an intrinsic, that if\/else will reduce entirely to either:<\/p>\n

UseInterlockedCompareExchangeToAdvance();<\/p>\n

or<\/p>\n

UseALockAroundAReadIncrementStoreToAdvance();<\/p>\n

based on the nature of T.<\/p>\n

typeof(T).IsGenericType and typeof(T).GetGenericTypeDefinition were also made into intrinsics, by dotnet\/runtime#99555<\/a> and dotnet\/runtime#103528<\/a>, respectively. Imagine code like that in ASP.NET where it wants to special-case APIs that return Task<T> vs ValueTask<T> vs IAsyncEnumerable<T> vs T vs other types; it\u2019ll often use members like IsGenericType and GetGenericTypeDefinition (which will throw an exception if IsGenericType is false) to determine whether a concrete instantiation of a generic type is the one in question. With this benchmark:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public bool Test() => IsTaskT<Task<string>>();<\/p>\n

private static bool IsTaskT<T>() =>
\n typeof(T).IsGenericType &&
\n typeof(T).GetGenericTypeDefinition() == typeof(Task<>);
\n}<\/p>\n

on .NET 8 we end up with over 250 bytes of assembly code for implementing this operation. On .NET 9, we get just this:<\/p>\n

; Tests.Test()
\n mov eax,1
\n ret
\n; Total bytes of code 6<\/p>\n

The magic of intrinsics.<\/p>\n

Numerics<\/h2>\n

Primitive Types<\/h3>\n

The core data types in .NET sit at the very bottom of the stack and are used everywhere. It\u2019s thus a desire every release to whittle away at any overheads we can avoid. .NET 9 is no exception, where a multitude of PRs have gone into reducing overheads of various operations on these core types.<\/p>\n

Consider DateTime. When it comes to performance optimization, we typically focus on the happy path, on the \u201chot path,\u201d on the successful path. Exceptions already add significant expense to error paths, and are intended to be \u201cexceptional\u201d and relatively rare, and so we generally don\u2019t worry about an extra operation here or an extra allocation there. But, sometimes, one type\u2019s error path is another type\u2019s success path. This is especially true with Try methods, where failure is conveyed via a bool rather than with an expensive exception. As part of profiling a commonly-used .NET library, the profiler highlighted some unexpected allocations coming from DateTime handling, unexpected because we\u2019ve spent a lot of time over the years trying to eliminate allocations in this area of the code. The allocation, it turned out, was occurring on an error path, both with DateTime.Parse when an exception would be thrown, but also<\/em> with DateTime.TryParse when false would be returned. As it happened, deep in the call tree where parsing work is going on, if an error is encountered, the code stores information about the failure (e.g. a ParseFailureKind enum value); after unwinding the call stack back to the public method, Parse uses that to throw an appropriately-detailed exception, while TryParse just ignores it and returns false. But the way the code was written, that enum value would end up getting boxed when it was stored, resulting in an allocation as part of TryParse returning false. The consuming library was using TryParse on a bunch of different data primitive types as part of interpreting the data, e.g.<\/p>\n

if (int.TryParse(value, out int parsedInt32)) { … }
\nelse if (DateTime.TryParse(value, out DateTime parsedDateTime)) { … }
\nelse if (double.TryParse(value, out double parsedDouble)) { … }
\nelse if …<\/p>\n

such that its<\/em> success path might include the failure path from some number of these primitives\u2019 TryParse methods. dotnet\/runtime#91303<\/a> tweaked how the information is stored to avoid that boxing, while also reducing a bit of additional overhead along the way.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “input”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(“hello”)]
\n public bool TryParse(string input) => DateTime.TryParse(input, out _);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

TryParse
\n.NET 8.0
\n31.95 ns
\n1.00
\n24 B
\n1.00<\/p>\n

TryParse
\n.NET 9.0
\n25.96 ns
\n0.81
\n\u2013
\n0.00<\/p>\n

Both DateTime and TimeSpan also saw parsing and formatting gains from dotnet\/runtime#101640<\/a> from @lilinus<\/a>. The PR takes advantage of an existing internal CountDigits helper that was optimized in .NET 8 as part of integer parsing; it employs a lookup table to compute the number of digits that will be required for a number, doing so in just a few instructions. And it replaces a switch with a lookup table as part of computing powers of ten, replacing a method like Pow10_Old in this benchmark with one more like Pow10_New:<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “input”)]
\npublic class Tests
\n{
\n private int _pow = 3;<\/p>\n

[Benchmark(Baseline = true)]
\n public long Pow10_Old() =>
\n _pow switch
\n {
\n 0 => 1,
\n 1 => 10,
\n 2 => 100,
\n 3 => 1000,
\n 4 => 10000,
\n 5 => 100000,
\n 6 => 1000000,
\n _ => 10000000, \/\/ _pow will never be greater than 7
\n };<\/p>\n

[Benchmark]
\n public long Pow10_New()
\n {
\n ReadOnlySpan<int> powersOfTen =
\n [
\n 1,
\n 10,
\n 100,
\n 1000,
\n 10000,
\n 100000,
\n 1000000,
\n 10000000, \/\/ _pow will never be greater than 7
\n ];
\n return powersOfTen[_pow];
\n }
\n}<\/p>\n

The JIT is able to do a bit better job with the latter, for the former producing:<\/p>\n

; Tests.Pow10_Old()
\n push rbp
\n mov rbp,rsp
\nM00_L00:
\n mov ecx,[rdi+8]
\n cmp ecx,3
\n jne short M00_L02
\n mov edx,3E8
\nM00_L01:
\n movsxd rax,edx
\n pop rbp
\n ret
\nM00_L02:
\n cmp ecx,6
\n ja short M00_L03
\n mov edx,ecx
\n lea rax,[7F3D29A690E8]
\n mov eax,[rax+rdx*4]
\n lea rcx,[M00_L00]
\n add rax,rcx
\n jmp rax
\nM00_L03:
\n mov edx,989680
\n jmp short M00_L01
\n mov edx,0F4240
\n jmp short M00_L01
\n mov edx,186A0
\n jmp short M00_L01
\n mov edx,2710
\n jmp short M00_L01
\n mov edx,64
\n jmp short M00_L01
\n mov edx,0A
\n jmp short M00_L01
\n mov edx,1
\n jmp short M00_L01
\n; Total bytes of code 100<\/p>\n

but for the latter producing:<\/p>\n

; Tests.Pow10_New()
\n push rax
\n mov eax,[rdi+8]
\n cmp eax,8
\n jae short M00_L00
\n mov rcx,7F3CC0AE6018
\n movsxd rax,dword ptr [rcx+rax*4]
\n add rsp,8
\n ret
\nM00_L00:
\n call CORINFO_HELP_RNGCHKFAIL
\n int 3
\n; Total bytes of code 34<\/p>\n

The net result is a nice improvement to these operations, e.g.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _input = TimeSpan.FromMilliseconds(12345.6789).ToString();<\/p>\n

[Benchmark]
\n public TimeSpan Parse() => TimeSpan.Parse(_input);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Parse
\n.NET 8.0
\n137.55 ns
\n1.00<\/p>\n

Parse
\n.NET 9.0
\n117.78 ns
\n0.86<\/p>\n

Various operations on the primitive types were also improved across a plethora of PRs:<\/p>\n

Round.<\/strong> dotnet\/runtime#98186<\/a> from @MichalPetryka<\/a> optimized the various Math.Round and MathF.Round overloads (which are the same implementations as double.Round and float.Round).
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private double _value = 12345.6789;<\/p>\n

[Benchmark]
\n public double RoundDigits() => Math.Round(_value, 2);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

RoundDigits
\n.NET 8.0
\n1.6930 ns
\n1.00<\/p>\n

RoundDigits
\n.NET 9.0
\n0.3496 ns
\n0.21<\/p>\n

SinCos.<\/strong> dotnet\/runtime#103724<\/a> updated Math.SinCos and MathF.SinCos to use the internal RuntimeHelpers.IsKnownConstant intrinsic. This method enables code in CoreLib to check whether the argument to the method is coming in as a constant the JIT can see, at which point the implementation might choose to do something special. In this case, the Sin and Cos functions are already capable of producing constant results for constant input, so rather than doing the normal implementation, which tries to reuse most of the computation that\u2019s shared between Sin and Cos, it instead just calls to each, knowing that the constant input for each will result in a constant output overall.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public float Sum() => SumSinCos(123.456f);<\/p>\n

private float SumSinCos(float f)
\n {
\n (float sin, float cos) = MathF.SinCos(f);
\n return sin + cos;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size<\/p>\n

Sum
\n.NET 8.0
\n5.2719 ns
\n1.000
\n46 B<\/p>\n

Sum
\n.NET 9.0
\n0.0177 ns
\n0.003
\n9 B<\/p>\n

In cases like this, it\u2019s helpful to pay attention to the warnings BenchmarkDotNet issues:<\/p>\n

\/\/ * Warnings *
\nZeroMeasurement
\n Tests.Sum: Runtime=.NET 9.0, Toolchain=net9.0 -> The method duration is indistinguishable from the empty method duration<\/p>\n

The .NET 9 run is indistinguishable from an empty method because it is<\/em> an empty method, or at least a method that just returns a constant. We can see that by looking at the disassembly. The .NET 8 code has a few moves and loads and then calls to SinCos:<\/p>\n

; Tests.Sum()
\n push rax
\n vzeroupper
\n vmovss xmm0,dword ptr [7F7686979610]
\n lea rdi,[rsp+4]
\n lea rsi,[rsp]
\n call System.MathF.SinCos(Single, Single*, Single*)
\n vmovss xmm0,dword ptr [rsp+4]
\n vmovss xmm1,dword ptr [rsp]
\n vaddss xmm0,xmm0,xmm1
\n add rsp,8
\n ret
\n; Total bytes of code 46<\/p>\n

In contrast, here\u2019s .NET 9:<\/p>\n

; Tests.Sum()
\n vmovss xmm0,dword ptr [7F4D10FD9080]
\n ret
\n; Total bytes of code 9<\/p>\n

It\u2019s simply loading a value and returning it, since the whole operation compiled down to a constant.<\/p>\n

Enum.{Try}Parse.<\/strong> Interop scenarios drove the introduction of two new RuntimeHelpers APIs, SizeOf in dotnet\/runtime#100618<\/a> and Box in dotnet\/runtime#100561<\/a>. But, dotnet\/runtime#100846<\/a> was then able to utilize these APIs to optimize the implementation of the non-generic Enum.Parse and Enum.TryParse overloads, which give back the parsed enum value as object. This is a special kind of boxing, because the parse methods internally extract a numerical value but then need the boxed object to be of the enum type (rather than the numerical type) specified via the Type argument.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _input = “Monday”;<\/p>\n

[Benchmark]
\n public object Parse() => Enum.Parse(typeof(DayOfWeek), _input);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Parse
\n.NET 8.0
\n62.01 ns
\n1.00<\/p>\n

Parse
\n.NET 9.0
\n28.13 ns
\n0.45<\/p>\n

Integer Division.<\/strong> Consider this benchmark:
\n\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(5)]
\n public uint DivideBy4_UInt32(uint value) => value \/ 4;<\/p>\n

[Benchmark]
\n [Arguments(5)]
\n public int DivideBy4_Int32(int value) => value \/ 4;
\n}<\/p>\n

With the uint-based example, dividing by 4 is already optimized into a simple right shift, since for a uint, value \/ 4 and value >> 2 are functionally equivalent. However, that\u2019s not the case for an int, or at least, not always. For a non-negative int, the same optimization could be employed, but if the int is negative, for some values switching from value \/ 4 to value >> 2 would be functionally incorrect. Consider -5 \/ 4\u2026 the answer is -1. But -5 >> 2 is -2. Oops. So when you look at the assembly code for the int case (here on .NET 8), it\u2019s more complex:<\/p>\n

; Tests.DivideBy4_UInt32(UInt32)
\n mov eax,esi
\n shr eax,2
\n ret
\n; Total bytes of code 6<\/p>\n

; Tests.DivideBy4_Int32(Int32)
\n mov eax,esi
\n sar eax,1F
\n and eax,3
\n add eax,esi
\n sar eax,2
\n ret
\n; Total bytes of code 14<\/p>\n

Given that, you might hope that if the compiler could prove that the int was non-negative, it could still employ the simpler shifting:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(5)]
\n public int DivideBy4_Int32(int value) => value < 4 ? 0 : value \/ 4;
\n}<\/p>\n

But alas, on .NET 8, we still get:<\/p>\n

; Tests.DivideBy4_Int32(Int32)
\n cmp esi,4
\n jl short M00_L00
\n mov eax,esi
\n sar eax,1F
\n and eax,3
\n add eax,esi
\n sar eax,2
\n ret
\nM00_L00:
\n xor eax,eax
\n ret
\n; Total bytes of code 22<\/p>\n

On .NET 9, dotnet\/runtime#94347<\/a> updates the JIT for exactly that, replacing signed division with unsigned division if it can prove that both the numerator and denominator are non-negative.<\/p>\n

; Tests.DivideBy4_Int32(Int32)
\n cmp esi,4
\n jl short M00_L00
\n mov eax,esi
\n shr eax,2
\n ret
\nM00_L00:
\n xor eax,eax
\n ret
\n; Total bytes of code 14<\/p>\n

Nullable.<\/strong> Several optimizations went into making Nullable<T> cheaper, in particular when used with generics. Consider this benchmark:
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public void TestStruct() => Dispose<List.Enumerator>(default);<\/p>\n

[Benchmark]
\n public void TestNullableStruct() => Dispose<List<int>.Enumerator?>(default);<\/p>\n

[MethodImpl(MethodImplOptions.AggressiveInlining)]
\n private static void Dispose<T>(T t)
\n {
\n if (t is IDisposable)
\n {
\n ((IDisposable)t).Dispose();
\n }
\n }
\n}<\/p>\n

We have an unconstrained generic method Dispose whose job it is to cast the argument to IDisposable and invoke its Dispose. While such an operation would seemingly box if T were a value type, for a long time now the JIT has had optimizations that end up eliminating that boxing. In the case of the List<T>.Enumerator, its Dispose implementation is a nop, so with Dispose<T> getting inlined, no boxing, and the IDisposable.Dispose implementation nop\u2019ing, this whole method is a nop (on both .NET 8 and .NET 9):<\/p>\n

; Tests.TestStruct()
\n ret
\n; Total bytes of code 1<\/p>\n

That\u2019s unfortunately not the case for TestNullableStruct. The only<\/em> difference between TestStruct and TestNullableStruct is that pesky ? in the generic type argument, which means T will be a Nullable<List<int>.Enumerator> rather than List<int>.Enumerator. That complicates things. Nullable<T> is very special, with a boxed nullable implementing the same interfaces as does the underlying struct, but it ends up being very hard for the JIT to deal with. On .NET 8, we end up with this assembly:<\/p>\n

; Tests.TestNullableStruct()
\n push rbp
\n sub rsp,20
\n lea rbp,[rsp+20]
\n vxorps xmm8,xmm8,xmm8
\n vmovdqa xmmword ptr [rbp-20],xmm8
\n vmovdqa xmmword ptr [rbp-10],xmm8
\n cmp byte ptr [rbp-20],0
\n jne short M00_L01
\nM00_L00:
\n add rsp,20
\n pop rbp
\n ret
\nM00_L01:
\n lea rsi,[rbp-20]
\n mov rdi,offset MT_System.Nullable`1[[System.Collections.Generic.List`1+Enumerator[[System.Int32, System.Private.CoreLib]], System.Private.CoreLib]]
\n call CORINFO_HELP_BOX_NULLABLE
\n mov rsi,rax
\n mov rdi,offset MT_System.IDisposable
\n call qword ptr [7F178F2543C0]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
\n mov rdi,rax
\n mov r11,7F178E5904C0
\n call qword ptr [r11]
\n jmp short M00_L00
\n; Total bytes of code 93<\/p>\n

That\u2019s a whole lot more than a ret. Thankfully, for .NET 9, dotnet\/runtime#95764<\/a> makes this better by optimizing castclass for Nullable<T>:<\/p>\n

; Tests.TestNullableStruct()
\n sub rsp,28
\n xor eax,eax
\n mov [rsp+8],rax
\n vxorps xmm8,xmm8,xmm8
\n vmovdqa xmmword ptr [rsp+10],xmm8
\n mov [rsp+20],rax
\n cmp byte ptr [rsp+8],0
\n jne short M00_L01
\nM00_L00:
\n add rsp,28
\n ret
\nM00_L01:
\n lea rsi,[rsp+8]
\n mov rdi,offset MT_System.Nullable`1[[System.Collections.Generic.List`1+Enumerator[[System.Int32, System.Private.CoreLib]], System.Private.CoreLib]]
\n call CORINFO_HELP_BOX_NULLABLE
\n movsx rax,byte ptr [rax+8]
\n jmp short M00_L00
\n; Total bytes of code 66<\/p>\n

We still have the call to CORINFO_HELP_BOX_NULLABLE, but the relatively expensive call to ChkCastInterface is now gone. While this may seem a little corner case, it actually shows up in well-known places. For example:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int? _value = 42;<\/p>\n

[Benchmark]
\n public string Interpolate() => $”{_value}”;
\n}<\/p>\n

Here we\u2019re just doing string interpolation, using a nullable value type as one of the arguments. The DefaultInterpolatedStringHandler has a generic AppendFormatted method which this will end up using, passing the Nullable<int> as its argument, and that method employs similar patterns of type testing for an interface and using it if it\u2019s available. And as a result, this optimization can have a measurable impact on such interpolated string use:<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Interpolate
\n.NET 8.0
\n78.12 ns
\n1.00<\/p>\n

Interpolate
\n.NET 9.0
\n62.95 ns
\n0.81<\/p>\n

Another Nullable<T>-related optimization is dotnet\/runtime#95711<\/a>, which ends up avoiding boxing for some forms of type testing. Consider this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int? _value = 42;<\/p>\n

[Benchmark]
\n public bool Test() => IsInt(_value);<\/p>\n

private static bool IsInt<T>(T value) => value is int;
\n}<\/p>\n

This should be relatively straightforward: the JIT can see that T is a Nullable<int>, and then whether it satisfies the type test is a question of whether the value is null or not, since if it\u2019s null, it\u2019s not an int, and if it\u2019s not null, it is an int. Unfortunately, on .NET 8, not so much:<\/p>\n

; Tests.Test()
\n push rbp
\n sub rsp,10
\n lea rbp,[rsp+10]
\n mov rsi,[rdi+8]
\n mov [rbp-8],rsi
\n lea rsi,[rbp-8]
\n mov rdi,offset MT_System.Nullable`1[[System.Int32, System.Private.CoreLib]]
\n call CORINFO_HELP_BOX_NULLABLE
\n test rax,rax
\n je short M00_L00
\n mov rcx,offset MT_System.Int32
\n xor edx,edx
\n cmp [rax],rcx
\n cmovne rax,rdx
\nM00_L00:
\n test rax,rax
\n setne al
\n movzx eax,al
\n add rsp,10
\n pop rbp
\n ret
\n; Total bytes of code 76<\/p>\n

In fact, we can see it\u2019s using CORINFO_HELP_BOX_NULLABLE to box the Nullable<int>, which means we actually end up with an allocation as part of this type test. And that\u2019s visible in the benchmark results:<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size
\nAllocated
\nAlloc Ratio<\/p>\n

Test
\n.NET 8.0
\n39.1567 ns
\n1.000
\n76 B
\n24 B
\n1.00<\/p>\n

Test
\n.NET 9.0
\n0.0006 ns
\n0.000
\n5 B
\n\u2013
\n0.00<\/p>\n

On .NET 9, it ends up being what we thought it should be, a simple null check:<\/p>\n

; Tests.Test()
\n movzx eax,byte ptr [rdi+8]
\n ret
\n; Total bytes of code 5<\/p>\n

where the result of the method is simply Nullable<T>.HasValue.<\/p>\n

As a small tangent since we\u2019re talking about optimizing casting, dotnet\/runtime#98284<\/a> improves code generation for casts where the JIT can end up seeing that the object being cast is null (while you\u2019d probably never explicitly write if (null is SomeClass), you might very well write if (GetObject() is SomeClass) were GetObject() might get inlined and return null, especially if GetObject() is virtual and due to dynamic PGO a null-returning override gets inlined).<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public Tests? NullCast() => GetObj() as Tests;<\/p>\n

private object? GetObj() => null;
\n}<\/p>\n

On .NET 8, it doesn\u2019t pay attention to whether it knows that the source will be null, but now in .NET 9, it does:<\/p>\n

\/\/ .NET 8
\n; Tests.NullCast()
\n push rax
\n mov rdi,offset MT_Tests
\n xor esi,esi
\n call qword ptr [7F0457E24360]; System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
\n nop
\n add rsp,8
\n ret
\n; Total bytes of code 25<\/p>\n

\/\/ .NET 9
\n; Tests.NullCast()
\n push rax
\n xor eax,eax
\n add rsp,8
\n ret
\n; Total bytes of code 8<\/p>\n

Back to Nullable<T>, dotnet\/runtime#105073<\/a> enables the JIT to inline the fast path of the unboxing helper that\u2019s used when extracting a Nullable<T> from an object. There\u2019s an CORINFO_HELP_UNBOX_NULLABLE helper function that\u2019s called to perform the unboxing (e.g. (int?)o for some object o), but the success path (where the object is either null or the boxed target type) is small and it\u2019s worth inlining that.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private object _o = 42;<\/p>\n

[Benchmark]
\n public int? Unbox() => (int?)_o;
\n}<\/p>\n

On .NET 8, we get the following, effectively just a call to CORINFO_HELP_UNBOX_NULLABLE:<\/p>\n

; Tests.Unbox()
\n push rax
\n mov rdx,[rdi+8]
\n lea rdi,[rsp]
\n mov rsi,offset MT_System.Nullable`1[[System.Int32, System.Private.CoreLib]]
\n call CORINFO_HELP_UNBOX_NULLABLE
\n mov rax,[rsp]
\n add rsp,8
\n ret
\n; Total bytes of code 33<\/p>\n

whereas on .NET 9, we get the following, which is creating a default Nullable<int> if the object is null, or a Nullable<int> with the value from the object if it\u2019s a boxed int, or calling CORINFO_HELP_UNBOX_NULLABLE if it\u2019s something else (in which case we\u2019ll be throwing an exception shortly):<\/p>\n

; Tests.Unbox()
\n push rbp
\n sub rsp,10
\n lea rbp,[rsp+10]
\n mov rdx,[rdi+8]
\n test rdx,rdx
\n jne short M00_L00
\n xor edx,edx
\n mov [rbp-8],rdx
\n jmp short M00_L01
\nM00_L00:
\n mov rax,offset MT_System.Int32
\n cmp [rdx],rax
\n jne short M00_L02
\n mov byte ptr [rbp-8],1
\n mov eax,[rdx+8]
\n mov [rbp-4],eax
\nM00_L01:
\n mov rax,[rbp-8]
\n add rsp,10
\n pop rbp
\n ret
\nM00_L02:
\n lea rdi,[rbp-8]
\n mov rsi,offset MT_System.Nullable`1[[System.Int32, System.Private.CoreLib]]
\n call CORINFO_HELP_UNBOX_NULLABLE
\n jmp short M00_L01
\n; Total bytes of code 83<\/p>\n

This is one of those cases where you actually want the code to be larger, at least for the micro-benchmark, because the inlining is the purpose and is bringing in more code.<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nCode Size<\/p>\n

Unbox
\n.NET 8.0
\n6.014 ns
\n1.00
\n33 B<\/p>\n

Unbox
\n.NET 9.0
\n2.854 ns
\n0.47
\n83 B<\/p>\n

BigInteger<\/h3>\n

Not exactly a \u201cprimitive\u201d type, but in the same ballpark, is BigInteger. As with sbyte, short, int, and long, System.Numerics.BigInteger is an IBinaryInteger<> and ISignedNumber<>. Unlike those types, which are all of a fixed bit size (8, 16, 32, and 64 bits, respectively), BigInteger can represent signed integers with any number of bits (within reason\u2026 the current representation allows up to Array.MaxLength \/ 64 bits, which means representing numbers up to 2^33,554,432\u2026 that\u2019s\u2026 big). Such large sizes brings with it performance complexities, and historically BigInteger hasn\u2019t been a beacon of high throughput. While there\u2019s still more that can be done (and in fact there are several pending PRs even as I write this), a bunch of nice changes have landed for .NET 9.<\/p>\n

dotnet\/runtime#91176<\/a> from @Rob-Hague<\/a> improved BigInteger\u2018s byte-based constructors (e.g. public BigInteger(byte[] value)) by utilizing vectorized operations from MemoryMarshal and BinaryPrimitives. In particular, a lot of the time spent in these BigInteger constructors is in walking the list of bytes, building up integers out of each grouping of four, and storing those into a destination uint[]. With spans, however, that whole operation is unnecessary and can be achieved with an optimized CopyTo operation (effectively a memcpy) with the destination just being that uint[] reinterpreted as a span of bytes.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;
\nusing System.Security.Cryptography;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _bytes;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _bytes = new byte[10_000];
\n new Random(42).NextBytes(_bytes);
\n }<\/p>\n

[Benchmark]
\n public BigInteger NewBigInteger() => new BigInteger(_bytes);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

NewBigInteger
\n.NET 8.0
\n5.886 us
\n1.00<\/p>\n

NewBigInteger
\n.NET 9.0
\n1.434 us
\n0.24<\/p>\n

Parsing is another common way of creating BigIntegers. dotnet\/runtime#95543<\/a> improved the performance of parsing hex and binary-formatted values (this is on top of the .NET 9 addition in dotnet\/runtime#85392<\/a> from @lateapexearlyspeed<\/a> that added support for the “b” format specifier for formatting and parsing BigInteger as binary). Previously, parsing would go digit-by-digit, but with the new algorithm, it parses multiple chars at the same time, using a vectorized implementation for larger inputs.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Globalization;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _hex = string.Create(1024, 0, (dest, _) => new Random(42).GetItems<char>(“0123456789abcdef”, dest));<\/p>\n

[Benchmark]
\n public BigInteger ParseHex() => BigInteger.Parse(_hex, NumberStyles.HexNumber);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

ParseHex
\n.NET 8.0
\n5,155.5 ns
\n1.00
\n5208 B
\n1.00<\/p>\n

ParseHex
\n.NET 9.0
\n236.8 ns
\n0.05
\n536 B
\n0.10<\/p>\n

This isn\u2019t the first time efforts have been made to improve BigInteger parsing. .NET 7, for example, included a change that introduced a new parsing algorithm. The previous algorithm was O(N^2) in the number of digits, and the new algorithm had a lower algorithmic complexity, but due to the constants involved was only worthwhile with a larger number of digits. Both algorithms were included, switching between them based on a cut-off of 20,000 digits. As it turns out, with more analysis, that threshold was significantly higher than it needed to be, and dotnet\/runtime#97101<\/a> from @kzrnm<\/a> lowered the threshold to a much smaller value (1233). On top of this, dotnet\/runtime#97589<\/a> from @kzrnm<\/a> improves parsing further by a) recognizing that the multiplier being used during parsing (shifting down digits to make room for adding in the next set) includes many leading zeros that can be ignored during the operation, and b) trailing zeros when parsing powers of 10 could be calculated more efficiently.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _digits = string.Create(2000, 0, (dest, _) => new Random(42).GetItems<char>(“0123456789”, dest));<\/p>\n

[Benchmark]
\n public BigInteger ParseDecimal() => BigInteger.Parse(_digits);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

ParseDecimal
\n.NET 8.0
\n24.60 us
\n1.00
\n5528 B
\n1.00<\/p>\n

ParseDecimal
\n.NET 9.0
\n18.95 us
\n0.77
\n856 B
\n0.15<\/p>\n

Once you have a BigInteger, there are of course various operations you can do with it. BigInteger.Equals was improved by dotnet\/runtime#91416<\/a> from @Rob-Hague<\/a>, which changed the implementation to use the optimized MemoryExtensions.SequenceEqual rather than walking the arrays backing each BigInteger element-by-element. dotnet\/runtime#104513<\/a> from @Rob-Hague<\/a> improved BigInteger.IsPowerOfTwo by similarly replacing a manual walk of the elements with a call to ContainsAnyExcept, looking to see whether all elements after a certain point were 0.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private BigInteger _value1, _value2;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n var value1 = new byte[10_000];
\n new Random(42).NextBytes(value1);<\/p>\n

_value1 = new BigInteger(value1);
\n _value2 = new BigInteger(value1.AsSpan().ToArray());
\n }<\/p>\n

[Benchmark]
\n public bool Equals() => _value1 == _value2;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Equals
\n.NET 8.0
\n1,110.38 ns
\n1.00<\/p>\n

Equals
\n.NET 9.0
\n79.80 ns
\n0.07<\/p>\n

dotnet\/runtime#92208<\/a> from @kzrnm<\/a> also improved BigInteger.Multiply, and in particular when multiplying a first value that\u2019s much larger than a second value.<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private BigInteger _value1 = BigInteger.Parse(string.Concat(Enumerable.Repeat(“1234567890”, 1000)));
\n private BigInteger _value2 = BigInteger.Parse(string.Concat(Enumerable.Repeat(“1234567890”, 300)));<\/p>\n

[Benchmark]
\n public BigInteger MultiplyLargeSmall() => _value1 * _value2;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

MultiplyLargeSmall
\n.NET 8.0
\n231.0 us
\n1.00<\/p>\n

MultiplyLargeSmall
\n.NET 9.0
\n118.8 us
\n0.51<\/p>\n

Lastly, in addition to parsing, BigInteger formatting also saw some improvements. dotnet\/runtime#100181<\/a> removed various temporary buffer allocations that were occurring as part of formatting and optimized various calculations in order to reduce overheads while formatting these values.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private BigInteger _value = BigInteger.Parse(string.Concat(Enumerable.Repeat(“1234567890”, 300)));
\n private char[] _dest = new char[10_000];<\/p>\n

[Benchmark]
\n public bool TryFormat() => _value.TryFormat(_dest, out _);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

TryFormat
\n.NET 8.0
\n102.49 us
\n1.00
\n7456 B
\n1.00<\/p>\n

TryFormat
\n.NET 9.0
\n94.52 us
\n0.92
\n\u2013
\n0.00<\/p>\n

TensorPrimitives<\/h3>\n

Numerics has been a big focus for .NET over the last several releases. A large stable of numerical operations is now exposed on every numerical type as well as on a set of generic interfaces those types implement. But sometimes you want to perform the same operation on a set of values rather than on an individual value, and for that, we have TensorPrimitives. .NET 8 introduced the TensorPrimitive type, which provides a plethora of numerical APIs, but for spans of them rather than for individual values. For example, float has a Cosh method:<\/p>\n

public static float Cosh(float x);<\/p>\n

which provides the hyberbolic cosine<\/a> of one float, and a corresponding method shows up on the IHyperbolicFunctions<TSelf> interface:<\/p>\n

static abstract TSelf Cosh(TSelf x);<\/p>\n

TensorPrimitives then has a corresponding method, but rather than accepting one float, it accepts a span of them, and rather than returning the results, it writes the results into a provided destination span:<\/p>\n

public static void Cosh(ReadOnlySpan<float> x, Span<float> destination);<\/p>\n

In .NET 8, TensorPrimitives provided approximately 40 such methods, and only did so for float. Now in .NET 9, this has been significantly expanded. There are now over 200 overloads on TensorPrimitives, covering most of the numerical operations that are also exposed on the generic math interfaces (and some that aren\u2019t), and<\/em> they\u2019re exposed using generics, such that they can work with many more data types than just float. For example, while it maintains its float-specific overload of Cosh for backwards binary compatibility, TensorPrimitives now also sports this overload:<\/p>\n

public static void Cosh<T>(ReadOnlySpan<T> x, Span<T> destination)
\n where T : IHyperbolicFunctions<T><\/p>\n

such that it can be used with Half, float, double, NFloat, or any custom floating-point type you might have, as long as it implements the relevant interface. Most of these operations are also vectorized, such that it\u2019s more than just a simple loop around the corresponding scalar function.<\/p>\n

\/\/ Add a <PackageReference Include=”System.Numerics.Tensors” Version=”9.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics.Tensors;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private float[] _source, _destination;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n var r = new Random(42);
\n _source = Enumerable.Range(0, 1024).Select(_ => (float)r.NextSingle()).ToArray();
\n _destination = new float[1024];
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public void ManualLoop()
\n {
\n ReadOnlySpan<float> source = _source;
\n Span<float> destination = _destination;
\n for (int i = 0; i < source.Length; i++)
\n {
\n destination[i] = float.Cosh(source[i]);
\n }
\n }<\/p>\n

[Benchmark]
\n public void BuiltIn()
\n {
\n TensorPrimitives.Cosh<float>(_source, _destination);
\n }
\n}<\/p>\n

Method
\nMean
\nRatio<\/p>\n

ManualLoop
\n7,804.4 ns
\n1.00<\/p>\n

BuiltIn
\n621.6 ns
\n0.08<\/p>\n

A huge number of APIs is available, most of which see similar or better gains over the simple loop. Here\u2019s what\u2019s currently available in .NET 9, all as generic methods, and with multiple overloads available for most:<\/p>\n

Abs, Acosh, AcosPi, Acos, AddMultiply, Add, Asinh, AsinPi, Asin, Atan2Pi, Atan2, Atanh, AtanPi, Atan, BitwiseAnd, BitwiseOr, Cbrt, Ceiling, ConvertChecked, ConvertSaturating, ConvertTruncating, ConvertToHalf, ConvertToSingle, CopySign, CosPi, Cos, Cosh, CosineSimilarity, DegreesToRadians, Distance, Divide, Dot, Exp, Exp10M1, Exp10, Exp2M1, Exp2, ExpM1, Floor, FusedMultiplyAdd, HammingDistance, HammingBitDistance, Hypot, Ieee754Remainder, ILogB, IndexOfMaxMagnitude, IndexOfMax, IndexOfMinMagnitude, IndexOfMin, LeadingZeroCount, Lerp, Log2, Log2P1, LogP1, Log, Log10P1, Log10, MaxMagnitude, MaxMagnitudeNumber, Max, MaxNumber, MinMagnitude, MinMagnitudeNumber, Min, MinNumber, MultiplyAdd, MultiplyAddEstimate, Multiply, Negate, Norm, OnesComplement, PopCount, Pow, ProductOfDifferences, ProductOfSums, Product, RadiansToDegrees, ReciprocalEstimate, ReciprocalSqrtEstimate, ReciprocalSqrt, Reciprocal, RootN, RotateLeft, RotateRight, Round, ScaleB, ShiftLeft, ShiftRightArithmetic, ShiftRightLogical, Sigmoid, SinCosPi, SinCos, Sinh, SinPi, Sin, SoftMax, Sqrt, Subtract, SumOfMagnitudes, SumOfSquares, Sum, Tanh, TanPi, Tan, TrailingZeroCount, Truncate, Xor<\/p>\n

The possible speedups are even more pronounced on other operations and data types; for example, here is a manual implementation of hamming distance on two input byte arrays (hamming distance is the number of elements that differ between the two inputs), and an implementation using TensorPrimitives.HammingDistance<byte>:<\/p>\n

\/\/ Add a <PackageReference Include=”System.Numerics.Tensors” Version=”9.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics.Tensors;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _x, _y;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n var r = new Random(42);
\n _x = Enumerable.Range(0, 1024).Select(_ => (byte)r.Next(0, 256)).ToArray();
\n _y = Enumerable.Range(0, 1024).Select(_ => (byte)r.Next(0, 256)).ToArray();
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public int ManualLoop()
\n {
\n ReadOnlySpan<byte> source = _x;
\n Span<byte> destination = _y;
\n int count = 0;
\n for (int i = 0; i < source.Length; i++)
\n {
\n if (source[i] != destination[i])
\n {
\n count++;
\n }
\n }
\n return count;
\n }<\/p>\n

[Benchmark]
\n public int BuiltIn() => TensorPrimitives.HammingDistance<byte>(_x, _y);
\n}<\/p>\n

Method
\nMean
\nRatio<\/p>\n

ManualLoop
\n484.61 ns
\n1.00<\/p>\n

BuiltIn
\n15.76 ns
\n0.03<\/p>\n

A slew of PRs went into making this happen. The generic method surface area was added via dotnet\/runtime#94555<\/a>, dotnet\/runtime#97192<\/a>, dotnet\/runtime#97572<\/a>, dotnet\/runtime#101435<\/a>, dotnet\/runtime#103305<\/a>, and dotnet\/runtime#104651<\/a>. And then many more PRs added or improved vectorization, including dotnet\/runtime#97361<\/a>, dotnet\/runtime#97623<\/a>, dotnet\/runtime#97682<\/a>, dotnet\/runtime#98281<\/a>, dotnet\/runtime#97835<\/a>, dotnet\/runtime#97846<\/a>, dotnet\/runtime#97874<\/a>, dotnet\/runtime#97999<\/a>, dotnet\/runtime#98877<\/a>, dotnet\/runtime#103214<\/a> from @neon-sunset<\/a>, and dotnet\/runtime#103820<\/a>.<\/p>\n

As part of all of this work, there was also a recognition that we had the scalar operations and we had the operations on an unbounded number of elements as part of spans, but doing the latter efficiently required effectively having the same set of operations on the various Vector128<T>, Vector256<T>, and Vector512<T> types, since the typical structure of one of these operations will process vectors of elements at time. As such, progress has been made towards also exposing the same set of operations on these vector types. That\u2019s been done in dotnet\/runtime#104848<\/a>, dotnet\/runtime#102181<\/a>, dotnet\/runtime#103837<\/a>, dotnet\/runtime#97114<\/a>, and dotnet\/runtime#96455<\/a>. More to come.<\/p>\n

Other related numerical types have also seen improvements. Quaternion multiplication was vectorized in dotnet\/runtime#96624<\/a> by @TJHeuvel<\/a>, and dotnet\/runtime#103527<\/a> accelerated a variety of operations on Quaternion, Plane, Vector2, Vector3, Vector4, Matrix4x4, and Matrix3x2.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Quaternion _value1 = Quaternion.CreateFromYawPitchRoll(0.5f, 0.3f, 0.2f);
\n private Quaternion _value2 = Quaternion.CreateFromYawPitchRoll(0.1f, 0.2f, 0.3f);<\/p>\n

[Benchmark]
\n public Quaternion Multiply() => _value1 * _value2;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Multiply
\n.NET 8.0
\n3.064 ns
\n1.00<\/p>\n

Multiply
\n.NET 9.0
\n1.086 ns
\n0.35<\/p>\n

dotnet\/runtime#102301<\/a> also moves a lot of the implementation for types like Quaternion out of the JIT \/ native code into C#, something that\u2019s only possible now because of many of the other improvements discussed elsewhere in this post.<\/p>\n

Strings, Arrays, Spans<\/h2>\n

IndexOf<\/h3>\n

As previously noted in Performance Improvements in .NET 8<\/a> and earlier in this post, my single favorite performance improvement in .NET 8 came from enabling dynamic PGO. But my second favorite improvement came from the introduction of SearchValues<T>. SearchValues<T> enables optimizing searches, by pre-computing an algorithm to use when searching for a specific set of values (or for anything other than those specific values) and storing that information for later repeated use. Internally, .NET 8 included upwards of 15 different implementations that might be chosen based on the nature of the supplied data. The type was so good at what it did that it was used in over 60 places as part of the .NET 8 release. In .NET 9, it\u2019s used even more, and it gets even better, in a multitude of ways.<\/p>\n

The SearchValues<T> type is generic, so in theory it can be used for any T, but in practice, the algorithms involved need to special-case the nature of the data, and so the SearchValues.Create factory methods only enabled creating SearchValues<byte> and SearchValues<char> instances for which dedicated implementations were provided. For example, many of the previously noted uses of SearchValues<T> are searching for a subset of ASCII, such as this use from Regex.Escape, which enables quickly searching for all characters that require escaping:<\/p>\n

private static readonly SearchValues<char> s_metachars = SearchValues.Create(“tnfr #$()*+.?[\\^{|”);<\/p>\n

If you print out the name of the type of the instance returned by that Create call, as an implementation detail today you\u2019ll see something like this:<\/p>\n

System.Buffers.AsciiCharSearchValues`1[System.Buffers.IndexOfAnyAsciiSearcher+Default]<\/p>\n

That type provides a specialization of SearchValues<char> optimized for searching for any ASCII subset, doing so with an implementation based on the \u201cUniversal algorithm\u201d described at http:\/\/0x80.pl\/articles\/simd-byte-lookup.html<\/a>. Essentially, the algorithm maintains an 8 by 16 bitmap, which not coincidentally is the size of ASCII (0 through 127). Each of the 128 bits in the bitmap represents whether the corresponding ASCII value is in the set. The input chars are mapped down to bytes in a way where chars greater than 127 are mapped to a value meaning no match. The lower nibble (4 bits) of the ASCII value is used to select one of the 16 bitmap rows, and the upper nibble is used to select one of the 8 bitmap columns. And the beauty of this algorithm is, on most supported platforms, there exist SIMD instructions that enable the processing of many characters concurrently as part of just a few instructions.<\/p>\n

So, in .NET 8, SearchValues<T> was only for byte and char. But, now in .NET 9, thanks to dotnet\/runtime#88394<\/a>,
\ndotnet\/runtime#96429<\/a>, dotnet\/runtime#96928<\/a>, dotnet\/runtime#98901<\/a>, and dotnet\/runtime#98902<\/a>, you can also create SearchValues<string> instances. The string handling is different from byte and char, however. With byte, you\u2019re searching for one of a set of bytes within a span of bytes. With char, you\u2019re searching for one of a set of chars within a span of chars. But with string, SearchValues<string> doesn\u2019t search for one of a set of strings within a span of strings, but rather it enables searching for one of a set of strings within a span of chars. In other words, it\u2019s a multi-string search. For example, let\u2019s say you want to search some text for the ISO 8601 days of the week, and to do so in an ordinal case-insensitive manner (such that, for example, both \u201cMonday\u201d and \u201cMONDAY\u201d would match). That can now be expressed like this:<\/p>\n

private static readonly SearchValues<string> s_daysOfWeek = SearchValues.Create(
\n [“Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”, “Sunday”],
\n StringComparison.OrdinalIgnoreCase);
\n…
\nReadOnlySpan<char> textToSearch = …;
\nint i = textToSearch.IndexOfAny(s_daysOfWeek);<\/p>\n

This also highlights another interesting difference from the existing byte and char support. For those types, SearchValues is purely an optimization: IndexOfAny overloads have long existed for searching for sets of T values within a larger collection of Ts (e.g. string.IndexOfAny(char[] anyOf) was introduced over two decades ago), and the SearchValues support simply makes those use cases faster (often much<\/em> faster). In contrast, until .NET 9 there have not been any built-in methods for doing multi-string search, so this new support both adds such support and adds it in a way that is highly-efficient.<\/p>\n

But, let\u2019s say we did want to perform such a search, without that functionality existing in the core libraries. One approach is to simply walk through the input, position by position, comparing each of the target values at that location:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;
\n private static readonly string[] s_daysOfWeek = [“Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”, “Sunday”];<\/p>\n

[Benchmark(Baseline = true)]
\n public bool Contains_Iterate()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

for (int i = 0; i < input.Length; i++)
\n {
\n foreach (string dow in s_daysOfWeek)
\n {
\n if (input.Slice(i).StartsWith(dow, StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n }
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Contains_Iterate
\n227.526 us
\n1.000<\/p>\n

Classic. Functional. And slow. This is doing a fair amount of work for every single character in the input, for each looping over every day name and doing a comparison. How can we do better? First, we could try making the inner loop more efficient. Rather than iterating through the strings, we could hardcode our own switch:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;<\/p>\n

[Benchmark]
\n public bool Contains_Iterate_Switch()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

for (int i = 0; i < input.Length; i++)
\n {
\n ReadOnlySpan<char> slice = input.Slice(i);
\n switch ((char)(input[i] | 0x20))
\n {
\n case ‘s’ when slice.StartsWith(“Sunday”, StringComparison.OrdinalIgnoreCase) || slice.StartsWith(“Saturday”, StringComparison.OrdinalIgnoreCase):
\n case ‘m’ when slice.StartsWith(“Monday”, StringComparison.OrdinalIgnoreCase):
\n case ‘t’ when slice.StartsWith(“Tuesday”, StringComparison.OrdinalIgnoreCase) || slice.StartsWith(“Thursday”, StringComparison.OrdinalIgnoreCase):
\n case ‘w’ when slice.StartsWith(“Wednesday”, StringComparison.OrdinalIgnoreCase):
\n case ‘f’ when slice.StartsWith(“Friday”, StringComparison.OrdinalIgnoreCase):
\n return true;
\n }
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

The main benefit of this is it makes the StartsWith calls much more efficient. Because each call is dedicated to a specific needle that the JIT can see, it can emit customized code to optimize that comparison (for context on my choice of language, \u201cneedle\u201d is often used when describing a thing being searched for, a reference to the proverbial \u201cneedle in a haystack,\u201d and thus \u201chaystack\u201d is used to describe the thing being searched). We\u2019re also reducing the number of cases in the switch by employing an ASCII casing trick; the upper case ASCII letters differ in numerical value from the lower case ASCII letters by a single bit, so we simply ensure that bit is set and then compare against only the lower case letters.<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Contains_Iterate
\n227.526 us
\n1.000<\/p>\n

Contains_Iterate_Switch
\n13.885 us
\n0.061<\/p>\n

Much better, more than a 16x improvement. What if we instead just kept things simple and searched for each individual string using the already-optimized IndexOf?<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;
\n private static readonly string[] s_daysOfWeek = [“Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”, “Sunday”];<\/p>\n

[Benchmark]
\n public bool Contains_ContainsEachNeedle()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

foreach (string dow in s_daysOfWeek)
\n {
\n if (input.Contains(dow, StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

Nice and simple, but\u2026<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Contains_Iterate
\n227.526 us
\n1.000<\/p>\n

Contains_Iterate_Switch
\n13.885 us
\n0.061<\/p>\n

Contains_ContainsEachNeedle
\n302.330 us
\n1.329<\/p>\n

Ouch. On the positive side, this approach benefits from vectorization, as the Contains operation itself is vectorized to efficiently check multiple locations at once using SIMD. Unfortunately, this case is heavily impacted by the order in which we perform the search. As it turns out, most of the days of the week show up in the input text (in this case, \u201cWar and Peace\u201d), but at very different positions, and Monday doesn\u2019t show up at all. This:<\/p>\n

using var hc = new HttpClient();
\nvar s = await hc.GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”);
\nConsole.WriteLine($”Length: {s.Length}”);
\nConsole.WriteLine($”Monday: {s.IndexOf(“Monday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Tuesday: {s.IndexOf(“Tuesday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Wednesday: {s.IndexOf(“Wednesday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Thursday: {s.IndexOf(“Thursday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Friday: {s.IndexOf(“Friday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Saturday: {s.IndexOf(“Saturday”, StringComparison.OrdinalIgnoreCase)}”);
\nConsole.WriteLine($”Sunday: {s.IndexOf(“Sunday”, StringComparison.OrdinalIgnoreCase)}”);<\/p>\n

yields this:<\/p>\n

Length: 3293614
\nMonday: -1
\nTuesday: 971396
\nWednesday: 10652
\nThursday: 107470
\nFriday: 640801
\nSaturday: 1529549
\nSunday: 891753<\/p>\n

That means that whereas Contains_Iterate_Switch only needs to examine 10,652 positions (the position of the first \u201cWednesday\u201d) before it finds a match, Contains_ContainsEachNeedle needs to examine 3,293,614 (no match found for \u201cMonday\u201d so it\u2019ll look at everything) + 971,396 (the index of \u201cTuesday\u201d) == 4,265,010 positions before it finds a match. That\u2019s 400x as many positions to be examined as the iterative approach. Even the SIMD vectorization gains can\u2019t match that gap in amount of work to be performed.<\/p>\n

Ok, so what if we changed approach, and instead searched for the first letter in each word in order to quickly skip past the locations that couldn\u2019t possibly match. We could even use SearchValues<char> to perform that search.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;
\n private static readonly SearchValues<char> s_daysOfWeekFCSV = SearchValues.Create([‘S’, ‘s’, ‘M’, ‘m’, ‘T’, ‘t’, ‘W’, ‘w’, ‘F’, ‘f’]);<\/p>\n

[Benchmark]
\n public bool Contains_IndexOfAnyFirstChars_SearchValues()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

int i;
\n while ((i = input.IndexOfAny(s_daysOfWeekFCSV)) >= 0)
\n {
\n ReadOnlySpan<char> slice = input.Slice(i);
\n switch ((char)(input[i] | 0x20))
\n {
\n case ‘s’ when slice.StartsWith(“Sunday”, StringComparison.OrdinalIgnoreCase) || slice.StartsWith(“Saturday”, StringComparison.OrdinalIgnoreCase):
\n case ‘m’ when slice.StartsWith(“Monday”, StringComparison.OrdinalIgnoreCase):
\n case ‘t’ when slice.StartsWith(“Tuesday”, StringComparison.OrdinalIgnoreCase) || slice.StartsWith(“Thursday”, StringComparison.OrdinalIgnoreCase):
\n case ‘w’ when slice.StartsWith(“Wednesday”, StringComparison.OrdinalIgnoreCase):
\n case ‘f’ when slice.StartsWith(“Friday”, StringComparison.OrdinalIgnoreCase):
\n return true;
\n }<\/p>\n

input = input.Slice(i + 1);
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

In some situations, this is a very viable strategy; in fact, it\u2019s a technique often employed by Regex. In other situations, it\u2019s less appropriate. The potential problem is that letters like \u2018s\u2019 and \u2018t\u2019 are incredibly common. The characters here (\u2018s\u2019, \u2018m\u2019, \u2018t\u2019, \u2018w\u2019, and \u2018f\u2019), both upper- and lower-case variants, make up ~17% of the input text (in contrast to just the capital subset, which makes up only ~0.54%). That means that, on average, this IndexOfAny call needs to break out of its inner vectorized processing loop every six characters, which decreases the possible efficiency gains from said vectorization. Even so, this is still our best so far:<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Contains_Iterate
\n227.526 us
\n1.000<\/p>\n

Contains_Iterate_Switch
\n13.885 us
\n0.061<\/p>\n

Contains_ContainsEachNeedle
\n302.330 us
\n1.329<\/p>\n

Contains_IndexOfAnyFirstChars_SearchValues
\n7.151 us
\n0.031<\/p>\n

Now, let\u2019s try with SearchValues<string>:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;
\n private static readonly SearchValues<string> s_daysOfWeekSV = SearchValues.Create(
\n [“Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”, “Sunday”],
\n StringComparison.OrdinalIgnoreCase);<\/p>\n

[Benchmark]
\n public bool Contains_StringSearchValues() =>
\n s_input.AsSpan().ContainsAny(s_daysOfWeekSV);
\n}<\/p>\n

The functionality is built-in, so we haven\u2019t had to write any custom logic other than the call to ContainsAny. And the results:<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Contains_Iterate
\n227.526 us
\n1.000<\/p>\n

Contains_Iterate_Switch
\n13.885 us
\n0.061<\/p>\n

Contains_ContainsEachNeedle
\n302.330 us
\n1.329<\/p>\n

Contains_IndexOfAnyFirstChars_SearchValues
\n7.151 us
\n0.031<\/p>\n

Contains_StringSearchValues
\n2.153 us
\n0.009<\/p>\n

Not only simpler, then, but also several times faster than the fastest result we\u2019d previously managed, and ~105x faster than our original attempt. Sweet!<\/p>\n

How does this all work? The algorithms behind it are quite fascinating. As with byte and char, there are multiple concrete implementations that might be employed, selected based on the exact needle values passed to Create. The simplest implementations are those for handling degenerate cases, like zero inputs (in which case all of the methods can just return hard-coded \u201cnot found\u201d results). There\u2019s also a dedicated implementation for a single input, in which case it can perform the same search as IndexOf(needle) would have done, but lifting out the choice of characters within the needle for which to perform a vectorized search. IndexOf(string) chooses a couple of characters from the needle (typically just the first and last character in the needle), creates a vector for each of those, and then with appropriate offsets based on the distance between the chosen characters, iterates through the input, comparing against those vectors, and doing a full string comparison only if both vectors match at a particular location. SearchValues<string> does the same thing (in an internal implementation today called SingleStringSearchValuesThreeChars), except it uses three instead of two characters, and it employs frequency analysis to choose those characters rather than simply picking the first and last, trying to use characters that are less likely to appear in general (e.g. given the string \u201camazing\u201d, it\u2019d likely pick the \u2018m\u2019, \u2018z\u2019, and \u2018g\u2019, as it deems those statistically less likely in average inputs than \u2018a\u2019, \u2018i\u2019, or \u2018n\u2019). It can take more time to do this given it can perform the computation once and then cache it for all subsequent searches. We\u2019ll refer back to this in a bit.<\/p>\n

Beyond those special cases, it starts to get really interesting. There\u2019s been a lot of research done over the last 50 years for the most efficient ways to perform a multi-string search. One popular algorithm is Rabin-Karp, which was created by Richard Karp and Michael Rabin in the 1980s, and which works via a \u201crolling hash.\u201d Imagine creating a hash of the first N characters in the haystack (input) text, where N is the length of the needle (the substring) for which you\u2019re searching, and comparing the haystack hash against the needle hash; if they match, do the actual full comparison at that location, otherwise continue. Then update the hash by removing the first character and adding the next character, and repeat the check. And then repeat, and repeat, and so on. Each time you move forward, you\u2019re just updating the hash via a fixed number of operations, meaning that all of the updates to the hash function for the whole operation are only O(Haystack). Best case, you only find a single location that the substring could match, and you\u2019ve got O(Haystack + Needle) algorithmic complexity. Worst case (but generally unlikely), every location is a possible match, and you\u2019ve got O(Haystack * Needle) algorithmic complexity. A simple implementation might look like this (for pedagogical purposes, this uses a terrible hash function that just sums the character\u2019s numerical values; the real algorithm recommends a better one):<\/p>\n

private static bool RabinKarpContains(ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle)
\n{
\n if (haystack.Length >= needle.Length)
\n {
\n \/\/ Hash the needle and the first needle.Length chars of the haystack.
\n \/\/ Super simple hash for pedagogical purposes: just sum the chars.
\n int i, rollingHash = 0, needleHash = 0;
\n for (i = 0; i < needle.Length; i++)
\n {
\n rollingHash += haystack[i];
\n needleHash += needle[i];
\n }<\/p>\n

while (true)
\n {
\n \/\/ If the hashes match, compare the strings.
\n if (needleHash == rollingHash && haystack.Slice(i – needle.Length).StartsWith(needle))
\n {
\n return true;
\n }<\/p>\n

\/\/ If we’ve reached the end of the haystack, break.
\n if (i == haystack.Length)
\n {
\n break;
\n }<\/p>\n

\/\/ Update the rolling hash.
\n rollingHash += haystack[i] – haystack[i – needle.Length];
\n i++;
\n }
\n }<\/p>\n

return needle.IsEmpty;
\n}<\/p>\n

This supports one needle, but extending to support multiple needles can be accomplished in a variety of ways, such as by bucketing needles by their hash codes (ala what a hash map does), and then either checking all needles in the corresponding bucket when there\u2019s a hit, or further reduction in what needs to be checked based on using a Bloom filter or similar technique. SearchValues<string> will utilize Rabin-Karp, but only for very short inputs, as for longer inputs there are more efficient approaches.<\/p>\n

Another popular algorithm is Aho-Corasick, which was designed by Alfred Aho and Margaret Corasick even earlier, in the 1970s. Its primary purpose is multi-string search, enabling finding a match to be performed in linear time in the length of the input, assuming a fixed set of needles. It works by building up a form of a trie, a finite automaton where you start at the root of the graph and transition to children based on matching the character associated with the edge to that child. But, it extends a typical trie with additional edges between nodes that can be used as fallbacks. For example, here\u2019s the automaton for the days of the week discussed earlier:<\/p>\n

Given the input text \u201cwednesunday\u201d, this will start at the root, progress through the \u201cw\u201d, \u201cwe\u201d, \u201cwed\u201d, \u201cwedn\u201d, \u201cwedne\u201d, and \u201cwednes\u201d nodes, but then upon encountering the subsequent \u2018u\u2019 and not being able to progress down that path, it\u2019ll employ the fallback link over to the \u201cs\u201d node, at which point it\u2019ll be able to traverse down through \u201cs\u201d, \u201csu\u201d, and so on, until it hits the leaf \u201csunday\u201d node and can declare success. Aho-Corasick efficiently supports larger strings, and is the go-to implementation SearchValues<string> uses as a general fallback. However, in many situations, it can do even better\u2026<\/p>\n

The real workhorse of SearchValues<string> that\u2019s chosen whenever possible is a vectorized implementation of the \u201cTeddy\u201d algorithm. This algorithm originated in Intel\u2019s Hyperscan<\/a> library, was later adopted by the Rust aho_corasick crate, and is now employed as part of SearchValues<string> in .NET 9. It is super cool, and super efficient.<\/p>\n

Earlier, I gave a rough summary of how the SingleStringSearchValuesThreeChars and IndexOfAnyAsciiSearcher implementations work. SingleStringSearchValuesThreeChars optimizes finding likely positions where a substring might start, reducing the number of false positives by checking for multiple contained characters, and then likely positions are validated by doing the full string comparison at that location. And IndexOfAnyAsciiSearcher optimizes finding the next position of any character in a large-ish set. You can think of Teddy as a combination of those. There\u2019s a really nice description of the algorithm in the source<\/a>, so I won\u2019t go into much detail here. In summary, though, it maintains a similar bitmap as with IndexOfAnyAsciiSearcher, but instead of a single bit per ASCII character, it maintains an 8-bit bitmap for each nibble, and instead of just one bitmap, it maintains two or three, each of which corresponds to a location in the substrings (e.g. one bitmap for the 0th character and one bitmap for the 1st character). Those 8 bits in the bitmap are used to indicate which of up to 8 needles contain that nibble at that location. If there are fewer than 8 needles being searched for, then each of these bits individually identifies one of them, and if there are more than 8 needles, just as with Rabin-Karp, we can create buckets of the needle substrings, with a bit in the bitmap referring to one of the buckets. If the comparisons against the bitmaps indicates a likely match, the full match is performed against the relevant needle (or needles, in the case of matching a bucket). And as with IndexOfAnyAsciiSearcher, all of this support employs SIMD instructions to perform the lookups on chunks of input text from between 16 and 64 characters at a time, yielding significant speedups.<\/p>\n

SearchValues<string> is great for larger numbers of strings, but it\u2019s relevant even for just a few. Consider, for example, this code from MSBuild that\u2019s part of parsing build output looking for warnings and errors:<\/p>\n

if (message.IndexOf(“warning”, StringComparison.OrdinalIgnoreCase) == -1 &&
\n message.IndexOf(“error”, StringComparison.OrdinalIgnoreCase) == -1)
\n{
\n return null;
\n}<\/p>\n

Rather than doing two individual searches, we can perform a single search:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;
\n private static readonly SearchValues<string> s_warningError = SearchValues.Create([“warning”, “error”], StringComparison.OrdinalIgnoreCase);<\/p>\n

[Benchmark(Baseline = true)]
\n public bool TwoContains() =>
\n s_input.Contains(“warning”, StringComparison.OrdinalIgnoreCase) ||
\n s_input.Contains(“error”, StringComparison.OrdinalIgnoreCase);<\/p>\n

[Benchmark]
\n public bool ContainsAny() =>
\n s_input.AsSpan().ContainsAny(s_warningError);
\n}<\/p>\n

This is searching \u201cWar and Peace\u201d for both \u201cwarning\u201d or \u201cerror\u201d, but even though both appear in the text, such that the second search for \u201cerror\u201d in the original code will never happen, the SearchValues<string> search ends up being faster because \u201cerror\u201d appears much earlier in the text than does \u201cwarning\u201d.<\/p>\n

Method
\nMean
\nRatio<\/p>\n

TwoContains
\n70.03 us
\n1.00<\/p>\n

ContainsAny
\n14.05 us
\n0.20<\/p>\n

Beyond SearchValues<string>, the existing SearchValues<byte> and SearchValues<char> support also gets a variety of boosts in .NET 9. dotnet\/runtime#96588<\/a>, for example, makes some common SearchValues<char> searches faster, specifically when there are 2 or 4 characters being searched for that represent 1 or 2 ASCII case-insensitive characters, such as [‘A’, ‘a’] or [‘A’, ‘a’, ‘B’, ‘b’]. In .NET 8, for [‘A’, ‘a’], for example, SearchValues.Create will end up picking an implementation that will create a vector for each of ‘A’ and ‘a’, and then in the inner loop of the search, it\u2019ll compare each vector against the input haystack text. This PR teaches it to do a similar ASCII trick we discussed earlier: rather than having two separate vectors, it can have a single vector for ‘a’, and then do a single comparison against the input vector that\u2019s been OR\u2019d with 0x20, such that any ‘A’s become ‘a’s. The OR plus a single comparison is cheaper than the two comparisons plus the OR of the resulting comparisons. Funnily enough, this needn\u2019t even be about casing: since all we\u2019re doing is OR\u2019ing in 0x20, it applies to any two characters that differ by that same single bit.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/100\/pg100.txt”).Result;
\n private static readonly SearchValues<char> s_symbols = SearchValues.Create(“@`”);<\/p>\n

[Benchmark]
\n public bool ContainsAny() => s_input.AsSpan().ContainsAny(s_symbols);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ContainsAny
\n.NET 8.0
\n262.7 us
\n1.02<\/p>\n

ContainsAny
\n.NET 9.0
\n232.3 us
\n0.90<\/p>\n

The same thing applies with four characters: instead of doing four vector comparisons and three OR operations to combine them, we can do a single OR on the input to mix in 0x20, two vector comparisons, and a single OR to combine those results. In fact, the four-vector approach was already more expensive than the IndexOfAnyAsciiSearcher implementation previously described, and since that supports any number of ASCII characters, when applicable even for just four-character needles, SearchValues.Create would have preferred that. But now in .NET 9 with this optimization, SearchValues.Create will prefer to use this specialized two-comparison path.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/100\/pg100.txt”).Result;
\n private static readonly SearchValues<char> s_symbols = SearchValues.Create(“@`^~”);<\/p>\n

[Benchmark]
\n public bool ContainsAny() => s_input.AsSpan().ContainsAny(s_symbols);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ContainsAny
\n.NET 8.0
\n247.5 us
\n1.01<\/p>\n

ContainsAny
\n.NET 9.0
\n196.2 us
\n0.80<\/p>\n

Other SearchValues implementations also improve in .NET 9, notably the \u201cProbabilisticMap\u201d implementations. These implementations are preferred by SearchValues<char> as a fallback when the faster vectorized implementations aren\u2019t applicable but when the number of characters in the needle isn\u2019t exorbitant (the current limit is 256). It works via a form of Bloom filter. Effectively, it maintains a 256-bit bitmap, with needle characters mapping to one or two bits, depending on the char. If any of the bits for a given char isn\u2019t 1, then the char is definitively not in the set. If all of the bits for a given char are 1, then the char may<\/em> be in the set, and a more expensive check needs to be performed to determine inclusion. Whether those bits are set is a vectorizable operation, and so as long as false positives are relatively rare (which is why there\u2019s a limit on the number of characters; the more characters are represented, the more false positives there\u2019s likely to be), it\u2019s an efficient means for doing the search. However, this vectorization only applies to positive cases (e.g. IndexOfAny \/ ContainsAny) but not negative cases (e.g. IndexOfAnyExcept \/ ContainsAnyExcept); for those \u201cExcept\u201d methods, the implementation still walks character by character, and the check it employed per character was O(Needle). Thanks to dotnet\/runtime#101001<\/a>, which replaces a linear search with a \u201cperfect hash,\u201d that O(Needle) drops to O(1), making such \u201cExcept\u201d calls much more efficient.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_aristotle = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/39963\/pg39963.txt”).Result;
\n public static readonly SearchValues<char> s_greekOrAsciiDigits = SearchValues.Create(
\n Enumerable.Range(0, char.MaxValue + 1)
\n .Where(i => Regex.IsMatch(((char)i).ToString(), @”[p{IsGreek}0-9]”))
\n .Select(i => (char)i)
\n .ToArray());<\/p>\n

[Benchmark]
\n public int CountNonGreekOrAsciiDigitsChars()
\n {
\n int count = 0;<\/p>\n

ReadOnlySpan<char> text = s_aristotle;
\n int index;
\n while ((index = text.IndexOfAnyExcept(s_greekOrAsciiDigits)) >= 0)
\n {
\n count++;
\n text = text.Slice(index + 1);
\n }<\/p>\n

return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CountNonGreekOrAsciiDigitsChars
\n.NET 8.0
\n1,814.7 us
\n1.00<\/p>\n

CountNonGreekOrAsciiDigitsChars
\n.NET 9.0
\n881.7 us
\n0.49<\/p>\n

That same PR also made another significant improvement related to the probabilistic map: not using it as much. It\u2019s a terrific implementation for some sets of inputs, but for others it can end up performing poorly. .NET 8 included a Latin1CharSearchValues, which was used when some of the needle characters were non-ASCII but all were less than 256. In such cases, if the probabilistic map couldn\u2019t vectorize, SearchValues.Create would return a Latin1CharSearchValues instance, which maintained a simple 256-bit bitmap that detailed whether each character is in the needle. For .NET 9, that type has been replaced by a more general one that supports arbitrarily large bitmaps, used when there are simply too many for the probabilistic map implementation to handle well or when the values are sufficiently dense. Consider a case like this:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_markTwain = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;
\n public static readonly SearchValues<char> s_greekChars = SearchValues.Create(
\n Enumerable.Range(0, char.MaxValue + 1)
\n .Where(i => Regex.IsMatch(((char)i).ToString(), @”[p{IsGreek}p{IsGreekExtended}]”))
\n .Select(i => (char)i)
\n .ToArray());<\/p>\n

[Benchmark]
\n public int CountGreekChars()
\n {
\n int count = 0;<\/p>\n

ReadOnlySpan<char> text = s_markTwain;
\n int index;
\n while ((index = text.IndexOfAny(s_greekChars)) >= 0)
\n {
\n count++;
\n text = text.Slice(index + 1);
\n }<\/p>\n

return count;
\n }
\n}<\/p>\n

The needle here includes all of the characters in the Greek and Greek Extended Unicode blocks, approximately 400 characters. With the way the probabilistic map builds up its filter bitmap, every single bit in the bitmap ends up being set, which means every examined character will fall back to the expensive path. Now in .NET 9, it\u2019ll use a simpler, non-probabilistic bitmap, and even though it\u2019s not vectorized, it yields significantly faster throughput.<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CountGreekChars
\n.NET 8.0
\n126.454 ms
\n1.00<\/p>\n

CountGreekChars
\n.NET 9.0
\n8.956 ms
\n0.07<\/p>\n

dotnet\/runtime#96931<\/a> also extended this probabilistic map support to benefit from AVX512 such that when the probabilistic map implementation is used, it can be significantly faster. Previously, its implementation would utilize 128-bit or 256-bit vectors, depending on hardware support, but now in .NET 9, it can also use 512-bit vectors. This not only possibly doubles throughput due to vector width, AVX512 also includes some applicable instructions that the older instruction sets don\u2019t have (e.g. VPERMB, which is exposed as Avx512Vbmi.PermuteVar64x8), enabling even faster processing due to employing those more sophisticated instructions where relevant. This ends up being particularly impactful when searching for a reasonably small number of non-ASCII characters.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_aristotle = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/39963\/pg39963.txt”).Result;
\n public static readonly SearchValues<char> s_setSymbols = SearchValues.Create(“\u2282\u2283\u2286\u2287\u2284\u2229\u222a\u2208\u220a\u2209\u220b\u220d\u220c\u2205”);<\/p>\n

[Benchmark]
\n public int Count()
\n {
\n int count = 0;<\/p>\n

ReadOnlySpan<char> text = s_aristotle;
\n int index;
\n while ((index = text.IndexOfAny(s_setSymbols)) >= 0)
\n {
\n count++;
\n text = text.Slice(index + 1);
\n }<\/p>\n

return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n28.35 us
\n1.00<\/p>\n

Count
\n.NET 9.0
\n13.19 us
\n0.47<\/p>\n

Further, while the probabilistic map implementations were previously vectorized for IndexOfAny (and therefore implicitly for ContainsAny), they weren\u2019t for LastIndexOfAny, which means just changing whether you were searching from start to end or from end to start could have a significant impact on throughput. dotnet\/runtime#102331<\/a> improves that as well, enabling the LastIndexOfAny path to also take advantage of SIMD.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_markTwain = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;
\n public static readonly SearchValues<char> s_accentedChars = SearchValues.Create(
\n [‘\u00c0’, ‘\u00c8’, ‘\u00cc’, ‘\u00d2’, ‘\u00d9’, ‘\u00c1’, ‘\u00c9’, ‘\u00cd’, ‘\u00d3’, ‘\u00da’,
\n ‘\u00c2’, ‘\u00ca’, ‘\u00ce’, ‘\u00d4’, ‘\u00db’, ‘\u00c3’, ‘\u1ebc’, ‘\u0128’, ‘\u00d5’, ‘\u0168’,
\n ‘\u00c4’, ‘\u00cb’, ‘\u00cf’, ‘\u00d6’, ‘\u00dc’, ‘\u0178’]);<\/p>\n

[Benchmark]
\n public bool HasAnyAccented_IndexOfAny() => s_markTwain.AsSpan().IndexOfAny(s_accentedChars) >= 0;<\/p>\n

[Benchmark]
\n public bool HasAnyAccented_LastIndexOfAny() => s_markTwain.AsSpan().LastIndexOfAny(s_accentedChars) >= 0;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

HasAnyAccented_IndexOfAny
\n.NET 8.0
\n7.910 ms
\n1.00<\/p>\n

HasAnyAccented_IndexOfAny
\n.NET 9.0
\n4.476 ms
\n0.57<\/p>\n

HasAnyAccented_LastIndexOfAny
\n.NET 8.0
\n17.491 ms
\n1.00<\/p>\n

HasAnyAccented_LastIndexOfAny
\n.NET 9.0
\n5.253 ms
\n0.30<\/p>\n

In many of my examples, I\u2019ve used ContainsAny rather than IndexOfAny. The former is functionally equivalent to input.IndexOfAny(searchValues) >= 0, and in fact that was the entirety of the implementation in .NET 8. However, as IndexOfAny employs vectorization and is comparing multiple elements as part of the same instruction, when a match is found, there is a bit of overhead involved to then determine exactly which element matched (or if multiple matched, which match had the lowest index). ContainsAny doesn\u2019t actually need to care about the exact index: as exemplified by its implementation, it only cares about whether there was a match rather than where one was. As such, we can shave off some cycles by customizing the implementation for ContainsAny to avoid that unnecessary computation, and that\u2019s exactly what dotnet\/runtime#96924<\/a> does. The effects of this are most notable where that overhead would be measurable, which is when a match is found really early in the input.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _haystack = “Hello, world! How are you today?”;
\n private static readonly SearchValues<char> s_vowels = SearchValues.Create(“aeiou”);<\/p>\n

[Benchmark]
\n public bool ContainsAny() => _haystack.AsSpan().ContainsAny(s_vowels);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ContainsAny
\n.NET 8.0
\n3.640 ns
\n1.00<\/p>\n

ContainsAny
\n.NET 9.0
\n2.382 ns
\n0.65<\/p>\n

Improvements around SearchValues aren\u2019t limited just to new APIs or the implementation of the APIs; there\u2019s also been work to help developers better consume SearchValues. dotnet\/roslyn-analyzers#6898<\/a> and dotnet\/roslyn-analyzers#7252<\/a> added a new analyzer (CA1870<\/a>) that will find opportunities to use SearchValues and automatically fix the call sites to do so.<\/p>\n\n

It\u2019s also worth highlighting that there have been improvements around IndexOf \/ Contains in .NET 9 besides with SearchValues. One simple but interesting change is dotnet\/runtime#97632<\/a>. This simply added an if block to string.Contains(string):<\/p>\n

public bool Contains(string value)
\n{
\n if (value == null)
\n ThrowHelper.ThrowArgumentNullException(ExceptionArgument.value);<\/p>\n

\/\/ PR added this if block
\n if (RuntimeHelpers.IsKnownConstant(value) && value.Length == 1)
\n return Contains(value[0]);<\/p>\n

return SpanHelpers.IndexOf(
\n ref _firstChar,
\n Length,
\n ref value._firstChar,
\n value.Length) >= 0;
\n}<\/p>\n

What\u2019s interesting about this is the SpanHelpers.IndexOf it delegates to already contains a fast path that special-cases single-character strings:<\/p>\n

if (valueTailLength == 0)
\n{
\n \/\/ for single-char values use plain IndexOf
\n return IndexOfChar(ref searchSpace, value, searchSpaceLength);
\n}<\/p>\n

Why then is this extra if block helpful? It\u2019s taking advantage of that same internal IsKnownConstant intrinsic we saw earlier. The JIT will always compile this method down to a true\/false constant, so it ends up adding no runtime overhead. If the value is false, the whole if block evaporates. But if it\u2019s true, that necessarily means the argument passed to the method is recognized by the JIT as being a constant, e.g. a developer called someString.Contains(“-“) such that the JIT can see that value is “-“. In such a case, the JIT also knows value.Length, such that it can see at compile time whether it\u2019s 1 or not. And that in turn means this whole method becomes:<\/p>\n

public bool Contains(string value)
\n{
\n if (value == null)
\n ThrowHelper.ThrowArgumentNullException(ExceptionArgument.value);<\/p>\n

return SpanHelpers.IndexOf(
\n ref _firstChar,
\n Length,
\n ref value._firstChar,
\n value.Length) >= 0;
\n}<\/p>\n

if the JIT can\u2019t prove the argument is a constant or if it\u2019s not exactly one character in length, or:<\/p>\n

public bool Contains(string value)
\n{
\n return Contains(‘the constant char’);
\n}<\/p>\n

if it can. This eliminates a bit of overhead from the call.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _input = “!@#$%^&”;<\/p>\n

[Benchmark]
\n public bool Contains() => _input.Contains(“$”);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Contains
\n.NET 8.0
\n3.7649 ns
\n1.00<\/p>\n

Contains
\n.NET 9.0
\n0.9614 ns
\n0.26<\/p>\n

Regex<\/h3>\n

Regular expression support in .NET has received a lot of love over the past few years. The implementation was overhauled in .NET 5<\/a> to yield significant performance gains, and then in .NET 7<\/a> it not only saw another round of huge performance gains, it also gained a source generator, a new non-backtracking implementation, and more. In .NET 8, it saw additional performance improvements<\/a>, in part because of using SearchValues.<\/p>\n

Now in .NET 9, the trend continues. First and foremost, it\u2019s important to recognize that many of the changes discussed thus far implicitly accrue to Regex. Regex already uses SearchValues, and so improvements to SearchValues benefit Regex (it\u2019s one of my favorite things about working at the lowest levels of the stack: improvements there have a multiplicative effect, in that direct use of them improves, but so too does indirect use via intermediate components that instantly get better as the lower level does). Beyond that, though, Regex has increased its reliance on SearchValues.<\/p>\n

There are multiple engines backing Regex today:<\/p>\n

An interpreter, which is what you get when you don\u2019t explicitly ask for one of the other engines.
\nA reflection-emit-based compiler, which at run-time emits custom IL for the specific regular expression and options. This is what you get when you specify RegexOptions.Compiled.
\nA non-backtracking engine, which doesn\u2019t support all of Regex\u2018s features but which guarantees O(N) throughput in the length of the input. This is what you get when you specify RegexOptions.NonBacktracking.
\nAnd a source generator, which is very similar to the compiler, except it emits C# at build-time rather than emitting IL at run-time. This is what you get when you use [GeneratedRegex(…)].<\/p>\n

As of dotnet\/runtime#98791<\/a>, dotnet\/runtime#103496<\/a>, and dotnet\/runtime#98880<\/a>, all of the engines other than the interpreter avail themselves of the new SearchValues<string> support (the interpreter could as well, but we make an assumption that someone is using the interpreter in order to optimize for the speed of Regex construction, and the analyses involved in choosing to use SearchValues<string> can take measurable time). The best way to see what this looks like is via the source generator, as we can easily examine the code it outputs in both .NET 8 and .NET 9. Consider this code:<\/p>\n

using System.Text.RegularExpressions;<\/p>\n

internal partial class Example
\n{
\n [GeneratedRegex(“(Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday): (.*)”, RegexOptions.IgnoreCase)]
\n public static partial Regex ParseEntry();
\n}<\/p>\n

In Visual Studio, you can right-click on ParseEntry, select \u201cGo To Definition,\u201d and the tool will take you to the C# code for this pattern as generated by the regular expression source generator (the pattern is looking for a day of the week, followed by a colon, then followed by any text, and is capturing both the day and that subsequent text into capture groups for subsequent exploration). The generated code contains two relevant methods: a TryFindNextPossibleStartingPosition method, which is used to skip ahead as quickly as possible to the first location that might possibly match, and a TryMatchAtCurrentPosition method, which performs the full match attempt at that location. For our purposes here, we care about TryFindNextPossibleStartingPosition, as that\u2019s the most impactful place SearchValues shows up. On .NET 8, we see this code:<\/p>\n

private bool TryFindNextPossibleStartingPosition(ReadOnlySpan<char> inputSpan)
\n{
\n int pos = base.runtextpos;
\n ulong charMinusLowUInt64;<\/p>\n

\/\/ Any possible match is at least 8 characters.
\n if (pos <= inputSpan.Length – 8)
\n {
\n \/\/ The pattern matches a character in the set [ADSYadsy] at index 5.
\n \/\/ Find the next occurrence. If it can’t be found, there’s no match.
\n ReadOnlySpan<char> span = inputSpan.Slice(pos);
\n for (int i = 0; i < span.Length – 7; i++)
\n {
\n int indexOfPos = span.Slice(i + 5).IndexOfAny(Utilities.s_ascii_1200080212000802);
\n if (indexOfPos < 0)
\n {
\n goto NoMatchFound;
\n }
\n i += indexOfPos;<\/p>\n

if (((long)((0x8106400081064000UL << (int)(charMinusLowUInt64 = (uint)span[i] – ‘F’)) & (charMinusLowUInt64 – 64)) < 0) &&
\n ((long)((0x8023400080234000UL << (int)(charMinusLowUInt64 = (uint)span[i + 3] – ‘D’)) & (charMinusLowUInt64 – 64)) < 0))
\n {
\n base.runtextpos = pos + i;
\n return true;
\n }
\n }
\n }<\/p>\n

\/\/ No match found.
\n NoMatchFound:
\n base.runtextpos = inputSpan.Length;
\n return false;
\n}<\/p>\n

The code is using an IndexOfAny with Utilities.s_ascii_1200080212000802; what is that? It\u2019s a SearchValues<char>:<\/p>\n

\/\/\/ <summary>Supports searching for characters in or not in “ADSYadsy”.<\/summary>
\ninternal static readonly SearchValues<char> s_ascii_1200080212000802 = SearchValues.Create(“ADSYadsy”);<\/p>\n

The source generator is employing the approach we looked at earlier, searching for a single character from each string. Here it\u2019s decided that its best chance for an optimal search is to look for the character at offset 5 in each string, so \u2018y\u2019 for \u201cMonday\u201d, \u2018a\u2019 for \u201cTuesday\u201d, etc., plus looking for the upper-case variants since RegexOptions.IgnoreCase was specified. Then after the single-character search, it\u2019s doing a quick test for a couple of other positions in the string to try to weed out false positives, looking at the 0th offset to ensure the character is in the set [MTWFSmtwfs] and the 3rd offset to ensure the character is in the set [DSNRUdsnru]. (The check for those is obscured by it using a branchless technique to query a bitmap stored in a 64-bit ulong.)<\/p>\n

Now, here\u2019s what we get in .NET 9:<\/p>\n

private bool TryFindNextPossibleStartingPosition(ReadOnlySpan<char> inputSpan)
\n{
\n int pos = base.runtextpos;<\/p>\n

\/\/ Any possible match is at least 8 characters.
\n if (pos <= inputSpan.Length – 8)
\n {
\n \/\/ The pattern has multiple strings that could begin the match. Search for any of them.
\n \/\/ If none can be found, there’s no match.
\n int i = inputSpan.Slice(pos).IndexOfAny(Utilities.s_indexOfAnyStrings_OrdinalIgnoreCase_B7E3C0B8368AC400913BEA56D1872F43698FDA2C54D1AD4886F6734244613374);
\n if (i >= 0)
\n {
\n base.runtextpos = pos + i;
\n return true;
\n }
\n }<\/p>\n

\/\/ No match found.
\n base.runtextpos = inputSpan.Length;
\n return false;
\n}<\/p>\n

Again, we see an IndexOfAny, but notice that the subsequent checks for other positions are gone. Why? Because that SearchValues being passed to the IndexOfAny now is a SearchValues<string>, and thus already confirms that one of the provided strings matches:<\/p>\n

\/\/\/ <summary>Supports searching for the specified strings.<\/summary>
\ninternal static readonly SearchValues<string> s_indexOfAnyStrings_OrdinalIgnoreCase_B7E3C0B8368AC400913BEA56D1872F43698FDA2C54D1AD4886F6734244613374 =
\n SearchValues.Create(
\n [“monday”, “tuesday”, “wednesda”, “thursday”, “friday”, “saturday”, “sunday”],
\n StringComparison.OrdinalIgnoreCase);<\/p>\n

The sharp-eyed amongst you might notice that there\u2019s no \u2018y\u2019 at the end of \u201cWednesday\u201d; that\u2019s simply due to a heuristic in the Regex implementation. When it searches for strings to use as part of such a SearchValues<string>, it limits itself to using no more than length 8 strings. And \u201csearches\u201d is an appropriate word here, as the implementation isn\u2019t limited just to clean alternations as in the previous example. If I instead change the program to be:<\/p>\n

using System.Text.RegularExpressions;<\/p>\n

internal partial class Example
\n{
\n [GeneratedRegex(“[Aa]([Bb][Cc]|[Dd][Ee])”)]
\n public static partial Regex ParseEntry();
\n}<\/p>\n

we still end up with a SearchValues<string>, now for this:<\/p>\n

\/\/\/ <summary>Supports searching for the specified strings.<\/summary>
\ninternal static readonly SearchValues<string> s_indexOfAnyStrings_OrdinalIgnoreCase_33A76C255741CD9630059173F803FB92EBDDFBF62328261428CF8838D6379CE9 =
\n SearchValues.Create([“abc”, “ade”], StringComparison.OrdinalIgnoreCase);<\/p>\n

Interestingly, as of dotnet\/runtime#96402<\/a>, SearchValues<string> will also be used when doing a single string search. As previously noted, IndexOf(string) will try to pick two characters and do a vectorized search for both, whereas SearchValues<string> for that same input can spend a bit more time trying to pick more characters and characters that will be better for the search. As such, Regex now opts to use SearchValues<string> as part of TryFindNextPossibleStartingPosition. We can see this with the following benchmarks that count the number of occurrences of the words \u201cHello\u201d or \u201cearth\u201d:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic partial class Tests
\n{
\n public static readonly string s_markTwain = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;<\/p>\n

[GeneratedRegex(@”bHellob”)]
\n public static partial Regex FindHello();<\/p>\n

[GeneratedRegex(@”bearthb”)]
\n public static partial Regex FindEarth();<\/p>\n

[Benchmark]
\n public int CountHello() => FindHello().Count(s_markTwain);<\/p>\n

[Benchmark]
\n public int CountEarth() => FindEarth().Count(s_markTwain);
\n}<\/p>\n

On .NET 8, the code generated for TryFindNextPossibleStartingPosition for FindEarth includes:<\/p>\n

int i = inputSpan.Slice(pos).IndexOf(“earth”);<\/p>\n

whereas on .NET 9, the generated code is:<\/p>\n

int i = inputSpan.Slice(pos).IndexOfAny(Utilities.s_indexOfString_earth_Ordinal);
\n…
\ninternal static readonly SearchValues<string> s_indexOfString_earth_Ordinal =
\n SearchValues.Create([“earth”], StringComparison.Ordinal);<\/p>\n

And the results:<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CountHello
\n.NET 8.0
\n2.020 ms
\n1.00<\/p>\n

CountHello
\n.NET 9.0
\n2.042 ms
\n1.01<\/p>\n

CountEarth
\n.NET 8.0
\n2.738 ms
\n1.00<\/p>\n

CountEarth
\n.NET 9.0
\n2.339 ms
\n0.85<\/p>\n

This highlights that using SearchValues<string> in this one-string case doesn\u2019t always help, but it can improve things, in particular in situations where the extra one-time work done by the SearchValues.Create enables it to find meaningfully better characters for which to search.<\/p>\n

My seeming obsession with SearchValues<T> might lead one to believe that it\u2019s the only source of improvements in Regex, but that\u2019s far from the truth. There are many other PRs in .NET 9 focused on different aspects that improved the area.<\/p>\n

dotnet\/runtime#93190<\/a> is a nice addition. One of the optimizations introduced to Regex in .NET 7 was a \u201cliteral-after-loop\u201d search. A lot of effort goes into finding ways to help Regex\u2018s TryFindNextPossibleStartingPosition be as efficient as possible at skipping unnecessary locations, and this \u201cliteral-after-loop\u201d search is one such mechanism. It looks for a particular shape of pattern, where the pattern starts with a loop that\u2019s then followed by a literal. For example, the industry regex benchmarks in mariomka\/regex-benchmark<\/a> includes this pattern for finding URIs:<\/p>\n

@”[w]+:\/\/[^\/s?#]+[^s?#]+(?:?[^s#]*)?(?:#[^s]*)?”<\/p>\n

The pattern starts with a word character loop. We don\u2019t have a good way to vectorize a search for any word character, nor would we really want to; there are over 50,000 word characters that are part of the w set, and in most inputs we\u2019d typically find an occurrence so quickly that it wouldn\u2019t be worth the vectorization. However, the “:\/\/” that follows is easily searchable and much less likely to occur, making it a good candidate for TryFindNextPossibleStartingPosition. However, we can\u2019t just search for the “:\/\/” because it doesn\u2019t start the pattern, nor is it at a fixed-offset from the beginning of the pattern that would enable us to find the “:\/\/” and then jump backwards a known number of positions. Instead, with the \u201cliteral-after-loop\u201d optimization, we can find the “:\/\/” and then iterate backwards to the beginning of the loop in order to find the actual starting position for the match attempt (we can also keep track of where the loop ends so that we don\u2019t need to re-match it).<\/p>\n

There were, however, a number of gaps in this optimization. Most notably, the implementation needs to examine the pattern to determine whether it\u2019s applicable. If the starting loop was wrapped in a capture or an atomic group, it was unnecessarily giving up and would fail to discover the loop for the purposes of enabling the \u201cliteral-after-loop\u201d mechanism. The search would also give up if the literal after the loop was a set inside of various grouping constructs, like a concatenation.<\/p>\n

This PR fixed those gaps. The impact of this can be seen by looking at another industry benchmark, this time from the BurntSushi\/rebar<\/a> repo:<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic partial class Tests
\n{
\n private static string s_haystack = new HttpClient().GetStringAsync(“https:\/\/raw.githubusercontent.com\/BurntSushi\/rebar\/master\/benchmarks\/haystacks\/wild\/cpython-226484e4.py”).Result;<\/p>\n

[GeneratedRegex(@”(s*)((?:# [Nn][Oo][Qq][Aa])(?::s?(([A-Z]+[0-9]+(?:[,s]+)?)+))?)”)]
\n private static partial Regex RuffNoQA();<\/p>\n

[Benchmark]
\n public int Count() => RuffNoQA().Count(s_haystack);
\n}<\/p>\n

The impact of the literal-after-loop optimization ends up being obvious in the resulting numbers:<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n197.47 ms
\n1.00<\/p>\n

Count
\n.NET 9.0
\n18.67 ms
\n0.09<\/p>\n

Improvements in Regex go beyond just the initial searching, as well. An interesting changes comes in dotnet\/runtime#98723<\/a>, not because it results in massive performance improvements (though it does yield some nice benefits), but rather because it highlights how improvement possibilities can be found in all manner of places. One of the areas we (and pretty much everyone else in the world it seems) has been investing a lot of energy into lately is AI, including into tokenizers, which are components that take text and translate it into a series of numbers that are meaningful to the model into which they\u2019ll be fed. Each model is trained on a set of tokens from a specific tokenizer, and in the case of OpenAI\u2019s models, that tokenizer algorithm is \u201ctiktoken.\u201d The official .NET implementation of tiktoken lives in the Microsoft.ML.Tokenizers<\/a> library, and as part of implementing tiktoken, it follows the reference implementation provided by OpenAI, which uses a regular expression as part of parsing<\/a>; for consistency and to help ensure correctness, therefore, the .NET implementation does as well. This regex includes the following pattern:<\/p>\n

(?i:’s|’t|’re|’ve|’m|’ll|’d)<\/p>\n

What jumped out about this pattern is that it should trigger an optimization in the regex source generator that emits alternations like this as a C# switch statement, as the analyzer should be able to determine that all branches of the alternation are distinct, such that picking one branch because the first character matches necessarily means that no other branch could match. The benefit of a switch here is it allows the C# compiler to implement a jump table, which means we\u2019re not stuck exploring each branch when we could instead jump right to the correct one. But that optimization wasn\u2019t kicking in. Why? A series of unfortunate events. An earlier optimization was seeing the ll and rewriting that into a repeater (l{2}), which then defeated this alternation optimization because the implementation wasn\u2019t written to examine loops. Loops were explicitly being skipped because a loop could be empty, and if empty, it wouldn\u2019t have a first character required by the switch. However, we can see whether a loop has a non-zero minimum bound on number of iterations set, as it does in this case, and in such cases we can still factor in up to that minimum number, which are all guaranteed iterations. This PR improved the analysis to handle loops well, as evidenced by this micro-benchmark (which has been crafted to accentuate this aspect of the pattern):<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic partial class Tests
\n{
\n private static string s_haystack = new string(‘y’, 10_000);<\/p>\n

[GeneratedRegex(“(?i:a|bb|c|dd|e|ff|g|hh|i|jj|k|ll|m|nn|o|pp|q|rr|s|tt|u|vv|w|xx|y|zz)”)]
\n public static partial Regex Parse();<\/p>\n

[Benchmark]
\n public int Count() => Parse().Count(s_haystack);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n315.7 us
\n1.00<\/p>\n

Count
\n.NET 9.0
\n211.6 us
\n0.67<\/p>\n

The non-backtracking engine also got some nice attention in .NET 9. dotnet\/runtime#102655<\/a> from @ieviev<\/a> (who submitting a small subset of the changes they\u2019d made as part of some exciting regex research<\/a> being done in a fork of the library), followed by dotnet\/runtime#104766<\/a> and dotnet\/runtime#105668<\/a> made a variety of changes to the non-backtracking implementation, including:<\/p>\n

DFA limits.<\/strong> The non-backtracking implementation works by constructing a finite automata, which can be thought of as a graph, with the implementation walking around the graph as it consumes additional characters from the input and uses those to guide what node(s) it transitions to next. The graph is built out lazily, such that nodes are only added as those states are explored, and the nodes can be one of two kinds: DFA (deterministic) or NFA (non-deterministic). DFA nodes ensure that for any given character that comes next in the input, there\u2019s only ever one possible node to which to transition. Not so for NFA, where at any point in time there\u2019s a list of all the possible nodes the system could be in, and moving to the next state means examining each of the current states, finding all possible transitions out of each, and treating the union of all of those new positions as the next state. DFA is thus much<\/em> cheaper than NFA in terms of the overheads involved in walking around the graph, and we want to fall back to NFA only when we absolutely have to, which is when the DFA graph would be too large: some patterns have the potential to create massive numbers of DFA nodes. Thus, there\u2019s a threshold where once that number of constructed nodes in the graph is hit, new nodes are constructed as NFA rather than DFA. In .NET 8 and earlier, that limit was somewhat arbitrarily set at 10,000. For .NET 9 as part of this PR, analysis was done to show that a much higher limit was worth the memory trade-offs, and the limit was raised to 125,000, which means many more patterns can fully execute as DFA.
\nMinterm mappings.<\/strong> The implementation works in terms of \u201cminterms,\u201d which are equivalence classes for all characters that behave the same in the pattern. For example, with the pattern “[a-z]*”, the lowercase ASCII letters are all treated the same and are all treated differently from every other character, so there are two minterms here, one for the 26 lowercase ASCII letters, and the other for the remaining 65,510 characters. This is used as a compression mechanism, as rather than needing to describe the transitions between nodes for every character, the system can instead do so for every minterm. Of course, that means during matching there\u2019s a step where a character needs to be mapped to its minterm in order to know which edge to follow to the next state. Previously, that mapping was cached for all ASCII characters but recomputed each time for non-ASCII (recomputing it amounts to a binary search on a tree data structure). As you can imagine, this can lead to significant overhead when non-ASCII is encountered. Now in .NET 9, mappings for all characters represented in the pattern are stored. In degenerate cases, this can measurably increase memory consumption for the Regex instance, but on average it doesn\u2019t; in fact, for common cases the new scheme actually reduces memory consumption, as it takes into account the fact that all but the most niche patterns have fewer than 256 minterms, and the per-character mapping can thus be stored in a byte rather than a ushort or uint. Additionally, for cases where only a subset of ASCII is used in the pattern (which is common), the Regex instance needn\u2019t allocate an array to represent all 128 ASCII characters, but can instead be shrunk to only support those characters that need be represented.
\nTimeout checks.<\/strong> Regex has long supported a timeout mechanism, where if a match operation takes longer than a specified limit, an exception is thrown. This mechanism exists to help mitigate possible regex denial of service (ReDOS) attacks, where a maliciously-constructed pattern when fed to a backtracking engine could lead to \u201ccatastrophic backtracking\u201d (you can see an example of this in my Deep .NET discussion on Regex<\/a> with Scott Hanselman). These timeouts are thus enabled in the interpreter, the compiler, and the source generator. For the non-backtracking engine, timeouts aren\u2019t necessary to avoid catastrophic backtracking, as there is no backtracking. The engine still pays some attention to timeouts, though, purely for consistency with the other engines, yet the frequency of the checks was actually adding measurable overhead in some cases. The PR reduced the frequency of the checks to mitigate that overhead while not meaningfully affecting the effectiveness of the checks.
\nHot path inner loop.<\/strong> The inner matching loop is the hot path for a matching operation: read the next character, look up its minterm, follow the corresponding edge to the next node in the graph, rinse and repeat. Performance of the engine is tied to efficiency of this loop. These PRs recognized that there were some checks being performed in that inner loop which were only relevant to a minority of patterns. For the majority, the code could be specialized such that those checks wouldn\u2019t be needed in the hot path.
\nGeneral good hygiene.<\/strong> Care was taken to remove unnecessary overheads, such as duplicate array lookups that could be removed, bounds checks that could be avoided, indirect reads via refs that could instead be done against locals, and so on.<\/p>\n

The net result of these changes is most patterns get faster, some significantly, especially on non-ASCII inputs.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_aristotle = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/39963\/pg39963.txt”).Result;
\n private readonly Regex _word = new Regex(@”b[p{IsGreek}]+b”, RegexOptions.NonBacktracking);<\/p>\n

[Benchmark]
\n public int CountWords() => _word.Matches(s_aristotle).Count;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CountWords
\n.NET 8.0
\n14.808 ms
\n1.00<\/p>\n

CountWords
\n.NET 9.0
\n9.673 ms
\n0.65<\/p>\n

The dotnet\/runtime<\/a> employs an automated performance regression testing system, with tests in dotnet\/performance<\/a> constantly running on various operating systems and hardware, with the goal of detecting regressions. When a possible regression is noticed, an issue is opened containing the details. However, the system also notices statistically-significant improvements and also opens issues on those, just to ensure that we\u2019re all aware of when and how things change in a meaningful way. When possible, the issues reference the PR known to have caused the regression or improvement, so it\u2019s always a treat to see a list of references like this on a PR, as was the case with dotnet\/runtime#102655<\/a>:\n<\/p>\n

Both the non-backtracking engine and the interpreter now also gain additional optimized searching for certain classes of prefixes they didn\u2019t previously support. With dotnet\/runtime#100315<\/a>, patterns that begin with ranges can now be optimized with an IndexOfAny{Except}InRange call, whereas previously such patterns would likely result in walking character by character.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_markTwain = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;<\/p>\n

private readonly Regex _interpreter = new Regex(@”b[0-9]+b”);
\n private readonly Regex _nonBacktracking = new Regex(@”b[0-9]+b”, RegexOptions.NonBacktracking);<\/p>\n

[Benchmark]
\n public int Interpreter() => _interpreter.Count(s_markTwain);<\/p>\n

[Benchmark]
\n public int NonBacktracking() => _nonBacktracking.Count(s_markTwain);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Interpreter
\n.NET 8.0
\n21.223 ms
\n1.00<\/p>\n

Interpreter
\n.NET 9.0
\n1.726 ms
\n0.08<\/p>\n

NonBacktracking
\n.NET 8.0
\n21.945 ms
\n1.00<\/p>\n

NonBacktracking
\n.NET 9.0
\n1.749 ms
\n0.08<\/p>\n

Finally, Regex gains some new APIs in .NET 9, focused on performance. Regex currently has a set of Split overloads; these logically behave like Match, except instead of returning what matched, they effectively return what\u2019s between the matches, treating the match as a split separator. As with string.Split, these Regex.Split methods return a string[], which means allocating the array to store all the results and allocating each of the individual strings. There was also no overload for supporting span inputs, which meant if one had a span to search, that span would first need to be converted into a string, yet another allocation. .NET 7 saw a similar predicament fixed with the introduction of the EnumerateMatches method, which provided an allocation-free alternative to Match or Matches. Now in .NET 9, thanks to dotnet\/runtime#103307<\/a>, Regex gets new EnumerateSplits methods, that similarly provide an allocation-free way to access the same splits. The method accepts a ReadOnlySpan<char>, and then rather than returning an array of strings, it returns an enumerator of Ranges pointing into the original.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n public static readonly string s_markTwain = new HttpClient().GetStringAsync(“https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;
\n private readonly Regex _whitespace = new Regex(@”s+”, RegexOptions.Compiled);<\/p>\n

[Benchmark(Baseline = true)]
\n public int SplitOnWhitespace_Split()
\n {
\n int lengths = 0;
\n foreach (string split in _whitespace.Split(s_markTwain))
\n {
\n lengths += split.Length;
\n }
\n return lengths;
\n }<\/p>\n

[Benchmark]
\n public int SplitOnWhitespace_EnumerateSplits()
\n {
\n int lengths = 0;
\n foreach (Range range in _whitespace.EnumerateSplits(s_markTwain))
\n {
\n ReadOnlySpan<char> split = s_markTwain.AsSpan(range);
\n lengths += split.Length;
\n }
\n return lengths;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

SplitOnWhitespace_Split
\n189.1 ms
\n1.00
\n185305389 B
\n1.000<\/p>\n

SplitOnWhitespace_EnumerateSplits
\n116.6 ms
\n0.62
\n272 B
\n0.000<\/p>\n

Encoding<\/h3>\n

Base64 encoding has been supported in .NET since the beginning, with methods like Convert.ToBase64String and Convert.FromBase64CharArray. More recently, a plethora of Base64-related APIs have been added, including span-based APIs on Convert but also a dedicated System.Buffers.Text.Base64 with methods for encoding and decoding between arbitrary bytes and UTF8 text, and most recently for very efficiently checking whether UTF8 and UTF16 text represents a valid Base64 payload.<\/p>\n

Base64 is a fairly simple encoding scheme for taking arbitrary binary data and converting it to ASCII text, splitting the input up into groups of 6 bits (2^6 == 64 possible values) and mapping each of those values to a specific character in the Base64 alphabet: the 26 upper-case ASCII letters, the 26 lower-case ASCII letters, the 10 ASCII digits, ‘+’, and ‘\/’. While this is an incredibly popular encoding mechanism, it runs into problems for some use cases because of the exact choice of alphabet. Including Base64 data in a URI is possibly problematic, as ‘+’ and ‘\/’ both have special meaning in URIs, as does the special ‘=’ symbol used for padding Base64 data out to a specific length. That means that in addition to Base64-encoding data, the resulting data might also need to be URL-encoded for such a use, both taking additional time and further increasing the size of the payload. To address this, a variant was introduced, Base64Url, which does away with the need for padding, and which uses a slightly different alphabet, ‘-‘ instead of ‘+’ and ‘_’ instead of ‘\/’. Base64Url is used in a variety of domains, including as part of JSON Web Tokens (JWT)<\/a>, where it\u2019s used to encode each segment of the token.<\/p>\n

While .NET has had Base64 support for a long time, it hasn\u2019t had Base64Url support, and as such, developers have had to craft their own. Many have done so by layering on top of the Base64 implementations in Convert or Base64. For example, here\u2019s what the core part of ASP.NET\u2019s implementation for WebEncoders.Base64UrlEncode looked like in .NET 8:<\/p>\n

private static int Base64UrlEncode(ReadOnlySpan<byte> input, Span<char> output)
\n{
\n if (input.IsEmpty)
\n return 0;<\/p>\n

Convert.TryToBase64Chars(input, output, out int charsWritten);<\/p>\n

for (var i = 0; i < charsWritten; i++)
\n {
\n var ch = output[i];
\n if (ch == ‘+’) output[i] = ‘-‘;
\n else if (ch == ‘\/’) output[i] = ‘_’;
\n else if (ch == ‘=’) return i;
\n }<\/p>\n

return charsWritten;
\n}<\/p>\n

We can obviously write more code to do that more efficiently, but with .NET 9 we don\u2019t have to. With dotnet\/runtime#102364<\/a>, .NET now has a fully-featured Base64Url type that is also very efficient. It actually shares almost all of its implementation with the same functionality on Base64 and Convert, using generic tricks to substitute the different alphabets in an optimized manner. (The ASP.NET implementation has also been updated to use Base64Url with dotnet\/aspnetcore#56959<\/a> and dotnet\/aspnetcore#57050<\/a>.)<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers.Text;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _data;
\n private char[] _destination = new char[Base64.GetMaxEncodedToUtf8Length(1024 * 1024)];<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _data = new byte[1024 * 1024];
\n new Random(42).NextBytes(_data);
\n }<\/p>\n

[Benchmark(Baseline = true)]
\n public int Old() => Base64UrlOld(_data, _destination);<\/p>\n

[Benchmark]
\n public int New() => Base64Url.EncodeToChars(_data, _destination);<\/p>\n

static int Base64UrlOld(ReadOnlySpan<byte> input, Span<char> output)
\n {
\n if (input.IsEmpty)
\n return 0;<\/p>\n

Convert.TryToBase64Chars(input, output, out int charsWritten);<\/p>\n

for (var i = 0; i < charsWritten; i++)
\n {
\n var ch = output[i];
\n if (ch == ‘+’)
\n {
\n output[i] = ‘-‘;
\n }
\n else if (ch == ‘\/’)
\n {
\n output[i] = ‘_’;
\n }
\n else if (ch == ‘=’)
\n {
\n return i;
\n }
\n }<\/p>\n

return charsWritten;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio<\/p>\n

Old
\n1,314.20 us
\n1.00<\/p>\n

New
\n81.36 us
\n0.06<\/p>\n

This also benefits from a set of changes that improved the performance of Base64, and thus also Base64Url, since they now share the same code. dotnet\/runtime#92241<\/a> from @DeepakRajendrakumaran<\/a> added an AVX512-optimized Base64 encoding\/decoding implementation, and dotnet\/runtime#95513<\/a> from @SwapnilGaikwad<\/a> and dotnet\/runtime#100589<\/a> from @SwapnilGaikwad<\/a> optimized Base64 encoding and decoding for Arm64.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _toEncode;
\n private char[] _encoded;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _toEncode = new byte[1000];
\n new Random(42).NextBytes(_toEncode);
\n _encoded = new char[Convert.ToBase64String(_toEncode).Length];
\n }<\/p>\n

[Benchmark]
\n public void ConvertToBase64() => Convert.ToBase64CharArray(_toEncode, 0, _toEncode.Length, _encoded, 0);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ConvertToBase64
\n.NET 8.0
\n104.55 ns
\n1.00<\/p>\n

ConvertToBase64
\n.NET 9.0
\n60.19 ns
\n0.58<\/p>\n

Another simpler form of encoding is hex, effectively employing an alphabet of 16 characters (for each group of 4 bits) rather than 64 (for each group of 6 bits). .NET 5 introduced the Convert.ToHexString set of methods, which take an input ReadOnlySpan<byte> or byte[] and produce an output string with two hex chars per input byte. The alphabet selected for that encoding are the hexademical characters of \u20180\u2019 through \u20189\u2019 and then upper-case \u2018A\u2019 through \u2018F\u2019. That\u2019s great when you want upper-case, but sometimes you want the lower-case \u2018a\u2019 through \u2018f\u2019 instead. As a result, it\u2019s not uncommon now to see calls like this:<\/p>\n

string result = Convert.ToHexString(bytes).ToLowerInvariant();<\/p>\n

where ToHexString produces one string and then ToLowerInvariant possibly produces another (\u201cpossibly\u201d because it\u2019ll only need to create a new string if the data contained any letters). With .NET 9 and dotnet\/runtime#92483<\/a> from @determ1ne<\/a>, the new Convert.ToHexStringLower methods may be used to go directly to the lower-case version; that PR also introduced the TryToHexString and TryToHexStringLower methods, which format directly into a provided destination span rather than allocating anything.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers.Text;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _data = new byte[100];
\n private char[] _dest = new char[200];<\/p>\n

[GlobalSetup]
\n public void Setup() => new Random(42).NextBytes(_data);<\/p>\n

[Benchmark(Baseline = true)]
\n public string Old() => Convert.ToHexString(_data).ToLowerInvariant();<\/p>\n

[Benchmark]
\n public string New() => Convert.ToHexStringLower(_data).ToLowerInvariant();<\/p>\n

[Benchmark]
\n public bool NewTry() => Convert.TryToHexStringLower(_data, _dest, out int charsWritten);
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Old
\n136.69 ns
\n1.00
\n848 B
\n1.00<\/p>\n

New
\n119.09 ns
\n0.87
\n424 B
\n0.50<\/p>\n

NewTry
\n21.97 ns
\n0.16
\n\u2013
\n0.00<\/p>\n

Before .NET 5 introduced Convert.ToHexString, there actually already was some functionality in .NET for converting bytes to hex: BitConverter.ToString. BitConverter.ToString does the same thing Convert.ToHexString now does, except inserting dashes between every two hex characters (i.e. between every byte). As a result, it became fairly common for folks that wanted the equivalent of ToHexString to instead write BitConverter.ToString(bytes).Replace(“-“, “”). It\u2019s so common to want the dashes removed, in fact, that it\u2019s what GitHub copilot suggests for me just by typing BitConverter.ToString:<\/p>\n

Of course, that operation is much more expensive (and more complicated) than just using Convert.ToHexString, so it\u2019d be nice to help developers switch over to ToHexString{Lower}. That\u2019s exactly what dotnet\/roslyn-analyzers#6967<\/a> from @mpidash<\/a> does. CA1872<\/a> will now flag both cases that can be converted to Convert.ToHexString:<\/p>\n

and cases that can be converted to Convert.ToHexStringLower:<\/p>\n

And that\u2019s good for performance, as the difference is quite stark:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _bytes = Enumerable.Range(0, 100).Select(i => (byte) i).ToArray();<\/p>\n

[Benchmark(Baseline = true)]
\n public string WithBitConverter() => BitConverter.ToString(_bytes).Replace(“-“, “”).ToLowerInvariant();<\/p>\n

[Benchmark]
\n public string WithConvert() => Convert.ToHexStringLower(_bytes);
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

WithBitConverter
\n1,707.46 ns
\n1.00
\n1472 B
\n1.00<\/p>\n

WithConvert
\n61.66 ns
\n0.04
\n424 B
\n0.29<\/p>\n

There are a variety of reasons for that difference, including the obvious one that the Replace is having to search the input, find all the dashes, and allocate a brand new string without them. However, BitConverter.ToString is also slower in general as it\u2019s not as easily vectorized, due to needing to insert dashes between the resulting characters.<\/p>\n

In the other direction, Convert.FromHexString decodes a string of hex back into a new byte[]. dotnet\/runtime#86556<\/a> from @hrrrrustic<\/a> adds overloads of FromHexString that write into a destination span rather than allocating a new byte[] each time.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _hex = string.Concat(Enumerable.Repeat(“0123456789abcdef”, 10));
\n private byte[] _dest = new byte[100];<\/p>\n

[Benchmark(Baseline = true)]
\n public byte[] FromHexString() => Convert.FromHexString(_hex);<\/p>\n

[Benchmark]
\n public OperationStatus FromHexStringSpan() => Convert.FromHexString(_hex.AsSpan(), _dest, out int charsWritten, out int bytesWritten);
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

FromHexString
\n33.78 ns
\n1.00
\n104 B
\n1.00<\/p>\n

FromHexStringSpan
\n18.22 ns
\n0.54
\n\u2013
\n0.00<\/p>\n

Span, Span, and more Span<\/h3>\n

The introduction of Span<T> and ReadOnlySpan<T> back in .NET Core 2.1 have revolutionized how we write .NET code (especially in the core libraries) and what APIs we expose (see A Complete .NET Developer\u2019s Guide to Span<\/a> if you\u2019re interested in a deeper dive.) .NET 9 has continued the trend of doubling-down on spans as a great way to both implicitly provide performance boosts and also expose APIs that enables developers to do more for performance in their own code.<\/p>\n

One great example of this is the new C# 13 support for \u201cparams collections,\u201d which merged into the C# compiler\u2019s main branch in dotnet\/roslyn#72511<\/a>. This feature enables the C# params keyword to be used with more than just array parameters, but rather any collection type that\u2019s usable with collection expressions\u2026 that includes span. In fact, the feature makes it so that if there are two overloads, one taking a params T[] and one taking a params ReadOnlySpan<T>, the latter overload will win overload resolution. Moreover, the code generated for a call site for a params ReadOnlySpan<T> is the same non-allocating approach you get for collection expressions, e.g. given code like this:<\/p>\n

using System;<\/p>\n

public class C
\n{
\n public void M()
\n {
\n Helpers.DoAwesomeStuff(“Hello”, “World”);
\n }
\n}<\/p>\n

public static class Helpers
\n{
\n public static void DoAwesomeStuff<T>(params T[] values) { }
\n public static void DoAwesomeStuff<T>(params ReadOnlySpan<T> values) { }
\n}<\/p>\n

the IL the C# compiler generates for C.M will be equivalent to something like the following C#:<\/p>\n

<>y__InlineArray2<string> buffer = default;
\n<PrivateImplementationDetails>.InlineArrayElementRef<<>y__InlineArray2<string>, string>(ref buffer, 0) = “Hello”;
\n<PrivateImplementationDetails>.InlineArrayElementRef<<>y__InlineArray2<string>, string>(ref buffer, 1) = “World”;
\nHelpers.DoAwesomeStuff(<PrivateImplementationDetails>.InlineArrayAsReadOnlySpan<<>y__InlineArray2<string>, string>(ref buffer, 2));<\/p>\n

This is using the [InlineArray] feature introduced in .NET 8 to stack-allocate a span of strings, and then pass that span into the method. No heap allocation. This is awesome for library developers, because it means any place where you have a method taking a params T[], you can add a params ReadOnlySpan<T> overload, and when consuming code calling that method recompiles, it just gets better. dotnet\/runtime#101308<\/a> and dotnet\/runtime#101499<\/a> rely on that to add ~40 new overloads for methods that didn\u2019t previously accept spans and now do, and added params to over 20 existing overloads that were already taking spans. For example, if code had been using Path.Join to build up a path comprised of five or more segments, it previously would have been using the params string[] overload, but now upon recompilation it\u2019ll switch to using the params ReadOnlySpan<string> overload, and won\u2019t need to allocate a string[] for the inputs.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;
\nusing System.Numerics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public string Join() => Path.Join(“a”, “b”, “c”, “d”, “e”);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Join
\n.NET 8.0
\n30.83 ns
\n1.00
\n104 B
\n1.00<\/p>\n

Join
\n.NET 9.0
\n24.85 ns
\n0.81
\n40 B
\n0.38<\/p>\n

The C# compiler has also improved around spans in other ways. For example, dotnet\/roslyn#71261<\/a> extends the assembly data support for initializing arrays and ReadOnlySpan<T> to also apply to stackalloc. If you have code like this:<\/p>\n

var array = new char[] { ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’ };<\/p>\n

the compiler will generate code along the lines of the following:<\/p>\n

char[] array = new char[7];
\nRuntimeHelpers.InitializeArray(array, (RuntimeFieldHandle)&<PrivateImplementationDetails>.FD43C34A357FF620C00C04D0247059F8628CBB3DB349DF05DFA15EF6C7AC514C2);<\/p>\n

The compiler has taken that char data and blit it into the assembly; then when it creates the array, rather than setting each individual value into the array, it just copies that data directly from the assembly into the array. Similarly, if you have:<\/p>\n

ReadOnlySpan<char> span = new char[] { ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’ };<\/p>\n

the compiler recognizes that all of the data is constant and is being stored into a \u201cread-only\u201d location, so it doesn\u2019t actually need to allocate an array. Instead, it emits code like:<\/p>\n

ReadOnlySpan<char> span =
\nRuntimeHelpers.CreateSpan<char>((RuntimeFieldHandle)&<PrivateImplementationDetails>.FD43C34A357FF620C00C04D0247059F8628CBB3DB349DF05DFA15EF6C7AC514C2);<\/p>\n

which effectively creates a span that points directly into the assembly data; no allocation and<\/em> no copy needed. However, if you have this:<\/p>\n

ReadOnlySpan<char> span = stackalloc char[] { ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’ };<\/p>\n

or this:<\/p>\n

Span<char> span = stackalloc char[] { ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’ };<\/p>\n

you\u2019d get codegen more like this:<\/p>\n

char* ptr = stackalloc char[7];
\n*(char*)ptr = 97;
\n*(char*)(ptr + 1) = 98;
\n*(char*)(ptr + 2) = 99;
\n*(char*)(ptr + 3) = 100;
\n*(char*)(ptr + 4) = 101;
\n*(char*)(ptr + 5) = 102;
\n*(char*)(ptr + 6) = 103;
\nSpan<char> span = new Span<char>(ptr, 7);<\/p>\n

But now, thanks to dotnet\/roslyn#71261<\/a>, that last example will also be unified with the same approach for the other constructions, resulting in code more like this:<\/p>\n

char* ptr = stackalloc char[7];
\nUnsafe.CopyBlockUnaligned(ptr, &<PrivateImplementationDetails>.FD43C34A357FF620C00C04D0247059F8628CBB3DB349DF05DFA15EF6C7AC514C2, 14);
\nSpan<char> span = new Span<char>(ptr, 7);<\/p>\n

(the compiler will actually generate a cpblk IL instruction rather than a call to Unsafe.CopyBlockUnaligned).<\/p>\n

The C# compiler has also improved its ability to avoid allocations when creating ReadOnlySpan<T> from some expressed array constructions or collection expressions. One of the really nice optimizations the C# compiler added several years back was the ability to recognize when a new byte\/sbyte\/bool array was being constructed, filled with only constants, and directly assigned to a ReadOnlySpan<T>. In such a case, it would recognize that the data was all blittable and could never be modified, so rather than allocating an array and wrapping a span around it, it would blit the data into the assembly and then just construct a span around a pointer into the assembly data with the appropriate length. So this:<\/p>\n

ReadOnlySpan<byte> Values => new[] { (byte)0, (byte)1, (byte)2 };<\/p>\n

got lowered into something more like this:<\/p>\n

ReadOnlySpan<byte> Values => new ReadOnlySpan<byte>(
\n &<PrivateImplementationDetails>.AE4B3280E56E2FAF83F414A6E3DABE9D5FBE18976544C05FED121ACCB85B53FC),
\n 3);<\/p>\n

The optimization at the time was limited to only single-byte primitive types because of endianness concerns, but .NET 7 added a RuntimeHelpers.CreateSpan method which handled such endianness concerns, so then this was expanded to all such primitive types regardless of size. So this:<\/p>\n

ReadOnlySpan<char> Values1 => new[] { ‘a’, ‘b’, ‘c’ };
\nReadOnlySpan<int> Values2 => new[] { 1, 2, 3 };
\nReadOnlySpan<long> Values3 => new[] { 1L, 2, 3 };
\nReadOnlySpan<DayOfWeek> Values4 => new[] { DayOfWeek.Monday, DayOfWeek.Friday };<\/p>\n

gets lowered into something more like this:<\/p>\n

ReadOnlySpan<byte> Values1 => new ReadOnlySpan<byte>(
\n &<PrivateImplementationDetails>.13E228567E8249FCE53337F25D7970DE3BD68AB2653424C7B8F9FD05E33CAEDF2),
\n 3);<\/p>\n

ReadOnlySpan<byte> Values2 => new ReadOnlySpan<byte>(
\n &<PrivateImplementationDetails>.4636993D3E1DA4E9D6B8F87B79E8F7C6D018580D52661950EABC3845C5897A4D4),
\n 3);<\/p>\n

ReadOnlySpan<byte> Values3 => new ReadOnlySpan<byte>(
\n &<PrivateImplementationDetails>.E2E2033AE7E19D680599D4EB0A1359A2B48EC5BAAC75066C317FBF85159C54EF8),
\n 3);<\/p>\n

ReadOnlySpan<byte> Values3 => new ReadOnlySpan<byte>(
\n &<PrivateImplementationDetails>.ECA75F8497701D6223817CDE38BF42CDD1124E01EF6B705BCFE9A584F7B42F0F4),
\n 2);<\/p>\n

Lovely. But\u2026 what about types that are supported as constants at the C# level but that aren\u2019t blittable in this fashion? That includes nint and nuint (which vary in size based on the bitness of the process), decimal (for which a constant is actually represented in metadata via a [DecimalConstant(…)] attribute), and string (which is a reference type). In those cases, even though we\u2019re still targeting something that can be mutated and we\u2019re still using constants, we still get the array allocation:<\/p>\n

ReadOnlySpan<nint> Values1 => new nint[] { 1, 2, 3 };
\nReadOnlySpan<nuint> Values2 => new nuint[] { 1, 2, 3 };
\nReadOnlySpan<decimal> Values3 => new[] { 1m, 2m, 3m };
\nReadOnlySpan<string> Values4 => new[] { “a”, “b”, “c” };<\/p>\n

which are lowered to, well, themselves, such that there\u2019s still an allocation. Or, at least there was. Thanks to dotnet\/roslyn#69820<\/a>, these cases are now handled as well. They\u2019re addressed by lazily allocating an array that\u2019s then cached for all subsequent use. So now, that same example gets lowered into the equivalent of something more like this:<\/p>\n

ReadOnlySpan<nint> Values1 =>
\n <PrivateImplementationDetails>.4636993D3E1DA4E9D6B8F87B79E8F7C6D018580D52661950EABC3845C5897A4D_B8 ??=
\n new nint[] { 1, 2, 3 };<\/p>\n

ReadOnlySpan<nuint> Values2 =>
\n <PrivateImplementationDetails>.4636993D3E1DA4E9D6B8F87B79E8F7C6D018580D52661950EABC3845C5897A4D_B16 ??=
\n new nuint[] { 1, 2, 3 };<\/p>\n

ReadOnlySpan<decimal> Values3 =>
\n <PrivateImplementationDetails>.04B64E80BCEFE521678C4D6565B6EEBCE2791130A600CCB5D23E1B5538155110_B18 ??=
\n new[] { 1m, 2m, 3m };<\/p>\n

ReadOnlySpan<string> Values4 =>
\n <PrivateImplementationDetails>.13E228567E8249FCE53337F25D7970DE3BD68AB2653424C7B8F9FD05E33CAEDF_B11 ??=
\n new[] { “a”, “b”, “c” };<\/p>\n

There are, of course, many more span-related improvements in the libraries, too. One improvement for an existing span-related method is dotnet\/runtime#103728<\/a>, which further optimizes MemoryExtensions.Count used to count the number of occurrences of an element in a span. The implementation is vectorized, processing a vector\u2019s worth of data at a time, e.g. if 256-bit vectors are hardware accelerated, and it\u2019s searching chars, it\u2019ll process 16 chars at a time (16 chars 2 bytes per char <\/em> 8 bits per byte == 256). What happens if the number of elements isn\u2019t an even multiple of 16? Then we\u2019re left with some remaining elements after processing the last full vector. Previously the implementation would fall back to processing those remaining elements one at a time; now, it\u2019ll process one last vector at the end of the input. Doing so means we\u2019ll end up re-examining one ore more elements we already examined, but that doesn\u2019t really matter, as we can examine all of the elements in approximately the same number of instructions as processing just a single element.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Buffers;
\nusing System.Numerics;
\nusing System.Runtime.CompilerServices;
\nusing System.Runtime.InteropServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private char[][] _values = new char[10_000][];<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n var rng = new Random(42);
\n for (int i = 0; i < _values.Length; i++)
\n {
\n _values[i] = new char[rng.Next(0, 128)];
\n rng.NextBytes(MemoryMarshal.AsBytes(_values[i].AsSpan()));
\n }
\n }<\/p>\n

[Benchmark]
\n public int Count()
\n {
\n int count = 0;
\n foreach (char[] numbers in _values)
\n {
\n count += numbers.AsSpan().Count(‘a’);
\n }
\n return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n133.25 us
\n1.00<\/p>\n

Count
\n.NET 9.0
\n74.30 us
\n0.56<\/p>\n

New span-related functionality also shows up in .NET 9. String splitting is an operation that\u2019s used all over the place; a search for \u201c.Split(\u201d in C# code on GitHub yields millions of hits, and data from a variety of sources suggests that just the simplest overload Split(params char[]? separator) is used by upwards of 90% of applications and 20% of nuget packages. So it should come as no surprise that a request to have this functionality for spans is very popular.<\/p>\n\n

The devil is in the details, of course, and it\u2019s taken a long time to figure out exactly how it should be exposed. There are largely two different use cases for splitting we see in the wild. One case is where the content being split has an expected or max number of segments, and splitting is used to extract them. For example, FileVersionInfo needs to be able to take a version string and parse from it up to 4 components separated by periods. .NET 8 introduced new Split extension methods on MemoryExtensions to address this use case, by having Split take a destination Span<Range> to write the bounds of each segment into. That, however, still leaves the second major category of usage, which is for iterating through an unbounded number of segments. A representative example there is this snippet from HttpListener\u2018s web sockets implementation:<\/p>\n

string[] requestProtocols = clientSecWebSocketProtocol.Split(‘,’, StringSplitOptions.TrimEntries | StringSplitOptions.RemoveEmptyEntries);
\nfor (int i = 0; i < requestProtocols.Length; i++)
\n{
\n if (string.Equals(acceptProtocol, requestProtocols[i], StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n}<\/p>\n

The clientSecWebSocketProtocol string is composed of comma-separated values, and this is iterating through them to see if any is equal to the target acceptProtocol. It\u2019s doing that, though, with a relatively expensive operation. That Split call needs to allocate the string[] that\u2019s returned and that holds all the constituent strings, and then each segment results in a string being allocated. We can do better, and dotnet\/runtime#104534<\/a> from @bbartels<\/a> enables that. It adds four new overloads of MemoryExtensions.Split and MemoryExtensions.SplitAny:<\/p>\n

public static SpanSplitEnumerator<T> Split<T>(this ReadOnlySpan<T> source, T separator) where T : IEquatable<T>;
\npublic static SpanSplitEnumerator<T> Split<T>(this ReadOnlySpan<T> source, ReadOnlySpan<T> separator) where T : IEquatable<T>;
\npublic static SpanSplitEnumerator<T> SplitAny<T>(this ReadOnlySpan<T> source, params ReadOnlySpan<T> separators) where T : IEquatable<T>;
\npublic static SpanSplitEnumerator<T> SplitAny<T>(this ReadOnlySpan<T> source, SearchValues<T> separators) where T : IEquatable<T>;<\/p>\n

With that, this same operation can be written as:<\/p>\n

foreach (Range r in clientSecWebSocketProtocol.AsSpan().Split(‘,’))
\n{
\n if (clientSecWebSocketProtocol.AsSpan(r).Trim().Equals(acceptProtocol, StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n}<\/p>\n

In doing so, it becomes allocation-free, as this Split doesn\u2019t need to allocate a string[] to hold results and doesn\u2019t need to allocate a string for each segment: instead, it\u2019s returning a ref struct enumerator that yields a Range representing each segment. The caller can then use that Range to slice the input. It\u2019s yielding a Range rather than, say, a ReadOnlySpan<T>, to enable the splitting to be used with original sources other than spans and be able to get the segments in the original form. For example, if I had a ReadOnlyMemory<T> and wanted to add segments from it into a list, I could do:<\/p>\n

ReadOnlyMemory<T> source = …;
\nList<ReadOnlyMemory<T>> list = …;
\nforeach (Range r in source.Split(separator))
\n{
\n list.Add(source.Slice(r));
\n}<\/p>\n

whereas that wouldn\u2019t be possible if Split forced all yielded results to be spans.<\/p>\n

You might notice that there\u2019s no StringSplitOptions on these overloads. That\u2019s because it\u2019s both not applicable and not necessary. It\u2019s not applicable because we\u2019re working here with T, which might be something other than char, but an option like StringSplitOptions.TrimEntries implies a notion of whitespace, and that\u2019s only relevant for text. And it\u2019s not necessary, because the main benefit of StringSplitOptions, both TrimEntries and RemoveEmptyEntries, is reducing allocation overheads. If these options didn\u2019t exist with the string overloads, and you wanted to simulate them with our original example (and spans didn\u2019t exist), it would end up looking like this:<\/p>\n

string[] requestProtocols = clientSecWebSocketProtocol.Split(‘,’);
\nfor (int i = 0; i < requestProtocols.Length; i++)
\n{
\n if (string.Equals(acceptProtocol, requestProtocols[i].Trim(), StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n}<\/p>\n

There are several possible performance problems here. Imagine the clientSecWebSocketProtocol input was “a , b, , , , , , c”. There are only three entries we care about here (“a”, “b”, and “c”), but the returned array is going to be a string[8] instead of a string[3], because it\u2019s going to have entries for each of those whitespace-only segments. That\u2019s a larger allocation than is necessary. Then, we\u2019ll be producing strings for all eight of those segments, even though only three of the strings were necessary. And, all of “a “, ” b”, and ” c” have some extraneous whitespace that needs to be trimmed, such that the following Trim() call will allocate a new string for each. The StringSplitOptions enables the implementation of Split to avoid all of that overhead, by only allocating what\u2019s desired. But with the span version, none of that allocation exists anyway. The consuming loop can trim the spans itself without incurring more overhead than would the Split implementation, and the consuming loop can choose to ignore empty entries without increasing the size of a string[] allocation.<\/p>\n

The net result is such operations can be significantly more efficient while not sacrificing much if anything in the way of maintainability.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string _input = “a , b, , , , , , c”;
\n private string _target = “d”;<\/p>\n

[Benchmark(Baseline = true)]
\n public bool ContainsString()
\n {
\n foreach (string item in _input.Split(‘,’, StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
\n {
\n if (item.Equals(_target, StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n }<\/p>\n

return false;
\n }<\/p>\n

[Benchmark]
\n public bool ContainsSpan()
\n {
\n foreach (Range r in _input.AsSpan().Split(‘,’))
\n {
\n if (_input.AsSpan(r).Trim().Equals(_target, StringComparison.OrdinalIgnoreCase))
\n {
\n return true;
\n }
\n }<\/p>\n

return false;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

ContainsString
\n127.26 ns
\n1.00
\n208 B
\n1.00<\/p>\n

ContainsSpan
\n61.89 ns
\n0.49
\n\u2013
\n0.00<\/p>\n

The nature of this new set of splitting APIs is that they find just the next separator \/ segment; that\u2019s both practical and possibly a performance improvement by itself. It\u2019s practical because we\u2019re only yielding a single segment at a time, and we don\u2019t have anywhere to store all possible found separator positions (nor do we want to allocate space to do so). And it\u2019s desirable because the consumer may early-exit from the consuming loop, in which case we don\u2019t want to have spent time unnecessarily searching for additional segments that are going to be ignored. The existing set of splitting APIs, however, hand back all found segments in one go, either via a returned string[] or via ranges being written to a destination span. And as such, it makes more sense for those overloads to find all separators at once because that operation can be vectorized. In fact, previous versions have done so. But, that vectorization has only benefited from 128-bit vectors. With dotnet\/runtime#93043<\/a> from @khushal1996<\/a> in .NET 9, that vectorization will now light-up with 512-bit or 256-bit vectors if they\u2019re available, enabling that separator searching that happens as part of splitting to run up to four times faster.<\/p>\n

Spans show up in other new methods as well. dotnet\/runtime#93938<\/a> from @TheMaximum<\/a> added new overloads of StringBuilder.Replace that accept ReadOnlySpan<char> instead of string. As is the case with most such overloads, they share the same implementation, with the string-based overloads just creating a span from the string and using a span-based implementation. In practice, the majority of use of StringBuilder.Replace uses constant strings as arguments, for example to escape some known delimiter (Replace(“$”, “\\$”)), or use previously-created string instances, such as to remove some substring from text (Replace(substring, “”)). But, there are a minority of cases where Replace is used with something that\u2019s created on the spot, and for that, these new overloads can help to avoid allocation for creating the arguments. For example, here\u2019s some escaping code used today by MSBuild:<\/p>\n

char[] charsToEscape = …;
\nStringBuilder escapedString = …;
\nforeach (char unescapedChar in charsToEscape)
\n{
\n string escapedCharacterCode = string.Format(CultureInfo.InvariantCulture, “%{0:x00}”, (int)unescapedChar);
\n escapedString.Replace(unescapedChar.ToString(CultureInfo.InvariantCulture), escapedCharacterCode);
\n}<\/p>\n

This is having to perform two string allocations to create the input to this Replace, which is going to be invoked for each char in charsToEscape. If charsToEscape is something fixed, it could be better to avoid these formatting operations per iteration, and instead just cache the necessary strings for all uses, e.g.<\/p>\n

private static readonly char[] charsToEscape = …;
\nprivate static readonly string[] escapedCharsToEscape = charsToEscape.Select(c => $”%{(uint)unescapedChar:x00}”).ToArray();
\nprivate static readonly string[] stringsToEscape = charsToEscape.Select(c => c.ToString()).ToArray();
\n…
\nfor (int i = 0; i < charsToEscape.Length; i++)
\n{
\n escapedString.Replace(stringsToEscape[i], escapedCharsToEscape[i]);
\n}<\/p>\n

but if charsToEscape isn\u2019t predictable, then we can at least avoid the allocation by employing the new overloads, e.g.<\/p>\n

char[] charsToEscape = …;
\nStringBuilder escapedString = …;
\nSpan<char> escapedSpan = stackalloc char[5];
\nforeach (char unescapedChar in charsToEscape)
\n{
\n escapedSpan.TryWrite($”%{(uint)unescapedChar:x00}”, out int charsWritten);
\n escapedString.Replace(new ReadOnlySpan<char>(in unescapedChar), escapedSpan.Slice(0, charsWritten));
\n}<\/p>\n

and, boom, no more allocation for the arguments.<\/p>\n

A variety of other improvements were made to string manipulation, mainly around better employing vectorization. StringComparison.OrdinalIgnoreCase operations were previously vectorized, but only with 128-bit vectors, which means handling up to 8 chars at a time. Thanks to dotnet\/runtime#93116<\/a>, those code paths have been updated to support 256-bit and 512-bit vectors, which means handling up to 16 or 32 chars at a time on hardware accelerated to support it.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static string s_s1 = “””
\n Let me not to the marriage of true minds
\n Admit impediments; love is not love
\n Which alters when it alteration finds,
\n Or bends with the remover to remove.
\n O no, it is an ever-fixed mark
\n That looks on tempests and is never shaken;
\n It is the star to every wand’ring bark
\n Whose worth’s unknown, although his height be taken.
\n Love’s not time’s fool, though rosy lips and cheeks
\n Within his bending sickle’s compass come.
\n Love alters not with his brief hours and weeks,
\n But bears it out even to the edge of doom:
\n If this be error and upon me proved,
\n I never writ, nor no man ever loved.
\n “””;
\n private static string s_s2 = s_s1[0..^1] + “!”;<\/p>\n

[Benchmark]
\n public bool EqualsIgnoreCase() => s_s1.Equals(s_s2, StringComparison.OrdinalIgnoreCase);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

EqualsIgnoreCase
\n.NET 8.0
\n86.79 ns
\n1.00<\/p>\n

EqualsIgnoreCase
\n.NET 9.0
\n20.97 ns
\n0.24<\/p>\n

EndsWith also gets better, for both strings and spans. Previous releases saw StartsWith become a JIT intrinsic, enabling the JIT to generate dedicated SIMD code for StartsWith in the case where it\u2019s passed a constant. Now with dotnet\/runtime#98593<\/a>, the same thing is done for EndsWith.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser(maxDepth: 0)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(“helloworld.txt”)]
\n public bool EndsWith(string path) => path.EndsWith(“.txt”, StringComparison.OrdinalIgnoreCase);
\n}<\/p>\n

Method
\nRuntime
\npath
\nMean
\nRatio
\nCode Size<\/p>\n

EndsWith
\n.NET 8.0
\nhelloworld.txt
\n3.5006 ns
\n1.00
\n26 B<\/p>\n

EndsWith
\n.NET 9.0
\nhelloworld.txt
\n0.6653 ns
\n0.19
\n61 B<\/p>\n

More interesting to me than these nice gains is the code that was generated to achieve them. This is what the assembly for this benchmark looked like with .NET 8:<\/p>\n

; Tests.EndsWith(System.String)
\n mov rdi,rsi
\n mov rsi,7EE3C2D25E38
\n mov edx,5
\n cmp [rdi],edi
\n jmp qword ptr [7F24678663A0]; System.String.EndsWith(System.String, System.StringComparison)
\n; Total bytes of code 26<\/p>\n

Pretty straightforward, a bit of argument manipulation and then jumping to the actual string.EndsWith implementation. Now here\u2019s .NET 9:<\/p>\n

; Tests.EndsWith(System.String)
\n push rbp
\n mov rbp,rsp
\n mov eax,[rsi+8]
\n cmp eax,4
\n jge short M00_L00
\n xor ecx,ecx
\n jmp short M00_L01
\nM00_L00:
\n mov ecx,eax
\n lea rax,[rsi+rcx*2-8]
\n mov rcx,20002000200000
\n or rcx,[rax+0C]
\n mov rax,7400780074002E
\n cmp rcx,rax
\n sete cl
\n movzx ecx,cl
\nM00_L01:
\n movzx eax,cl
\n pop rbp
\n ret
\n; Total bytes of code 61<\/p>\n

Notice there\u2019s no call to string.EndsWith in sight. That\u2019s because the JIT has implemented the EndsWith functionality here, specific to “.txt” and OrdinalIgnoreCase, in just a few instructions. The address of the string is being passed into this method in the rsi register, and the second mov instruction is grabbing its Length (which is stored 8 bytes from the start of the string object) and storing that into the eax register. It\u2019s then checking whether the string is at least 4 characters long; if it\u2019s not, it can\u2019t possibly end with “.txt” and thus it jumps to the end to return false. If it was at least 4 characters long, it then proceeds to load the last four characters of the string as a 64-bit value into rcx and OR it with the value 20002000200000. Why? It\u2019s playing the same ASCII trick we\u2019ve seen before. The ‘.’ is not subject to casing, so we don\u2019t need to manipulate its value, and hence the 16-bits that aligns with the ‘.’ are 0. But the other three characters all need to be comparable with both their lower-case and upper-case forms, so this is OR\u2019ing each of the three 16-bit characters with 0x2000 to produce the lower-case form. At that point, the 64-bit value can be compared against the 64-bit representation of “.txt”, which is 7400780074002E (the ASCII value for ‘.’ is 0x2E, for ‘t’ is 0x74, and for ‘x’ is 0x78). Then it\u2019s just a simple matter of whether that compared equally or not.<\/p>\n

Finally, we\u2019ve not talked much about arrays separate from spans, but there have been improvements there as well. dotnet\/runtime#102739<\/a> and dotnet\/runtime#104103<\/a> move more logic for array handling from native code in the runtime up into C# code in CoreLib. For example, Array.Copy has to handle a wide array of cases (pun intended), some of which can be implemented very efficiently and some of which are more laborious, and it tries to optimize the \u201csimple\u201d cases, such as whether the bits from one array can simply be memcpy\u2019d over to the other, with as little overhead as possible. Some of those cases are easy to determine, such as single-dimensional arrays having the exact same type, but other cases require more introspection, such as if one array is enums and the other array is of the underlying type of that enum. The checks to make those determinations previously lived in native code in the runtime, but as of this PR they\u2019re now implemented in C#, and in doing so, some of the overhead associated with the checks has been removed.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private DayOfWeek[] _enums = Enum.GetValues<DayOfWeek>();
\n private int[] _ints = new int[7];<\/p>\n

[Benchmark]
\n public void Copy() => Array.Copy(_enums, _ints, _enums.Length);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Copy
\n.NET 8.0
\n16.25 ns
\n1.00<\/p>\n

Copy
\n.NET 9.0
\n11.05 ns
\n0.68<\/p>\n

In addition to other benefits that come from moving such logic into managed code (better maintainability, more implicitly safe code, reduced overhead from transitioning between managed and native, etc.), there\u2019s another less obvious benefit: impact on GC pause times. And that\u2019s nowhere more obvious than with dotnet\/runtime#98623<\/a>, which moved the implementations of memset\/memcpy helpers used for core operations like Span<T>.Fill and Array.Copy from native to managed. Consider this C# console app:<\/p>\n

using System.Diagnostics;<\/p>\n

new Thread(() =>
\n{
\n var a = new object[1000];
\n while (true) a.AsSpan().Fill(a);
\n})
\n{ IsBackground = true }.Start();<\/p>\n

var sw = new Stopwatch();
\nwhile (true)
\n{
\n sw.Restart();
\n for (int i = 0; i < 10; i++)
\n {
\n GC.Collect();
\n Thread.Sleep(15);
\n }
\n Console.WriteLine(sw.Elapsed.TotalSeconds);
\n}<\/p>\n

This is sitting in a loop that simply times how long it takes to perform 10 gen2 collections, each spaced out by ~15 ms. If each collection were free then, this loop should take ~150 ms. Since it\u2019s not free, let\u2019s round up and estimate that the loop should be around ~200 ms. Before we run the loop, though, we launch a thread that just sits in an infinite loop filling a span. That shouldn\u2019t mess with our timing loop\u2026 or should it? When I run this on .NET 8, I get values like this:<\/p>\n

1.0683524
\n0.8884759
\n0.8420748
\n1.1101804
\n1.2730635<\/p>\n

Those values are in seconds, and that\u2019s approximately 5x larger than we\u2019d predicated. Now I try on .NET 9, and I get results like this:<\/p>\n

0.1638237
\n0.2129748
\n0.2859566
\n0.3020449
\n0.2871952<\/p>\n

What happened? In order to do some of its work, the GC needs to be able to get a consistent view of the world, which is violated if things are concurrently changing out from under it. As such, it may need to temporarily suspend all threads in the process, but to do that, it needs to wait for each thread to get to a safe point, and if a thread is executing code in the runtime, that can be hard to do. In this particular case, there\u2019s a thread spending almost all of its time sitting in a call to Span<T>.Fill, aka memset, which was implemented in .NET 8 as a native function in the runtime; this couldn\u2019t be interrupted, and the GC would need to wait until the call returned and it could catch it before it could interrupt that thread. In .NET 9, these implementations are all in managed code, and the GC can trivially get the threads to a safe point.<\/p>\n

Collections<\/h2>\n

LINQ<\/h3>\n

Language Integrated Query, or LINQ, is a mainstay of .NET. At its heart, LINQ is a specification for hundreds of overloads of methods that manipulate data, and then implementations of that specification for different types. One of the most prominent implementations comes from System.Linq.Enumerable, sometimes referred to as \u201cLINQ to Objects,\u201d which provides an implementation of these operations as methods for working with IEnumerable<T>. It\u2019s an incredibly useful set of operations, used ubiquitously, and thus it\u2019s a common target for performance optimization. In many .NET releases, it\u2019ll get a new additional method here or an optimized method there, a trickle of focused improvements. But in .NET 9, it\u2019s received a huge amount of attention, with some improvements localized to particular methods and others broadly applicable across much of the surface area.<\/p>\n

One of the more sweeping LINQ changes in .NET 9 has to do with how various optimizations are implemented. In the original implementation of LINQ circa 2007, almost every method was logically independent from every other. A method like SelectMany took in an IEnumerable<TSource> and didn\u2019t know anything about where that input came from; every enumerable was processed the same as every other. Some methods would special-case more optimizable data types, though, for example ToArray would check whether the incoming IEnumerable<TSource> implemented ICollection<TSource>, preferring if it did to use the collection\u2019s Count and CopyTo in order to avoid having to MoveNext\/Current through the whole input. But a couple of methods, in particular some overloads of Select and Where, did something more interesting. Much of LINQ was implemented using the C# compiler\u2019s support for iterators, where a method that returns IEnumerable<T> can use yield return t; to produce instances of T, and the compiler handles rewriting that method into a class that implements IEnumerable<T> and handles all the gnarly state-machine details for you. These few Select and Where overloads, however, didn\u2019t use iterators, with the developer that authored them instead preferring to write a custom enumerable by hand. Why? It\u2019s possible to hand-author an ever-so-slightly more efficient implementation in some cases, but the compiler is actually really good at doing it well, so that\u2019s not the reason. The reason is because it a) gives the type a name that can be referred to elsewhere in the code, and b) it allows that type to expose state that other code can interrogate. This enables information to flow from one query operator to the next. So, for example, Where could return a WhereEnumerableIterator instance:<\/p>\n

class WhereEnumerableIterator<TSource> : Iterator<TSource>
\n{
\n IEnumerable<TSource> source;
\n Func<TSource, bool> predicate;
\n …
\n}<\/p>\n

And then Select can look for that type, or, rather, its base type, Iterator<TSource>:<\/p>\n

public static IEnumerable<TResult> Select<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TResult> selector) {
\n if (source == null) throw Error.ArgumentNull(“source”);
\n if (selector == null) throw Error.ArgumentNull(“selector”);
\n if (source is Iterator<TSource>) return ((Iterator<TSource>)source).Select(selector);
\n …
\n}<\/p>\n

and that WhereEnumerableIterator<TSource> can override that virtual Select method on Iterator<TSource> to specialize what happens when a Where is followed by Select:<\/p>\n

public override IEnumerable<TResult> Select<TResult>(Func<TSource, TResult> selector) {
\n return new WhereSelectEnumerableIterator<TSource, TResult>(source, predicate, selector);
\n}<\/p>\n

This is useful because it allows for avoiding one of the major sources of overhead with enumerables. Without this optimization, if I had source.Where(x => true).Select(x => x), the resulting enumerable would be for the Select, which would in turn wrap the enumerable for the Where, which would in turn wrap the source enumerable. That means that when I call MoveNext on the select iterator, it in turn needs to call MoveNext on the Where, which will in turn call MoveNext on the source, and then the same for Current. That means for each element in the source, we end up making 6 interface calls. With the cited optimization, we no longer have separate iterators for the Select and Where. Those end up being combined into a single iterator that does the work of both, eliminating one level from the call chain, so instead of 6 interface calls per element, there\u2019s only 4. (See Deep Dive on LINQ<\/a> and An event DEEPER Dive into LINQ<\/a> for a more in-depth exploration of how exactly this works.)<\/p>\n

Over the last decade with .NET, those optimizations have been significantly extended, and in some cases to much greater benefit than just saving a few interface calls. For example, in a previous .NET release, a similar mechanism was used to special-case OrderBy followed by First. Without special-casing, the OrderBy would need to do both a full copy of the input source and an O(N log N) sort of the data, all as part of the first call to MoveNext from First. But with the optimization, First is able to see that its source is that OrderBy, in which case it doesn\u2019t need a copy or sort at all, and can instead simply do an O(N) search of OrderBy\u2018s source for its minimum value. That difference can yield monstrous wins.<\/p>\n

This additional special-casing was achieved with internal interfaces in the library. An IIListProvider<TElement> provided ToArray, ToList, and GetCount methods, and an IPartition<TElement> interface (which inherited IIListProvider<TElement>) provided additional methods like Skip, Take, and TryGetFirst. Custom iterators used to back various LINQ methods could then also implement one or more of these interfaces to specialize being followed by a call like ToArray or Count(). For example, it\u2019s very common (e.g. as part of \u201cpaging\u201d) to see call sequences like .Skip(…).Take(…); with these optimizations, those two operations can be consolidated down into a single iterator, and if it were followed by an operation like Last() or ToList(), those could see through both operators to the source in order to possibly optimize based on it (e.g. if the source were an array, Last() could calculate the exact element to return without needing to do any iteration at all).<\/p>\n

dotnet\/runtime#98969<\/a> and dotnet\/runtime#99344<\/a> remove those internal interfaces and consolidate all of their members down to the base Iterator<TSource> type. This has a variety of benefits. Not directly related to performance, it simplifies the code base, making it easier to maintain (and easier to maintain code is also generally easier to optimize); the interface members of IPartition<TElement> became virtual methods on the base class, which also resulted in some code reduction due to being able to share the same default implementation (though with the introduction of default interface methods a few releases ago, this could<\/em> have been done separately without this consolidation). On the performance front, though, there are three main benefits of this PR:<\/p>\n

Virtual dispatch is generally a bit cheaper than interface dispatch. All of those interface methods became virtual methods, enabling all call sites to them to be a bit cheaper.
\nIn various places, type tests were being done for multiple targets, and those could now be consolidated to reduce type checks. For example, Select looked something like this:
\nif (source is Iterator<TSource> iterator)
\n{
\n …
\n}<\/p>\n

if (source is IPartition partition)
\n{
\n …
\n}<\/p>\n

That means for non-specialized iterators, Select was incurring a type check for Iterator<TSource> and an interface check for IPartition<TSource>. With this change, the latter check is now removed.<\/p>\n

Some types were only inheriting from the base class but not implementing any of the interfaces, some were implementing an interface but not the other, some were even implementing one of the interfaces but not deriving from the base class. The new approach makes it such that all of the provided virtual methods are implemented by any iterator deriving from the base class.<\/p>\n

dotnet\/runtime#97905<\/a>, dotnet\/runtime#97956<\/a>, dotnet\/runtime#98874<\/a>, and dotnet\/runtime#99216<\/a> also added more implementations.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<int> _arrayDistinct = Enumerable.Range(0, 1000).ToArray().Distinct();
\n private IEnumerable<int> _appendSelect = Enumerable.Range(0, 1000).ToArray().Append(42).Select(i => i * 2);
\n private IEnumerable<int> _rangeReverse = Enumerable.Range(0, 1000).Reverse();
\n private IEnumerable<int> _listDefaultIfEmptySelect = Enumerable.Range(0, 1000).ToList().DefaultIfEmpty().Select(i => i * 2);
\n private IEnumerable<int> _listSkipTake = Enumerable.Range(0, 1000).ToList().Skip(500).Take(100);
\n private IEnumerable<int> _rangeUnion = Enumerable.Range(0, 1000).Union(Enumerable.Range(500, 1000));<\/p>\n

[Benchmark] public int DistinctFirst() => _arrayDistinct.First();
\n [Benchmark] public int AppendSelectLast() => _appendSelect.Last();
\n [Benchmark] public int RangeReverseCount() => _rangeReverse.Count();
\n [Benchmark] public int DefaultIfEmptySelectElementAt() => _listDefaultIfEmptySelect.ElementAt(999);
\n [Benchmark] public int ListSkipTakeElementAt() => _listSkipTake.ElementAt(99);
\n [Benchmark] public int RangeUnionFirst() => _rangeUnion.First();
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

DistinctFirst
\n.NET 8.0
\n49.844 ns
\n1.00
\n328 B
\n1.00<\/p>\n

DistinctFirst
\n.NET 9.0
\n7.928 ns
\n0.16
\n\u2013
\n0.00<\/p>\n

AppendSelectLast
\n.NET 8.0
\n3,668.347 ns
\n1.000
\n144 B
\n1.00<\/p>\n

AppendSelectLast
\n.NET 9.0
\n2.222 ns
\n0.001
\n\u2013
\n0.00<\/p>\n

RangeReverseCount
\n.NET 8.0
\n8.703 ns
\n1.00
\n\u2013
\nNA<\/p>\n

RangeReverseCount
\n.NET 9.0
\n3.465 ns
\n0.40
\n\u2013
\nNA<\/p>\n

DefaultIfEmptySelectElementAt
\n.NET 8.0
\n2,772.283 ns
\n1.000
\n144 B
\n1.00<\/p>\n

DefaultIfEmptySelectElementAt
\n.NET 9.0
\n4.399 ns
\n0.002
\n\u2013
\n0.00<\/p>\n

ListSkipTakeElementAt
\n.NET 8.0
\n3.699 ns
\n1.00
\n\u2013
\nNA<\/p>\n

ListSkipTakeElementAt
\n.NET 9.0
\n2.103 ns
\n0.57
\n\u2013
\nNA<\/p>\n

RangeUnionFirst
\n.NET 8.0
\n53.670 ns
\n1.00
\n344 B
\n1.00<\/p>\n

RangeUnionFirst
\n.NET 9.0
\n5.181 ns
\n0.10
\n\u2013
\n0.00<\/p>\n

Subsequent PRs also further benefited from this consolidation. dotnet\/runtime#99218<\/a>, for example, uses it to improve Enumerable.Any(IEnumerable<T>). Any just needs to say whether the source has any elements, and it tries hard to determine that without having to get an enumerator from the source, which allocates, and call MoveNext (an interface call) to see if it returns true. In .NET 8, it was doing this using Enumerable.TryGetNonEnumeratedCount, which uses Iterator<T>.GetCount(onlyIfCheap: true) (the \u201conlyIfCheap\u201d part basically means \u201cdon\u2019t enumerate to compute the count\u201d). However, for iterators where it\u2019s not \u201ccheap\u201d, TryGetNonEnumeratedCount would return false, and Any would still be forced to get an enumerator. However, now that every Iterator<T> provides a TryGetFirst, Any can use that in the case where the source is an Iterator<T> but GetCount isn\u2019t successful. Worst case, TryGetFirst will itself end up calling GetEnumerator, but best case, the iterator will have provided a more efficient implementation of TryGetFirst. And either way, it\u2019s still a win, because enumerating would require not only a GetEnumerator call on the Iterator<T>, but that in turn would need to call GetEnumerator<T> on whatever source it was wrapping, whereas this ends up saving one layer.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<int> _data1 = Iterations(100).Where(i => i % 2 == 0).Select(i => i);
\n private IEnumerable<int> _data2 = Enumerable.Range(0, 100).ToArray().Where(i => i % 2 == 0).Select(i => i);<\/p>\n

[Benchmark] public bool Any1() => _data1.Any();
\n [Benchmark] public bool Any2() => _data2.Any();<\/p>\n

private static IEnumerable<int> Iterations(int count)
\n {
\n for (int i = 0; i < count; i++) yield return i;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Any1
\n.NET 8.0
\n31.967 ns
\n1.00
\n104 B
\n1.00<\/p>\n

Any1
\n.NET 9.0
\n15.818 ns
\n0.49
\n40 B
\n0.38<\/p>\n

Any2
\n.NET 8.0
\n21.062 ns
\n1.00
\n56 B
\n1.00<\/p>\n

Any2
\n.NET 9.0
\n3.780 ns
\n0.18
\n\u2013
\n0.00<\/p>\n

Another cross-cutting improvement across LINQ comes in dotnet\/runtime#96602<\/a> and has to do with empty inputs. It\u2019s also a nice example of how what\u2019s considered an optimization ebbs and flows. In the beginning of LINQ, Enumerable.Empty<T>(), which is strongly-typed to return IEnumerable<T>, returned an empty T[] as the actual instance. When Array.Empty<T>() was introduced, it used that. Then, however, the aforementioned IPartition<T> was introduced internally in LINQ, and Enumerable.Empty<T>() was changed to return a singleton EmptyPartition<T>, an implementation of the interface with all of the methods dedicated to being efficient for empty inputs. This was helpful internally as an implementation detail, as methods that were typed to return IPartition<T> could return that EmptyPartition<T> instance, whereas they couldn\u2019t return a T[], since it doesn\u2019t implement that interface. However, it had a downside. A variety of APIs can optimize very well if they know the input is empty, e.g. a Take call can immediately return an empty singleton if it knows the input is empty. But, it can\u2019t be based solely on whether it\u2019s empty now<\/em>, but rather if it\u2019s empty now and for always; otherwise, you could call Take, it would see it was empty, then elements get added to the source, and then you call GetEnumerator on the enumerable returned from Take\u2026 according to the rules for how all of this behaves, that should yield the newly-added elements, but if Take had returned an empty singleton, it wouldn\u2019t. There are a variety of types that we know will always be empty once seen as empty (e.g. ImmutableArray<T>, T[], FrozenSet<T>, etc.), but it\u2019d be too costly to check for each of them individually. Instead, the implementation just picked the same type as Enumerable.Empty<T>() returned as the one to check for. That\u2019s fairly reasonable, but as it turns out, when that type is EmptyPartition<T>, there are a lot of empty arrays that are no longer noticed as being a special empty input. This gets even worse with collection expressions in the picture, as initializing an IEnumerable<T> with [] will, as an implementation detail, produce Array.Empty<T>(). So, this PR put everything back on a plan of Enumerable.Empty<T>() being Array.Empty<T>() and a T[0] being what\u2019s checked for when special-casing empty inputs. The PR also included new checks for empty in many different places that warranted it.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private string[] _values = [];<\/p>\n

[Benchmark] public object Chunk() => _values.Chunk(10);
\n [Benchmark] public object Distinct() => _values.Distinct();
\n [Benchmark] public object GroupJoin() => _values.GroupJoin(_values, i => i, i => i, (i, j) => i);
\n [Benchmark] public object Join() => _values.Join(_values, i => i, i => i, (i, j) => i);
\n [Benchmark] public object ToLookup() => _values.ToLookup(i => i);
\n [Benchmark] public object Reverse() => _values.Reverse();
\n [Benchmark] public object SelectIndex() => _values.Select((s, i) => i);
\n [Benchmark] public object SelectMany() => _values.SelectMany(i => i);
\n [Benchmark] public object SkipWhile() => _values.SkipWhile(i => true);
\n [Benchmark] public object TakeWhile() => _values.TakeWhile(i => true);
\n [Benchmark] public object WhereIndex() => _values.Where((s, i) => true);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Chunk
\n.NET 8.0
\n10.7213 ns
\n1.00
\n72 B
\n1.00<\/p>\n

Chunk
\n.NET 9.0
\n4.1320 ns
\n0.39
\n\u2013
\n0.00<\/p>\n

Distinct
\n.NET 8.0
\n9.4410 ns
\n1.00
\n64 B
\n1.00<\/p>\n

Distinct
\n.NET 9.0
\n0.7162 ns
\n0.08
\n\u2013
\n0.00<\/p>\n

GroupJoin
\n.NET 8.0
\n22.4746 ns
\n1.00
\n144 B
\n1.00<\/p>\n

GroupJoin
\n.NET 9.0
\n1.1356 ns
\n0.05
\n\u2013
\n0.00<\/p>\n

Join
\n.NET 8.0
\n18.6332 ns
\n1.00
\n168 B
\n1.00<\/p>\n

Join
\n.NET 9.0
\n1.3585 ns
\n0.07
\n\u2013
\n0.00<\/p>\n

ToLookup
\n.NET 8.0
\n23.3518 ns
\n1.00
\n128 B
\n1.00<\/p>\n

ToLookup
\n.NET 9.0
\n0.9539 ns
\n0.04
\n\u2013
\n0.00<\/p>\n

Reverse
\n.NET 8.0
\n9.5791 ns
\n1.00
\n48 B
\n1.00<\/p>\n

Reverse
\n.NET 9.0
\n0.9947 ns
\n0.10
\n\u2013
\n0.00<\/p>\n

SelectIndex
\n.NET 8.0
\n11.1235 ns
\n1.00
\n72 B
\n1.00<\/p>\n

SelectIndex
\n.NET 9.0
\n0.5603 ns
\n0.05
\n\u2013
\n0.00<\/p>\n

SelectMany
\n.NET 8.0
\n10.7537 ns
\n1.00
\n64 B
\n1.00<\/p>\n

SelectMany
\n.NET 9.0
\n0.9906 ns
\n0.09
\n\u2013
\n0.00<\/p>\n

SkipWhile
\n.NET 8.0
\n11.2900 ns
\n1.00
\n72 B
\n1.00<\/p>\n

SkipWhile
\n.NET 9.0
\n1.0988 ns
\n0.10
\n\u2013
\n0.00<\/p>\n

TakeWhile
\n.NET 8.0
\n11.8818 ns
\n1.00
\n72 B
\n1.00<\/p>\n

TakeWhile
\n.NET 9.0
\n1.0381 ns
\n0.09
\n\u2013
\n0.00<\/p>\n

WhereIndex
\n.NET 8.0
\n11.1751 ns
\n1.00
\n80 B
\n1.00<\/p>\n

WhereIndex
\n.NET 9.0
\n1.2185 ns
\n0.11
\n\u2013
\n0.00<\/p>\n

dotnet\/runtime#98963<\/a> also has to do with emptiness, but actually improves non-empty cases. DefaultIfEmpty needs to produce an IEnumerable<T> containing all of the elements from the source, or if the source is empty, an enumerable with a single default(T) value. In most cases, that means it has to allocate a new enumerable, because for the same reasons as just described, it can\u2019t know until GetEnumerator is called whether the source is empty. Except, it can if the source is a T[], which has an immutable length. This PR thus special-cases arrays, which are very common, such that if the array isn\u2019t empty, it\u2019s just returned directly rather than allocating a wrapper enumerable for it. That\u2019s more than just about avoiding an allocation: that wrapper object would be in the middle of all subsequent iterations of the object, so avoiding it not only avoids an allocation but also a layer of interface calls. And for subsequent code paths that special-case arrays, the result of DefaultIfEmpty will still be seen as an T[] and thus now special-cased, whereas it wouldn\u2019t if it were wrapped.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _data = Enumerable.Range(0, 1000).ToArray();<\/p>\n

[Benchmark]
\n public double Average() => _data.DefaultIfEmpty().Average();
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Average
\n.NET 8.0
\n1,915.4 ns
\n1.00
\n80 B
\n1.00<\/p>\n

Average
\n.NET 9.0
\n117.6 ns
\n0.06
\n\u2013
\n0.00<\/p>\n

Another change taking advantage of emptiness is dotnet\/runtime#99256<\/a>, this time for Enumerable.Chunk. Chunk(int size) creates an IEnumerable<T[]> that pages through the input size elements at a time. Normally, this requires iterating through the source and buffering until size elements have been reached, then yielding an array with those elements, and then rinsing and repeating. With an array input, we could do this much more efficiently, as we could just do math to compute the right bounds for each set to be yielded and do an efficient copy of the elements, rather than iterating through each element one by one. And while it might not be worth adding a specialized check for array here (Chunk isn\u2019t exactly a high-performance method to begin with, given it\u2019s allocating a new array for each set), as it turns out we now have a check for array, as part of determining whether the source is permanently empty. This PR just leverages that check to take advantage of both answers. If the array is empty, then it still just returns an empty array. But if it\u2019s not empty, rather than falling back to the normal iteration path, it employs a 7-line alternative that\u2019s specialized to arrays and much more efficient.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _values = Enumerable.Range(0, 1000).ToArray();<\/p>\n

[Benchmark]
\n public int Count()
\n {
\n int count = 0;
\n foreach (var chunk in _values.Chunk(10)) count += chunk.Length;
\n return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n3.612 us
\n1.00<\/p>\n

Count
\n.NET 9.0
\n1.334 us
\n0.37<\/p>\n

The statement about checking for empty now and permanently applies in particular to methods that accept and return enumerables. It\u2019s the laziness of these methods that makes that relevant. There is, however, a set of LINQ methods that are not lazy because they produce things that aren\u2019t enumerables, such as ToArray returning an array, Sum returning a single value, Count returning an int, and so on. These methods also received attention, thanks to dotnet\/runtime#102884<\/a> from @neon-sunset<\/a>. One of the optimizations applied in various LINQ methods is to special-case input types that are super common, in particular T[] and List<T>. These can be special-cased not just as IList<T>, which would generally be more efficient than enumerating an input via an IEnumerator<T>, but rather as a ReadOnlySpan<T>, which can be iterated through very efficiently. This PR extended that optimization to apply to most of these other non-enumerable producing methods, in particular overloads of Any, All, Count, First, and Single that take predicates. This is particularly helpful because recent additions to analyzers have resulted in developers being told about opportunities to simplify their LINQ usage. IDE0120<\/a> flags code like source.Where(predicate).First() and instead recommends simplifying it to source.First(predicate). And while that is a nice simplification and is likely to reduce allocation, Where is considerably more optimized than First(predicate) has been, with the former having special-casing for T[] and List<T> but the latter historically not. That difference is now addressed in .NET 9.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<int> _list = Enumerable.Range(0, 1000).ToList();<\/p>\n

[Benchmark] public bool Any() => _list.Any(i => i == 1000);
\n [Benchmark] public bool All() => _list.All(i => i >= 0);
\n [Benchmark] public int Count() => _list.Count(i => i == 0);
\n [Benchmark] public int First() => _list.First(i => i == 999);
\n [Benchmark] public int Single() => _list.Single(i => i == 0);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Any
\n.NET 8.0
\n1,553.3 ns
\n1.00
\n40 B
\n1.00<\/p>\n

Any
\n.NET 9.0
\n222.2 ns
\n0.14
\n\u2013
\n0.00<\/p>\n

All
\n.NET 8.0
\n1,586.0 ns
\n1.00
\n40 B
\n1.00<\/p>\n

All
\n.NET 9.0
\n224.9 ns
\n0.14
\n\u2013
\n0.00<\/p>\n

Count
\n.NET 8.0
\n1,535.6 ns
\n1.00
\n40 B
\n1.00<\/p>\n

Count
\n.NET 9.0
\n244.6 ns
\n0.16
\n\u2013
\n0.00<\/p>\n

First
\n.NET 8.0
\n1,600.7 ns
\n1.00
\n40 B
\n1.00<\/p>\n

First
\n.NET 9.0
\n245.4 ns
\n0.15
\n\u2013
\n0.00<\/p>\n

Single
\n.NET 8.0
\n1,550.6 ns
\n1.00
\n40 B
\n1.00<\/p>\n

Single
\n.NET 9.0
\n239.4 ns
\n0.15
\n\u2013
\n0.00<\/p>\n

dotnet\/runtime#97004<\/a> from @neon-sunset<\/a> uses that same mechanism to improve performance for List<T> inputs inside of Enumerable.SequenceEqual. Enumerable.SequenceEqual already had a special-case that checked whether both inputs were arrays, and if they were, it created spans from those arrays and delegated to MemoryExtensions.SequenceEquals, which will efficiently iterate through the spans, vectorizing if possible. This PR just tweaked that special-case to use the same helper that\u2019s used elsewhere to try to get a span from the source, and that gives this super power to List<T> as well.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<int> _source1, _source2;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _source1 = Enumerable.Range(0, 10_000).ToArray();
\n _source2 = _source1.ToList();
\n }<\/p>\n

[Benchmark]
\n public bool SequenceEqual() => _source1.SequenceEqual(_source2);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

SequenceEqual
\n.NET 8.0
\n26,623.3 ns
\n1.00<\/p>\n

SequenceEqual
\n.NET 9.0
\n913.4 ns
\n0.03<\/p>\n

ToArray and ToList were generally improved via a variety of PRs, in particular by dotnet\/runtime#96570<\/a>. ToArray in particular is used so ubiquitously that over the years, many folks have attempted to optimize it. In doing so, however, it\u2019s gotten too complex for its own good. This PR takes advantage of newer runtime capabilities to significantly simplify the implementation, while also improving common case performance. The easy cases were already handled well and continue to be: if the source is an ICollection<T>, its Count \/ CopyTo methods can be used to provide a very efficient ToArray, and if the source is an Iterator<TSource>, ToArray just delegates to the iterator\u2019s ToArray implementation. The challenge, instead, is in efficiently handling the case where we\u2019re dealing with an IEnumerable<T> of unknown length, needing to handle both short and long inputs, and doing so in a way that minimizes allocation and maximizes throughput. To achieve that, this PR deleted the internal ArrayBuilder, LargeArrayBuilder, and SparseArrayBuilder types that were previously being used and replaced them all with a simpler internal SegmentedArrayBuilder. The builder is seeded with an [InlineArray]-based struct that\u2019s large enough to hold eight T instances. For up to eight items, the builder can simply use that stack-based space to store the elements. For more than eight items, the builder contains another [InlineArray] of up to 27 T[]s. The arrays stored in there are rented from the ArrayPool<T>, and based on the starting size and standard doubling growth algorithm, 27 arrays is enough to store Array.MaxLength elements. This approach means that small inputs never need to allocate (other than for the final T[], which is unavoidable as it\u2019s the whole purpose of the method), and larger inputs can use ArrayPool<T> arrays without having to copy while growing, leading to on average significantly less allocation than before and generally faster throughput. There are trade-offs to this approach when compared to the previous one, with a few niche corner cases it doesn\u2019t handle quite as efficiently, but on the whole it\u2019s an improvement in both performance and maintainability.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(1)]
\n [Arguments(8)]
\n [Arguments(500)]
\n public string[] IteratorToArray(int count) => GetItems(count).ToArray();<\/p>\n

private IEnumerable<string> GetItems(int count)
\n {
\n for (int i = 0; i < count; i++)
\n {
\n yield return “.NET 9”;
\n }
\n }
\n}<\/p>\n

Method
\nRuntime
\ncount
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

IteratorToArray
\n.NET 8.0
\n1
\n65.51 ns
\n1.00
\n136 B
\n1.00<\/p>\n

IteratorToArray
\n.NET 9.0
\n1
\n41.39 ns
\n0.63
\n80 B
\n0.59<\/p>\n

IteratorToArray
\n.NET 8.0
\n8
\n103.30 ns
\n1.00
\n192 B
\n1.00<\/p>\n

IteratorToArray
\n.NET 9.0
\n8
\n74.66 ns
\n0.72
\n136 B
\n0.71<\/p>\n

IteratorToArray
\n.NET 8.0
\n500
\n3,100.69 ns
\n1.00
\n8536 B
\n1.00<\/p>\n

IteratorToArray
\n.NET 9.0
\n500
\n3,080.31 ns
\n0.99
\n4072 B
\n0.48<\/p>\n

dotnet\/runtime#104365<\/a> from @andrewjsaid<\/a> followed-up on this to use that same SegmentedArrayBuilder to improve ToList. Everything stays the same, except for the last step of constructing the final collection to be returned: rather than allocating an array, it allocates a List<T> and uses the CollectionsMarshal.SetCount method to set both the Capacity and Count of the list to the desired size, then copies the elements directly into the backing array for the list, thanks to CollectionsMarshal.AsSpan. ToList was also improved in dotnet\/runtime#86796<\/a> from @brantburnett<\/a>. In various Iterator<T>.ToList specializations, the common pattern is to use List<T>.Add to fill in the resulting collection. This PR used a similar approach as with the previous PR, using a combination of CollectionsMarshal.SetCount and CollectionsMarshal.AsSpan to get a span for the list and directly write into the span. This saves some of the overhead from List<T>.Add, including bounds checks that would otherwise occur when writing to its backing array.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public List<int> IteratorSelectToList() => GetItems(8).Select(i => i).ToList();<\/p>\n

[Benchmark]
\n public List<int> IteratorWhereSelectToList() => GetItems(8).Where(i => true).Select(i => i).ToList();<\/p>\n

private IEnumerable<int> GetItems(int count)
\n {
\n for (int i = 0; i < count; i++)
\n {
\n yield return i;
\n }
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

IteratorSelectToList
\n.NET 8.0
\n75.14 ns
\n1.00
\n224 B
\n1.00<\/p>\n

IteratorSelectToList
\n.NET 9.0
\n67.50 ns
\n0.90
\n184 B
\n0.82<\/p>\n

IteratorWhereSelectToList
\n.NET 8.0
\n94.84 ns
\n1.00
\n288 B
\n1.00<\/p>\n

IteratorWhereSelectToList
\n.NET 9.0
\n89.42 ns
\n0.94
\n248 B
\n0.86<\/p>\n

A few more tweaks were made to ToList and ToArray in dotnet\/runtime#95224<\/a> from @Windows10CE<\/a> and dotnet\/runtime#100218<\/a>. The former improved ToList on the result of a Distinct or Union by enabling HashSet<T>\u2018s CopyTo implementation to be used; previously a custom function was manually iterating through the set, and this PR deleted that code (yay!) and just used List<T>\u2018s constructor directly. The latter PR also improved Distinct and Union, but for ToArray, and specifically in the case where it would have allocated a 0-length array when the source was empty. dotnet\/runtime#99639<\/a> also improved ToArray and ToList on the result of an OrderBy; OrderBy\u2018s iterator already special-cased empty sources, but with small tweaks it could also be made to optimize sources with only a single element, in which case no additional work needs to be done (a length-1 array is inherently sorted).<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public int[] OrderByToArray() => GetItems(1).OrderBy(x => x).ToArray();<\/p>\n

private IEnumerable<int> GetItems(int count)
\n {
\n for (int i = 0; i < count; i++)
\n {
\n yield return i;
\n }
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

OrderByToArray
\n.NET 8.0
\n66.99 ns
\n1.00
\n352 B
\n1.00<\/p>\n

OrderByToArray
\n.NET 9.0
\n53.92 ns
\n0.81
\n160 B
\n0.45<\/p>\n

Not to be left out from the fun its To cousins are having, ToDictionary also sees improvements from dotnet\/runtime#96574<\/a> from @xin9le<\/a>. The PR changes the code to do a better job setting the capacity of the Dictionary<TKey, TValue> prior to filling it, and also using the CollectionsMarshal.AsSpan to share code for handling sources that are arrays and lists, while also shaving off some overhead by enumerating the span instead of the list directly.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private readonly IEnumerable<KeyValuePair<int, int>> _enumerable = Enumerable.Range(0, 10_000).Select(x => new KeyValuePair<int, int>(x, x));<\/p>\n

[Benchmark]
\n public Dictionary<int, KeyValuePair<int, int>> EnumerableToDictionary() => _enumerable.ToDictionary(x => x.Key);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

EnumerableToDictionary
\n.NET 8.0
\n284.3 us
\n1.00
\n788.73 KB
\n1.00<\/p>\n

EnumerableToDictionary
\n.NET 9.0
\n149.9 us
\n0.53
\n237.01 KB
\n0.30<\/p>\n

dotnet\/runtime#96605<\/a> updated Enumerable.Min and Enumerable.Max to specialize for char, Int128, and UInt128 (previous changes specialized other numerical primitives, but these had been left out). By taking advantage of the existing code paths for handling those other primitives, with just a few lines added\/changed, these types can now utilize those faster paths, which in particular special-case arrays and lists (which means it can then avoid an enumerator allocation in addition to faster access to each element).<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Int128[] _values = Enumerable.Range(0, 1000).Select(x => (Int128)x).ToArray();<\/p>\n

[Benchmark]
\n public Int128 Max() => _values.Max();
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Max
\n.NET 8.0
\n1,882.0 ns
\n1.00
\n32 B
\n1.00<\/p>\n

Max
\n.NET 9.0
\n624.7 ns
\n0.33
\n\u2013
\n0.00<\/p>\n

The aforementioned special code paths for the primitive types also support vectorization. Previously that vectorization only supported 128-bit and 256-bit vector widths, but as of dotnet\/runtime#93369<\/a> from @Spacefish<\/a>, it now also supports 512-bit vector widths, possibly doubling the throughput of Enumerable.Min and Enumerable.Max on supported hardware with the core numerical primitive types.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _values = Enumerable.Range(0, 10_000).ToArray();<\/p>\n

[Benchmark]
\n public int Max() => _values.Max();
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Max
\n.NET 8.0
\n327.6 ns
\n1.00<\/p>\n

Max
\n.NET 9.0
\n166.3 ns
\n0.51<\/p>\n

One caveat here about AVX512. Some AVX512 hardware, even on recent chips, can take a measurable amount of time to \u201cpower up,\u201d such that it might be tens, hundreds, or even thousands of cycles before AVX512 processing ends up actually dispatching 512-bit vectors. Until then, the hardware might end up doing the equivalent of dispatching two 256-bit vectors. On my machine, for example, if I lower the size in the previous benchmark from 10,000 elements to 1,000 elements, the .NET 9 improvement disappears and it ends up running at exactly the same throughput as on .NET 8; on a colleague\u2019s machine with a different processor, even at 1,000 elements the .NET 9 throughput is almost twice that of .NET 8. This is all to say, your mileage may vary. In some of the micro-benchmarks discussed in this post, small improvements are made to already very fast operations, and the gains then come from those operations being done many, many, many times on hot paths. In others, the gains come from taking an expensive operation and making it measurably cheaper. In general the benefits with using AVX512 in these kinds of vectorized implementations come for the latter case, where large data sizes lead to operations taking significant amounts of time, and the use of 512-bit vectors instead of 256-bit vectors measurably speeds up those longer operations.<\/p>\n

The OrderBy family of methods on Enumerable were also improved in several ways:<\/p>\n

Ordering operations followed by a First() or Last() call were already specialized to completely avoid the O(N log N) sort and instead do an O(N) search for the min or max. However, OrderBy in LINQ is fairly complicated because it needs to account for the possibility of one or more subsequent ThenBy operations that impact the sort order, and thus it uses a custom comparison mechanism that factors in the possibility of such refinement. That custom comparer mechanism was being used as part of those First\/Last specializations. dotnet\/runtime#97483<\/a> detects whether there are any ThenBys in play, and if there aren\u2019t, it bypasses that customization and, in doing so, avoids its overhead. That can be very measurable, but in certain cases, it can be enormous, as it can enable other optimizations to kick in, e.g. an Order().Last() on an int[] can just end up doing a vectorized search for the max.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.InteropServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private List _ints;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _ints = new(Enumerable.Range(-8000, 8000 * 2));
\n new Random(42).Shuffle(CollectionsMarshal.AsSpan(_ints));
\n }<\/p>\n

[Benchmark]
\n public int OrderByLast_Int32() => _ints.OrderBy(x => x).Last();<\/p>\n

[Benchmark]
\n public int OrderLast_Int32() => _ints.Order().Last();
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

OrderByLast_Int32
\n.NET 8.0
\n34,715.6 ns
\n1.00
\n136 B
\n1.00<\/p>\n

OrderByLast_Int32
\n.NET 9.0
\n25,001.1 ns
\n0.72
\n128 B
\n0.94<\/p>\n

OrderLast_Int32
\n.NET 8.0
\n36,064.9 ns
\n1.00
\n112 B
\n1.00<\/p>\n

OrderLast_Int32
\n.NET 9.0
\n693.8 ns
\n0.02
\n56 B
\n0.50<\/p>\n

In .NET 8, Enumerable.Order was updated to recognize that sorting of certain primitive types is implicitly stable even if an unstable sorting algorithm is used, because any two values of such types that compare equally are indistinguishable in memory (e.g. the only Int32 values that compare equally are those with the exact same bit patterns in memory). dotnet\/runtime#99533<\/a> improves this logic to also handle enums whose underlying type counts.
\ndotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<DayOfWeek> _days = new Random(42).GetItems(Enum.GetValues<DayOfWeek>(), 100);<\/p>\n

[Benchmark]
\n public int Order()
\n {
\n int sum = 0;
\n foreach (DayOfWeek dow in _days.Order())
\n {
\n sum += (int)dow;
\n }
\n return sum;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Order
\n.NET 8.0
\n1,652.9 ns
\n1.00
\n1088 B
\n1.00<\/p>\n

Order
\n.NET 9.0
\n873.0 ns
\n0.53
\n544 B
\n0.50<\/p>\n

In .NET 8, a change was submitted to Enumerable.Range to vectorize its operation when followed by methods like ToArray. At the time, we had some debate about whether to merge the change, with me asking questions like \u201cWho would actually use Enumerable.Range(…).ToArray() on code paths that care about performance?\u201d As it turns out, we do! As part of OrderBy\u2018s stable sort implementation, it had code like this:
\nint[] map = new int[count];
\nfor (int i = 0; i < map.Length; i++)
\n{
\n map[i] = i;
\n}<\/p>\n

For all intents and purposes, that\u2019s Enumerable.Range(0, count).ToArray(). dotnet\/runtime#99538<\/a> recognizes this and uses that same vectorized helper to fill this array in a vectorized manner, and that overhead can actually be measurable in some cases.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEnumerable<int> _data = Enumerable.Range(0, 1000).ToArray();<\/p>\n

[Benchmark]
\n public int OrderBy()
\n {
\n int sum = 0;
\n foreach (int value in _data.OrderBy(i => i))
\n {
\n sum += value;
\n }<\/p>\n

return sum;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

OrderBy
\n.NET 8.0
\n14.83 us
\n1.00<\/p>\n

OrderBy
\n.NET 9.0
\n13.48 us
\n0.91<\/p>\n

GroupBy and ToLookup also get some dedicated improvements in .NET 9, thanks to dotnet\/runtime#99365<\/a>. GetEnumerator on the grouping object returned by these methods was implemented using a simple C# iterator:<\/p>\n

public IEnumerator<TElement> GetEnumerator()
\n{
\n for (int i = 0; i < _count; i++)
\n {
\n yield return _elements[i];
\n }
\n}<\/p>\n

In general we favor using C# iterators over manual implementations (unless we\u2019re going to go all out and implement all of the Iterator<TSource> logic) because C# iterators make the code so simple and maintainable. In this particular case, however, this is a reasonably common hot path and we can actually do meaningfully better by hand than the compiler is able to do today. When the compiler generates a state machine for the previous iterator, it does so with a dedicated state field, but with a manual implementation, we can use the same field for state as we use for the iteration variable, which also means we only need to update one thing per loop iteration.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private ILookup<int, string> _stringsByLength =
\n (from i in Enumerable.Range(0, 10)
\n from item in Enumerable.Range(0, 8)
\n select new string((char)(‘a’ + item), i + 1)).ToLookup(s => s.Length);<\/p>\n

[Benchmark]
\n public int Sum()
\n {
\n int sum = 0;
\n foreach (IGrouping<int, string> group in _stringsByLength)
\n {
\n foreach (string item in group)
\n {
\n sum += item.Length;
\n }
\n }
\n return sum;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Sum
\n.NET 8.0
\n290.3 ns
\n1.00<\/p>\n

Sum
\n.NET 9.0
\n267.4 ns
\n0.92<\/p>\n

Core Collections<\/h3>\n

As shared in Performance Improvements in .NET 8<\/a>, Dictionary<TKey, TValue> is one of the most popular collections in all of .NET, by far (probably not surprising to anyone). And in .NET 9, it gets a performance-focused feature I\u2019ve been wanting for years.<\/p>\n

One of the most common uses for a dictionary is as a cache, often indexed by a string key. And for high-performance scenarios, such caches are frequently used in situations where an actual string object may not be available, but where the text is available, just in a different form, like a ReadOnlySpan<char> (or for caches indexed by UTF8 data, the key might be a byte[] yet the data to perform the lookup is only available as a ReadOnlySpan<byte>). Performing the lookup on the dictionary then would either require materializing a string from the data, which makes the lookup more costly (and in some cases can entirely defeat the purposes of the cache), or require using a custom key type that\u2019s capable of handling multiple forms of the data, which then also generally requires a custom comparer.<\/p>\n

This has been addressed in .NET 9 with the introduction of IAlternateEqualityComparer<TAlternate, T>. A comparer that implements IEqualityComparer<T> may now also implement this additional interface one or more times for other TAlternate types, making it possible for that comparer to treat alternate types the same as the T. Then a type like Dictionary<TKey, TValue> can expose additional methods that work in terms of a TAlternateKey and allow them to work if the comparer in that Dictionary<TKey, TValue> implements IAlternateEqualityComparer<TAlternateKey, TKey>. In .NET 9 with dotnet\/runtime#102907<\/a> and dotnet\/runtime#103191<\/a>, Dictionary<TKey, TValue>, ConcurrentDictionary<TKey, TValue>, FrozenDictionary<TKey, TValue>, HashSet<T>, and FrozenSet<T> all do exactly that. For example, here I have a Dictionary<string, int> I\u2019m using to count the number of occurrences of each word in a span:<\/p>\n

static Dictionary<string, int> CountWords1(ReadOnlySpan<char> input)
\n{
\n Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);<\/p>\n

foreach (ValueMatch match in Regex.EnumerateMatches(input, @”bw+b”))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n string key = word.ToString();
\n result[key] = result.TryGetValue(key, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n}<\/p>\n

I\u2019m returning a Dictionary<string, int>, so I certainly need to materialize the string for each ReadOnlySpan<char> in order to store<\/em> it in the dictionary, but I should only need to do so once, the first time the word is found. I shouldn\u2019t need to create a new string each time, yet I\u2019m having to in order to do the TryGetValue call. Now with .NET 9, a new GetAlternateLookup method (and a corresponding TryGetAlternateLookup) exists to produce a separate value type wrapper that enables using an alternate key type for all the relevant operations, which means I can now write this:<\/p>\n

static Dictionary<string, int> CountWords2(ReadOnlySpan<char> input)
\n{
\n Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);
\n Dictionary<string, int>.AlternateLookup<ReadOnlySpan<char>> alternate = result.GetAlternateLookup<ReadOnlySpan<char>>();<\/p>\n

foreach (ValueMatch match in Regex.EnumerateMatches(input, @”bw+b”))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n alternate[word] = alternate.TryGetValue(word, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n}<\/p>\n

Note the distinct lack of a ToString(), which means no allocation will occur here for words already seen. How then does the alternate[word] = … part work? Surely this isn\u2019t storing a ReadOnlySpan<char> into the dictionary? Nope. Rather, IAlternateEqualityComparer<TAlternate, T> looks like this:<\/p>\n

public interface IAlternateEqualityComparer<in TAlternate, T>
\n where TAlternate : allows ref struct
\n where T : allows ref struct
\n{
\n bool Equals(TAlternate alternate, T other);
\n int GetHashCode(TAlternate alternate);
\n T Create(TAlternate alternate);
\n}<\/p>\n

The Equals and GetHashCode should look familiar, the main difference from the corresponding members of IEqualityComparer<T> just being the type of the first parameter. But then there\u2019s this additional Create method. That method accepts a TAlternate and returns a T, giving the comparer the ability to map from one to the other. That setter we saw previously (and other methods like TryAdd) are able to use this to only create the TKey from the TAlternateKey when they have to, so the setter here will only allocate the string for the word if the word doesn\u2019t already exist in the collection.<\/p>\n

Another possibly perplexing thing for anyone reading this and who\u2019s well versed in the ways of ReadOnlySpan<T>: how in the world is Dictionary<string, int>.AlternateLookup<ReadOnlySpan<char>> valid? ref structs like span can\u2019t be used as generic parameters, right? Right\u2026 until now. C# 13 and .NET 9 now permit ref structs as generic parameters, but the generic parameter needs to opt-in to it via the new allows ref struct constraint (or \u201canti-constraint\u201d as some of us frequently refer to it). There are things a method can do with an instance of an unconstrained generic parameter, like cast it to object or store it into a field of a class, that can\u2019t be done with ref struct. By adding allows ref struct to a generic parameter, it tells the compiler compiling the consumer that it may specify a ref struct, and it tells the compiler compiling the type or method with the constraint that the generic instantiation might be a ref struct and thus the generic parameter can only be used in situations where a ref struct would be legal.<\/p>\n

Of course, all of this working hinges on the supplied comparer sporting the appropriate IAlternateEqualityComparer<TAlternate, T> implementation; if it doesn\u2019t, attempts to call GetAlternateLookup will throw an exception, and attempts to call TryGetAlternateLookup will return false. You can use these collection types with whatever comparer you want, and that comparer can provide implementations of this interface for whatever alternate key types you want. But with string and ReadOnlySpan<char> being so common, it\u2019d be a shame if there wasn\u2019t built-in support for this combination. And indeed, with the aforementioned PRs, all of the built-in StringComparer types implement IAlternateEqualityComparer<ReadOnlySpan<char>, string>. That\u2019s why the Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase); line is successful in the previous code example, as the subsequent call to result.GetAlternateLookup<ReadOnlySpan<char>>() will successfully find the interface on the supplied comparer.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic partial class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;<\/p>\n

[GeneratedRegex(@”bw+b”)]
\n private static partial Regex WordParser();<\/p>\n

[Benchmark(Baseline = true)]
\n public Dictionary<string, int> CountWords1()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);<\/p>\n

foreach (ValueMatch match in WordParser().EnumerateMatches(input))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n string key = word.ToString();
\n result[key] = result.TryGetValue(key, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n }<\/p>\n

[Benchmark]
\n public Dictionary<string, int> CountWords2()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);
\n Dictionary<string, int>.AlternateLookup<ReadOnlySpan<char>> alternate = result.GetAlternateLookup<ReadOnlySpan<char>>();<\/p>\n

foreach (ValueMatch match in WordParser().EnumerateMatches(input))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n alternate[word] = alternate.TryGetValue(word, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

CountWords1
\n60.35 ms
\n1.00
\n20.67 MB
\n1.00<\/p>\n

CountWords2
\n57.40 ms
\n0.95
\n2.54 MB
\n0.12<\/p>\n

Note the huge reduction in allocation.<\/p>\n

For fun, we can also take this example one step further. .NET 6 introduced the CollectionsMarshal.GetValueRefOrAddDefault method, which returns a writable ref to the actual location where the TValue for a given TKey is stored, creating the entry in the dictionary if it doesn\u2019t exist. This is very handy for operations like the one used above, as it helps to avoid an extra dictionary lookup. Without it, we\u2019re doing one lookup as part of the TryGetValue and then another lookup as part of the setter, but with it, we just do the single lookup as part of GetValueRefOrAddDefault and then no additional lookup is necessary because we already have the location into which we can directly write. And as the lookups in this benchmark are one of the more costly elements, eliminating half of them can significantly reduce the cost of the operation. As part of this alternate key effort, a new overload of GetValueRefOrAddDefault was added that works with it, such that the same operation can be performed with a TAlternateKey.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.InteropServices;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic partial class Tests
\n{
\n private static readonly string s_input = new HttpClient().GetStringAsync(“https:\/\/gutenberg.org\/cache\/epub\/2600\/pg2600.txt”).Result;<\/p>\n

[GeneratedRegex(@”bw+b”)]
\n private static partial Regex WordParser();<\/p>\n

[Benchmark(Baseline = true)]
\n public Dictionary<string, int> CountWords1()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);<\/p>\n

foreach (ValueMatch match in WordParser().EnumerateMatches(input))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n string key = word.ToString();
\n result[key] = result.TryGetValue(key, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n }<\/p>\n

[Benchmark]
\n public Dictionary<string, int> CountWords2()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);
\n Dictionary<string, int>.AlternateLookup<ReadOnlySpan<char>> alternate = result.GetAlternateLookup<ReadOnlySpan<char>>();<\/p>\n

foreach (ValueMatch match in WordParser().EnumerateMatches(input))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n alternate[word] = alternate.TryGetValue(word, out int count) ? count + 1 : 1;
\n }<\/p>\n

return result;
\n }<\/p>\n

[Benchmark]
\n public Dictionary<string, int> CountWords3()
\n {
\n ReadOnlySpan<char> input = s_input;<\/p>\n

Dictionary<string, int> result = new(StringComparer.OrdinalIgnoreCase);
\n Dictionary<string, int>.AlternateLookup<ReadOnlySpan<char>> alternate = result.GetAlternateLookup<ReadOnlySpan<char>>();<\/p>\n

foreach (ValueMatch match in WordParser().EnumerateMatches(input))
\n {
\n ReadOnlySpan<char> word = input.Slice(match.Index, match.Length);
\n CollectionsMarshal.GetValueRefOrAddDefault(alternate, word, out _)++;
\n }<\/p>\n

return result;
\n }
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

CountWords1
\n60.73 ms
\n1.00
\n20.67 MB
\n1.00<\/p>\n

CountWords2
\n54.01 ms
\n0.89
\n2.54 MB
\n0.12<\/p>\n

CountWords3
\n44.38 ms
\n0.73
\n2.54 MB
\n0.12<\/p>\n

\u201cBut wait, there\u2019s more!\u201d dotnet\/runtime#104202<\/a> extends the alternate comparer implementation for string\/ReadOnlySpan<char> further to also apply to EqualityComparer<string>.Default, which means that if you don\u2019t supply a comparer at all, these collection types will still support ReadOnlySpan<char> lookups. That change not only then improves the usability of these new APIs, but it actually had an additional unintended but welcome performance benefit. Previously, EqualityComparer<string>.Default would return an internal GenericEqualityComparer<string> type, derived from EqualityComparer<string>. It wouldn\u2019t be possible to implement IAlternateEqualityComparer<ReadOnlySpan<char>, string> on GenericEqualityComparer<string> because doing so would actually have to be done on GenericEqualityComparer<T>, which would mean every EqualityComparer<T>.Default would support IAlternateEqualityComparer<ReadOnlySpan<char>, T>, and we have no correct way of providing such an implementation for all Ts. Instead, we introduced a new internal non-generic StringEqualityComparer type and made EqualityComparer<T>.Default return an instance of that when T is string (the implementation of Default already knows about and returns a bunch of specialized comparers, this is just one more). In doing so, it made the type that\u2019s used non-generic, which in turn means that in some situations, it eliminates some of the overhead associated with generics.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.CompilerServices;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private IEqualityComparer<string> _comparer = EqualityComparer<string>.Default;
\n private string[] _values = Enumerable.Range(0, 1000).Select(i => i.ToString()).ToArray();<\/p>\n

[Benchmark]
\n public int Count() => CountEquals(_values, “500”, _comparer);<\/p>\n

[MethodImpl(MethodImplOptions.NoInlining)]
\n private static int CountEquals<T>(T[] haystack, T needle, IEqualityComparer<T> comparer)
\n {
\n int count = 0;
\n foreach (T value in haystack)
\n {
\n if (comparer.Equals(value, needle))
\n {
\n count++;
\n }
\n }
\n return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\n4.477 us
\n1.00<\/p>\n

Count
\n.NET 9.0
\n2.808 us
\n0.63<\/p>\n

HashSet<T> also gains all of these super powers, but several additional PRs went into making other performance improvements to it. dotnet\/runtime#85877<\/a> from @hrrrrustic<\/a> added a TrimExcess(int capacity) method to HashSet<T> (as well as to Queue<T> and Stack<T>), enabling more fine-grained control over how much memory to cull from a set that might have grown larger than is now required. And dotnet\/runtime#102758<\/a> from @lilinus<\/a> improved its IsSubsetOf, IsPropertSubsetOf, and SetEquals methods by tweaking the fast paths already present. The methods were attempting to early-exit as soon as the condition could be proved true or false, but some common conditions were being missed, and this rectified those.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private HashSet<int> _set = new(Enumerable.Range(0, 1000));
\n private List<int> _list = Enumerable.Range(0, 999).ToList();<\/p>\n

[Benchmark]
\n public bool IsSubsetOf() => _set.IsSubsetOf(_list);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

IsSubsetOf
\n.NET 8.0
\n7,351.373 ns
\n1.000
\n40 B
\n1.00<\/p>\n

IsSubsetOf
\n.NET 9.0
\n1.216 ns
\n0.000
\n\u2013
\n0.00<\/p>\n

dotnet\/runtime#96573<\/a> from @ndsvw<\/a> also identified a few places in various libraries where a Dictionary<T, T> was being used as a set and replaced them with HashSet<T>. The implementations of Dictionary<> and HashSet<> are very close in nature, but the latter consumes less memory because it doesn\u2019t need to store separate values. Using a Dictionary<T, T> effectively doubles the required storage, so if a HashSet<T> suffices, it\u2019s preferable.<\/p>\n

A variety of other collection types have also seen improvements in .NET 9:<\/p>\n

PriorityQueue<TElement, TPriority>.<\/strong> The EnqueueRange(IEnumerable<Telement>, TPriority) method enables multiple elements to all be inserted at the same priority. If there are already elements in the collection, this is akin to just calling Enqueue for each. However, if the collection is currently empty, then it can skip the per-element addition costs and instead just store the elements directly into the backing array. After doing so, it was then performing a heapify operation. But dotnet\/runtime#99139<\/a> from @skyoxZ<\/a> recognized that this heapify was entirely unnecessary, because all of the elements were inserted at the same priority, and there were no elements of any other priority already in the collection. Many performance optimizations come with trade-offs, making one common thing much faster at the expense of making some less common thing a little slower. This, however, is my favorite kind of optimization: elimination of unnecessary work with effectively zero downside.
\nusing BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[DisassemblyDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private PriorityQueue<int, int> _pq = new();
\n private int[] _elements = Enumerable.Range(0, 100).ToArray();<\/p>\n

[Benchmark]
\n public void EnqueueRange()
\n {
\n _pq.Clear();
\n _pq.EnqueueRange(_elements, priority: 42);
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

EnqueueRange
\n.NET 8.0
\n239.3 ns
\n1.00<\/p>\n

EnqueueRange
\n.NET 9.0
\n206.7 ns
\n0.86<\/p>\n

BitArray.<\/strong> Multiple methods on BitArray are already accelerated using Vector128 and Vector256, enabling much faster throughput for methods like And, Or, and Not. dotnet\/runtime#91903<\/a> from @khushal1996<\/a> adds Vector512 support to all of these as well, enabling hardware with AVX512 support to process these operations upwards of twice as fast as before.
\nusing BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Collections;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private BitArray _first = new BitArray(1024 * 1024);
\n private BitArray _second = new BitArray(1024 * 1024);<\/p>\n

[Benchmark]
\n public void Or() => _first.Or(_second);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Or
\n.NET 8.0
\n2.894 us
\n1.00<\/p>\n

Or
\n.NET 9.0
\n2.354 us
\n0.81<\/p>\n

List<T>.<\/strong> dotnet\/runtime#90089<\/a> from @karakasa<\/a> avoided an extra Array.Copy call as part of Insert. Previously the implementation may have done up to three Array.Copy operations, and as part of this change that can drop to just two.
\nFrozenDictionary<TKey, TValue> and FrozenSet<T>.<\/strong> These frozen collections introduced in .NET 8 have also received some attention in .NET 9. As a reminder, FrozenDictionary<TKey, TValue> and FrozenSet<T> are immutable collections optimized for reading, willing to spend more time and effort during construction to make subsequent operations on the collections as fast as possible. When the TKey\/T is a string, one optimization employed is to track the minimum and maximum lengths of all strings in the collection; if a string that\u2019s shorter or longer than that is used in a lookup, the collection can immediately report that it\u2019s not in the collection without having to actually perform any lookup, instead just comparing against the min and max. dotnet\/runtime#92546<\/a> from @andrewjsaid<\/a> extends this further by employing a bitmap of up to 64 bits corresponding to lengths of strings contained in the collection. On lookup, rather than only comparing against min\/max, the implementation can test whether the corresponding bit for the string\u2018s length is set, bailing immediately if it\u2019s not. dotnet\/runtime#100998<\/a> also reduced creation overheads with frozen collections created with string keys and StringComparer.OrdinalIgnoreCase. The implementation had been using its own custom comparison logic for hash code generation, in order to support building for netstandard2.0 in addition to .NET Core, but this PR specialized the code for .NET Core to use string.GetHashCode(ReadOnlySpan<char>, StringComparison), which is more efficient than the custom implementation.
\nusing BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Collections.Frozen;
\nusing System.Text.RegularExpressions;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly FrozenSet<string> s_words = Regex.Matches(“””
\n Let me not to the marriage of true minds
\n Admit impediments; love is not love
\n Which alters when it alteration finds,
\n Or bends with the remover to remove.
\n O no, it is an ever-fixed mark
\n That looks on tempests and is never shaken;
\n It is the star to every wand’ring bark
\n Whose worth’s unknown, although his height be taken.
\n Love’s not time’s fool, though rosy lips and cheeks
\n Within his bending sickle’s compass come.
\n Love alters not with his brief hours and weeks,
\n But bears it out even to the edge of doom:
\n If this be error and upon me proved,
\n I never writ, nor no man ever loved.
\n “””, @”bw+b”).Cast<Match>().Select(w => w.Value).ToFrozenSet();
\n private string _word = “quickness”;<\/p>\n

[GlobalSetup] public void Setup() => Console.WriteLine(s_words);<\/p>\n

[Benchmark]
\n public bool Contains() => s_words.Contains(_word);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Contains
\n.NET 8.0
\n4.373 ns
\n1.00<\/p>\n

Contains
\n.NET 9.0
\n1.154 ns
\n0.26<\/p>\n

Compression<\/h2>\n

It\u2019s an important goal of the core .NET libraries to be as platform-agnostic as possible. Things should generally behave the same way regardless of which operating system or which hardware is being used, excepting things that really are operating system or hardware specific (e.g. we purposefully don\u2019t try to paper over casing differences of different file systems). To that end, we generally implement as much as possible in C#, deferring down to the operating system and native platform libraries only when necessary; for example, the default .NET HTTP implementation, System.Net.Http.SocketsHttpHandler, is written in C# on top of System.Net.Sockets, System.Net.Dns, etc., and subject to the implementation of sockets on each platform (where behaviors are implemented by the operating system), generally behaves the same wherever you\u2019re running.<\/p>\n

There are, however, just a few specific places where we\u2019ve actively made the choice to defer more to something in the platform. The most important case here is cryptography, where we want to rely on the operating system for such security-related functionality; so on Windows, for example, TLS is implemented in terms of components like SChannel, on Linux in terms of OpenSSL, and on macOS in terms of SecureTransport. The other notable case has been compression, and in particular zlib. We decided long ago to simply use whatever zlib was distributed with the operating system. That has had various implications, however. Starting with the fact that Windows doesn\u2019t ship with zlib as a library exposed for consumption, so the .NET build targeting Windows still had to include its own copy of zlib. That was then improved but also complicated by a decision to switch to distribute a variant of zlib produced by Intel, which was nicely optimized further for x64, but which didn\u2019t have as much attention paid to other hardware, like Arm64. And very recently, the intel\/zlib repository was archived and is not actively being maintained by Intel.<\/p>\n

To simplify things, to improve consistency and performance across more platforms, and to move to an actively supported and evolving implementation, this changes for .NET 9. Thanks to a stream of PRs, and in particular dotnet\/runtime#104454<\/a> and dotnet\/runtime#105771<\/a>, .NET 9 now includes the zlib functionality built-in across Windows, Linux, and macOS, based on the newer zlib-ng\/zlib-ng<\/a>. zlib-ng is a zlib-compatible API that is actively maintained, includes improvements previously made to both Intel and Cloudflare\u2019s forks, and has received improvements across many different CPU intrinsics.<\/p>\n

Benchmarking just throughput is easy with BenchmarkDotNet. Unfortunately, while I love the tool, the lack of dotnet\/BenchmarkDotNet#784<\/a> makes it very challenging to appropriately benchmark compression, because throughput is only one part of the equation. Compression ratio is also a key part of the equation (you can make \u201ccompression\u201d super fast by just outputting the input without actually manipulating it at all), so we also need to know about compressed output size when discussing compression speeds. To do that for this post, I\u2019ve hacked up just enough in this benchmark to make it work for this example, implementing a custom column for BenchmarkDotNet, but please note this is not a general-purpose implementation.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Columns;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Reports;
\nusing BenchmarkDotNet.Running;
\nusing System.IO.Compression;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(
\n args,
\n DefaultConfig.Instance.AddColumn(new CompressedSizeColumn()));<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private byte[] _uncompressed = new HttpClient().GetByteArrayAsync(@”https:\/\/www.gutenberg.org\/cache\/epub\/3200\/pg3200.txt”).Result;<\/p>\n

[Params(CompressionLevel.NoCompression, CompressionLevel.Fastest, CompressionLevel.Optimal, CompressionLevel.SmallestSize)]
\n public CompressionLevel Level { get; set; }<\/p>\n

private MemoryStream _compressed = new MemoryStream();<\/p>\n

private long _compressedSize;<\/p>\n

[Benchmark]
\n public void Compress()
\n {
\n _compressed.Position = 0;
\n _compressed.SetLength(0);<\/p>\n

using (var ds = new DeflateStream(_compressed, Level, leaveOpen: true))
\n {
\n ds.Write(_uncompressed, 0, _uncompressed.Length);
\n }<\/p>\n

_compressedSize = _compressed.Length;
\n }<\/p>\n

[GlobalCleanup]
\n public void SaveSize()
\n {
\n File.WriteAllText(Path.Combine(Path.GetTempPath(), $”Compress_{Level}”), _compressedSize.ToString());
\n }
\n}<\/p>\n

public class CompressedSizeColumn : IColumn
\n{
\n public string Id => nameof(CompressedSizeColumn);
\n public string ColumnName { get; } = “CompressedSize”;
\n public bool AlwaysShow => true;
\n public ColumnCategory Category => ColumnCategory.Custom;
\n public int PriorityInCategory => 1;
\n public bool IsNumeric => true;
\n public UnitType UnitType { get; } = UnitType.Size;
\n public string Legend => “CompressedSize Bytes”;
\n public bool IsAvailable(Summary summary) => true;
\n public bool IsDefault(Summary summary, BenchmarkCase benchmarkCase) => true;
\n public string GetValue(Summary summary, BenchmarkCase benchmarkCase, SummaryStyle style) =>
\n GetValue(summary, benchmarkCase);
\n public string GetValue(Summary summary, BenchmarkCase benchmarkCase) =>
\n File.ReadAllText(Path.Combine(Path.GetTempPath(), $”Compress_{benchmarkCase.Parameters.Items[0].Value}”)).Trim();
\n}<\/p>\n

Running that for .NET 8, I get this:<\/p>\n

Method
\nLevel
\nMean
\nCompressedSize<\/p>\n

Compress
\nNoCompression
\n1.783 ms
\n16015049<\/p>\n

Compress
\nFastest
\n164.495 ms
\n7312367<\/p>\n

Compress
\nOptimal
\n620.987 ms
\n6235314<\/p>\n

Compress
\nSmallestSize
\n867.076 ms
\n6208245<\/p>\n

and for .NET 9, I get this:<\/p>\n

Method
\nLevel
\nMean
\nCompressedSize<\/p>\n

Compress
\nNoCompression
\n1.814 ms
\n16015049<\/p>\n

Compress
\nFastest
\n64.345 ms
\n9578398<\/p>\n

Compress
\nOptimal
\n230.646 ms
\n6276158<\/p>\n

Compress
\nSmallestSize
\n567.579 ms
\n6215048<\/p>\n

A few interesting things to note here:<\/p>\n

On both .NET 8 and .NET 9, there\u2019s an obvious correlation: the more compression is requested, the slower it gets and the smaller the file size becomes.
\nNoCompression, which really just echos the input bytes back as output, produces the exact same compressed size across .NET 8 and .NET 9, as one would hope; the compressed size should be identical to the input size.
\nThe compressed size for SmallestSize is almost the same between .NET 8 and .NET 9; they differ by only ~0.1%, but for that small increase, the SmallestSize throughput ends up being ~35% faster. In both cases, the .NET layer is just passing down a zlib compression level of 9, which is the largest value possible and denotes best-possible compression. It just happens that with zlib-ng, that best possible compression is significantly faster and just a tad bit worse compression-ratio-wise than with zlib.
\nFor Optimal, which is also the default and represents a balanced tradeoff between speed and compression ratio (with 20\/20 hindsight, the name for this member should have been Balanced), the .NET 9 version using zlib-ng is 60% faster while only sacrificing ~0.6% on compression ratio.
\nFastest is interesting. The .NET implementation is just passing down a compression level of 1 to the zlib-ng native code, indicating to choose the fastest speed while still doing some<\/em> compression (0 means don\u2019t compress at all). But the zlib-ng implementation is obviously making different trade-offs than did the older zlib code, as it\u2019s truer to its name: it\u2019s running more than 2x as fast and still compressing, but the compressed output is ~30% larger than the output on .NET 8.<\/p>\n

The net effect of this is, especially if you\u2019re using Fastest, you might want to re-evaluate to see whether the throughput \/ compression ratios meet your needs. If you want to tweak it further, though, you\u2019re no longer limited to just these options. dotnet\/runtime#105430<\/a> adds new constructors to DeflateStream, GZipStream, ZLibStream, and also the unrelated BrotliStream, enabling more fine-grained control over the parameters passed to the native implementations, e.g.<\/p>\n

private static readonly ZLibCompressionOptions s_options = new ZLibCompressionOptions()
\n{
\n CompressionLevel = 2,
\n};
\n…
\nStream sourceStream = …;
\nusing var ds = new DeflateStream(compressed, s_options, leaveOpen: true)
\n{
\n sourceStream.CopyTo(ds);
\n}<\/p>\n

Cryptography<\/h2>\n

Investments in System.Security.Cryptography are generally focused on improving the security of a system, supporting new cryptographic primitives, better integrating with security capabilities of the underlying operating system, and so on. But as cryptography is ever present in most modern systems, it\u2019s also impactful to make the existing functionality more efficient, and a variety of PRs in .NET 9 have done so.<\/p>\n

Let\u2019s start with random number generation. .NET 8 added a new GetItems method to both Random (the core non-cryptographically-secure random number generator) and RandomNumberGenerator (the core cryptographically-secure random number generator). This method is very handy when you need to randomly generate N elements sourced from a specific set of values. For example, if you wanted to write 100 random hex characters to a destination Span<char>, you could do:<\/p>\n

Span<char> dest = stackalloc char[100];
\nRandom.Shared.GetItems(“0123456789abcdef”, dest);<\/p>\n

The core implementation is very simple, and is just a convenience implementation for something you could easily do yourself:<\/p>\n

for (int i = 0; i < dest.Length; i++)
\n{
\n dest[i] = choices[Next(choices.Length)];
\n}<\/p>\n

Easy peasy. However, in some situations we can do better. This implementation ends up making a call to the random number generator for each element, and that roundtrip adds measurable overhead. If we could instead make fewer calls, we could ammortize that overhead across however many elements could be filled by that single call. That\u2019s exactly what dotnet\/runtime#92229<\/a> does. If the number of choices is less than or equal to 256 and a power of two, rather than asking for a random integer for each element, we can instead get a byte for each element, and we can do that in bulk with a single call to NextBytes. The max of 256 choices is because that\u2019s the number of values a byte can represent, and the power of two is so that we can simply mask off unnecessary bits from the byte, which helps to avoid bias. This makes a measurable impact for Random, but even more so for RandomNumberGenerator, where each call to get random bytes requires a transition into the operating system.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Security.Cryptography;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private char[] _dest = new char[100];<\/p>\n

[Benchmark]
\n public void GetRandomHex() => RandomNumberGenerator.GetItems<char>(“0123456789abcdef”, _dest);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

GetRandomHex
\n.NET 8.0
\n58,659.2 ns
\n1.00<\/p>\n

GetRandomHex
\n.NET 9.0
\n746.5 ns
\n0.01<\/p>\n

Sometimes performance improvements are about revisiting past assumptions. .NET 5 added a new GC.AllocateArray method which optionally allows that array to be created on the \u201cpinned object heap,\u201d or POH. Allocating on the POH is the same as allocating normally, except that the GC guarantees that objects on the POH won\u2019t be moved (normally the GC is free to compact the heap, moving objects around in order to reduce fragmentation). This is a useful guarantee for cryptography, which employs defense-in-depth measures like zero\u2019ing out buffers to reduce the chances of an attacker being able to find sensitive information in the memory (or memory dump) of a process. The crypto library wants to be able to allocate some memory, use it to temporarily contain some sensitive information, and then zero out the memory before stopping using it, but if the GC is able to move the object around in the interim, it could end up leaving shadows of the data on the heap. When the POH was introduced, then, System.Security.Cryptography started using it, including for relatively short-lived objects. This is potentially problematic, however. Because the nature of the POH is that objects can\u2019t be moved around, creating short-lived objects on the POH can significantly increase fragmentation, which can in turn increase memory consumption, GC costs, and so on. And as a result, the POH is really only recommended for long-lived objects, ideally ones that you create and then hold onto for the remainder of the process\u2019 lifetime. dotnet\/runtime#99168<\/a> undid System.Security.Cryptography\u2018s reliance on the POH, instead preferring to use native memory (e.g. via NativeMemory.Alloc and NativeMemory.Free) for such needs.<\/p>\n

On the subject of memory, multiple PRs went into the crypto libraries to reduce allocation. Here are some examples:<\/p>\n

Marshaling pointers instead of temporary arrays.<\/strong> The CngKey type exposes properties like ExportPolicy, IsMachineKey, and KeyUsage, all of which utilize an internal GetPropertyAsDword method that P\/Invokes to retrieve an integer from Windows. It was doing so, however, via a shared helper that was allocating a 4-byte byte[], passing that to the OS to fill, and then converting those four bytes into an int. dotnet\/runtime#91521<\/a> changed the interop path to instead just store the int on the stack, passing a pointer to it to the OS, avoiding the need to allocate and parse.
\nSpecial-casing empty.<\/strong> Throughout the core libraries, we rely heavily on Array.Empty<T>() to avoid allocating lots of empty arrays when we could instead just employ singletons. The crypto libraries work with a lot of arrays, and as part of defense-in-depth, will often hand out clones of those arrays rather than handing out the same array to everyone; that\u2019s handled by a shared CloneByteArray helper. As it turns out, however, it\u2019s reasonably common for arrays to be empty, yet CloneByteArray wasn\u2019t special-casing them, and was thus always allocating new arrays even if the input was empty. dotnet\/runtime#93231<\/a> simply special-cased empty input arrays to return themselves rather than clone them.
\nAvoiding unnecessary defensive copies.<\/strong> dotnet\/runtime#97108<\/a> avoids more defensive copies than just those for empty arrays mentioned above. The PublicKey type is passed two AsnEncodedData instances, one for parameters and one for a key value, and both of which it clones to avoid any issues that might arise with that provided instance being mutated. But in some internal uses, the caller is constructing a temporary AsnEncodedData and effectively transferring ownership, yet PublicKey would still then make a defensive copy, even though the temporary could have just been used in its stead. This change enables the original instances to just be used without copy in such cases.
\nUsing collection expressions with spans.<\/strong> One of the really neat things about the collection expressions feature introduced in C# 11 is it allows you to express your intent for what you want and allow the system to implement that as best it can. As part of initializing OidLookup, it had multiple lines that look like this:
\nAddEntry(“1.2.840.10045.3.1.7”, “ECDSA_P256”, new[] { “nistP256”, “secP256r1”, “x962P256v1”, “ECDH_P256” });
\nAddEntry(“1.3.132.0.34”, “ECDSA_P384”, new[] { “nistP384”, “secP384r1”, “ECDH_P384” });
\nAddEntry(“1.3.132.0.35”, “ECDSA_P521”, new[] { “nistP521”, “secP521r1”, “ECDH_P521” });<\/p>\n

This effectively forced it to allocate these arrays, even though the AddEntry method doesn\u2019t actually require the array-ness and just iterates through the supplied values. dotnet\/runtime#100252<\/a> changed AddEntry to take a ReadOnlySpan<string> instead of string[], and changed all the call sites to be collection expressions:<\/p>\n

AddEntry(“1.2.840.10045.3.1.7”, “ECDSA_P256”, [“nistP256”, “secP256r1”, “x962P256v1”, “ECDH_P256”]);
\nAddEntry(“1.3.132.0.34”, “ECDSA_P384”, [“nistP384”, “secP384r1”, “ECDH_P384”]);
\nAddEntry(“1.3.132.0.35”, “ECDSA_P521”, [“nistP521”, “secP521r1”, “ECDH_P521”]);<\/p>\n

allowing the compiler to do the \u201cright thing.\u201d All of those call sites then instead just end up using stack space to store the strings passed to AddEntry, rather than allocating any arrays at all.<\/p>\n

Presizing collections.<\/strong> Many collections, such as List<T> or Dictionary<TKey, TValue>, allow you to create a new one, with no a priori knowledge of how large they\u2019ll grow to be, and internally they handle growing their storage to accommodate additional data. The growth algorithm employed typically involves doubling capacity, as doing so strikes a reasonable balance between possibly wasting some memory and not having to re-grow too frequently. However, such growing does have overhead, avoiding it is desirable, and so many collections offer the ability to pre-size the capacity of the collection, e.g. List<T> has a constructor that accepts an int capacity, where the list will immediately create a backing store large enough to accommodate that many elements. The OidCollection in cryptography didn\u2019t have such a capability even though many of the places it was being created did know the exact required size, which in turn results in unnecessary allocation and copying as the collection grows to reach the target size. dotnet\/runtime#97106<\/a> added such a constructor internally and used it in various places, in order to avoid that overhead. As with OidCollection, CborWriter also lacked the ability to presize, making the aforementioned growth algorithm problem even more stark. dotnet\/runtime#92538<\/a> added such a constructor.
\nAvoiding O(N^2) growth algorithms.<\/strong> dotnet\/runtime#92435<\/a> from @MichalPetryka<\/a> fixes a good example of what happens when you don\u2019t<\/em> employ such a doubling scheme as part of collection resizing. The algorithm used to grow the buffer used by CborWriter would increase the backing buffer by a fixed number of elements each time. A doubling strategy ensures you need no more than O(log N) growth operations, and ensures that N items can be added to a collection in O(N) time, since the number of element copies will be O(2N), which is just O(N) (e.g. if N == 128, and you start with a buffer of size 1, and you grow to 2, then 4, 8, 16, 32, 64, and 128, that\u2019s 1 + 2 + 4 + 8 + 16 + 32 + 64 + 128, which is 255, or just under twice N). But increasing by a fixed number can mean O(N) such operations. And since each growth operation also needs to copy all the elements (assuming the growing is done by array resizing), that makes the algorithm O(N^2). In the extreme, if that fixed number was 1, and we were again growing from 1 to 128 one at a time, that\u2019s just summing all the numbers from 1 to 128, the formula for which is N(N+1)\/2, which is O(N^2). This PR changed CborWriters\u2018s growth strategy to use doubling instead.
\n\/\/ Add a <PackageReference Include=”System.Formats.Cbor” Version=”8.0.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Environments;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Running;
\nusing System.Formats.Cbor;<\/p>\n

var config = DefaultConfig.Instance
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core80).WithNuGet(“System.Formats.Cbor”, “8.0.0”).AsBaseline())
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core90).WithNuGet(“System.Formats.Cbor”, “9.0.0-rc.1.24431.7”));
\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, config);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public CborWriter Test()
\n {
\n const int NumArrayElements = 100_000;<\/p>\n

CborWriter writer = new();
\n writer.WriteStartArray(NumArrayElements);
\n for (int i = 0; i < NumArrayElements; i++)
\n {
\n writer.WriteInt32(i);
\n }
\n writer.WriteEndArray();<\/p>\n

return writer;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Test
\n.NET 8.0
\n25,185.2 us
\n1.00
\n65350.11 KB
\n1.00<\/p>\n

Test
\n.NET 9.0
\n697.2 us
\n0.03
\n1023.82 KB
\n0.02<\/p>\n

Of course, improving performance is more than just avoiding allocation. A variety of changes helped in other ways.<\/p>\n

dotnet\/runtime#99053<\/a> \u201cmemoizes\u201d (caches) various properties on CngKey that are accessed multiple times but where the answer stays the same from call to call; it does so simply by adding a few fields to the type to cache these values, which is a significant win if any is accessed multiple times over the lifetime of the object. The affected properties (Algorithm, AlgorithmGroup, and Provider) are particularly expensive because the OS implementation of these functions needs to make a remote procedure call to another Windows process to access the relevant data.<\/p>\n

\/\/ Windows-only test.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Security.Cryptography;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private RSACng _rsa = new RSACng(2048);<\/p>\n

[GlobalCleanup]
\n public void Cleanup() => _rsa.Dispose();<\/p>\n

[Benchmark]
\n public CngAlgorithm GetAlgorithm() => _rsa.Key.Algorithm;<\/p>\n

[Benchmark]
\n public CngAlgorithmGroup? GetAlgorithmGroup() => _rsa.Key.AlgorithmGroup;<\/p>\n

[Benchmark]
\n public CngProvider? GetProvider() => _rsa.Key.Provider;
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

GetAlgorithm
\n.NET 8.0
\n63,619.352 ns
\n1.000
\n88 B
\n1.00<\/p>\n

GetAlgorithm
\n.NET 9.0
\n10.216 ns
\n0.000
\n\u2013
\n0.00<\/p>\n

GetAlgorithmGroup
\n.NET 8.0
\n62,580.363 ns
\n1.000
\n88 B
\n1.00<\/p>\n

GetAlgorithmGroup
\n.NET 9.0
\n8.354 ns
\n0.000
\n\u2013
\n0.00<\/p>\n

GetProvider
\n.NET 8.0
\n62,108.489 ns
\n1.000
\n232 B
\n1.00<\/p>\n

GetProvider
\n.NET 9.0
\n8.393 ns
\n0.000
\n\u2013
\n0.00<\/p>\n

There were also several improvements related to loading certificates and keys. dotnet\/runtime#97267<\/a> from @birojnayak<\/a> addressed an issue on Linux where the same certificate was being processed multiple times rather than just once, and dotnet\/runtime#97827<\/a> improved the performance of RSA key loading by avoiding some unnecessary work that the key validation was performing.<\/p>\n

Networking<\/h2>\n

Quick, when was the last time you worked on a real application or service that didn\u2019t involve networking at all? I\u2019ll wait\u2026 (I\u2019m so funny.) Effectively every modern application relies on networking in one way, shape, or form, especially one that\u2019s following more cloud-native architectures, involving microservices, and the like. Driving down the costs associated with networking is something we take very seriously, and the .NET community whittles away at these costs every release, including in .NET 9.<\/p>\n

SslStream has been a key focus for performance optimization in past releases. It\u2019s used by a significant portion of traffic with both HttpClient and the ASP.NET Kestrel web server, putting it on the hot path for many systems. Previous improvements have targeted both steady-state throughput as well as creation overhead.<\/p>\n

In .NET 9, a few PRs focused on steady-state throughput, such as dotnet\/runtime#95595<\/a>, which addressed an issue where some packets were being unnecessarily split into two, leading to extra overhead associated with needing to send and receive that extra packet. This was particularly impactful when writing out exactly 16K, and especially on Windows (where I\u2019ve run this test):<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Net;
\nusing System.Net.Security;
\nusing System.Net.Sockets;
\nusing System.Runtime.InteropServices;
\nusing System.Security.Authentication;
\nusing System.Security.Cryptography.X509Certificates;
\nusing System.Security.Cryptography;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private SslStream _client, _server;
\n private byte[] _buffer = new byte[16 * 1024];
\n private readonly SslServerAuthenticationOptions _serverOptions = new SslServerAuthenticationOptions
\n {
\n ServerCertificateContext = SslStreamCertificateContext.Create(GetCertificate(), null),
\n EnabledSslProtocols = SslProtocols.Tls13,
\n };
\n private readonly SslClientAuthenticationOptions _clientOptions = new SslClientAuthenticationOptions
\n {
\n TargetHost = “localhost”,
\n RemoteCertificateValidationCallback = delegate { return true; },
\n EnabledSslProtocols = SslProtocols.Tls13,
\n };<\/p>\n

[GlobalSetup]
\n public async Task Setup()
\n {
\n using var listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
\n listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
\n listener.Listen(1);<\/p>\n

var client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
\n client.Connect(listener.LocalEndPoint!);<\/p>\n

Socket serverSocket = listener.Accept();
\n serverSocket.NoDelay = true;<\/p>\n

_client = new SslStream(new NetworkStream(client, ownsSocket: true), leaveInnerStreamOpen: true);
\n _server = new SslStream(new NetworkStream(serverSocket, ownsSocket: true), leaveInnerStreamOpen: true);<\/p>\n

await Task.WhenAll(
\n _client.AuthenticateAsClientAsync(_clientOptions),
\n _server.AuthenticateAsServerAsync(_serverOptions));
\n }<\/p>\n

[GlobalCleanup]
\n public void Cleanup()
\n {
\n _client.Dispose();
\n _server.Dispose();
\n }<\/p>\n

[Benchmark]
\n public async Task SendReceive()
\n {
\n await _client.WriteAsync(_buffer);
\n await _server.ReadExactlyAsync(_buffer);
\n }<\/p>\n

private static X509Certificate2 GetCertificate()
\n {
\n X509Certificate2 cert;
\n using (RSA rsa = RSA.Create())
\n {
\n var certReq = new CertificateRequest(“CN=localhost”, rsa, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
\n certReq.CertificateExtensions.Add(new X509BasicConstraintsExtension(false, false, 0, false));
\n certReq.CertificateExtensions.Add(new X509EnhancedKeyUsageExtension(new OidCollection { new Oid(“1.3.6.1.5.5.7.3.1”) }, false));
\n certReq.CertificateExtensions.Add(new X509KeyUsageExtension(X509KeyUsageFlags.DigitalSignature, false));
\n cert = certReq.CreateSelfSigned(DateTimeOffset.UtcNow.AddMonths(-1), DateTimeOffset.UtcNow.AddMonths(1));
\n if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
\n {
\n cert = new X509Certificate2(cert.Export(X509ContentType.Pfx));
\n }
\n }
\n return cert;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

SendReceive
\n.NET 8.0
\n43.07 us
\n1.00<\/p>\n

SendReceive
\n.NET 9.0
\n29.38 us
\n0.68<\/p>\n

dotnet\/runtime#100513<\/a> also reduced the cost of checking SslStream.IsMutuallyAuthenticated or SslStream.LocalCertificate from a client when a client certificate is being used.<\/p>\n

However, the bigger impacts in .NET 9 weren\u2019t on steady-state throughput but rather on TLS connection establishment, aka the handshake. Establishing a TLS connection requires the client and server to engage in a conversation where they agree on details like TLS version, what cipher suite to use, confirm the other is who they say they are, create and exchange dedicated symmetric keys for the communication, and so on. That\u2019s a relatively expensive endeavor. For long-lived connections, that overhead is generally not a big deal, but there are plenty of scenarios where connections are more routinely established and torn down, and for those, we want to drive down the overhead associated with setting up such a connection.<\/p>\n

dotnet\/runtime#87874<\/a> focused on reducing allocations associated with the TLS handshake, by renting some buffers from ArrayPool<byte> rather than always allocating. And dotnet\/runtime#97348<\/a> continued the effort by avoiding some unnecessary SafeHandle allocation. dotnet\/runtime#103814<\/a> also switched a rarely-needed ConcurrentDictionary<> in the Linux implementation to be lazily allocated rather than always being allocated as part of the handshake. These changes combine to significantly reduce the allocation incurred as part of setting up TLS:<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Net;
\nusing System.Net.Security;
\nusing System.Net.Sockets;
\nusing System.Runtime.InteropServices;
\nusing System.Security.Authentication;
\nusing System.Security.Cryptography.X509Certificates;
\nusing System.Security.Cryptography;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private NetworkStream _client, _server;
\n private readonly SslServerAuthenticationOptions _serverOptions = new SslServerAuthenticationOptions
\n {
\n ServerCertificateContext = SslStreamCertificateContext.Create(GetCertificate(), null),
\n EnabledSslProtocols = SslProtocols.Tls13,
\n };
\n private readonly SslClientAuthenticationOptions _clientOptions = new SslClientAuthenticationOptions
\n {
\n TargetHost = “localhost”,
\n RemoteCertificateValidationCallback = delegate { return true; },
\n EnabledSslProtocols = SslProtocols.Tls13,
\n };<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n using var listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
\n listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
\n listener.Listen(1);<\/p>\n

var client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
\n client.Connect(listener.LocalEndPoint!);<\/p>\n

Socket serverSocket = listener.Accept();
\n serverSocket.NoDelay = true;
\n _server = new NetworkStream(serverSocket, ownsSocket: true);
\n _client = new NetworkStream(client, ownsSocket: true);
\n }<\/p>\n

[GlobalCleanup]
\n public void Cleanup()
\n {
\n _client.Dispose();
\n _server.Dispose();
\n }<\/p>\n

[Benchmark]
\n public async Task Handshake()
\n {
\n using var client = new SslStream(_client, leaveInnerStreamOpen: true);
\n using var server = new SslStream(_server, leaveInnerStreamOpen: true);<\/p>\n

await Task.WhenAll(
\n client.AuthenticateAsClientAsync(_clientOptions),
\n server.AuthenticateAsServerAsync(_serverOptions));
\n }<\/p>\n

private static X509Certificate2 GetCertificate()
\n {
\n X509Certificate2 cert;
\n using (RSA rsa = RSA.Create())
\n {
\n var certReq = new CertificateRequest(“CN=localhost”, rsa, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
\n certReq.CertificateExtensions.Add(new X509BasicConstraintsExtension(false, false, 0, false));
\n certReq.CertificateExtensions.Add(new X509EnhancedKeyUsageExtension(new OidCollection { new Oid(“1.3.6.1.5.5.7.3.1”) }, false));
\n certReq.CertificateExtensions.Add(new X509KeyUsageExtension(X509KeyUsageFlags.DigitalSignature, false));
\n cert = certReq.CreateSelfSigned(DateTimeOffset.UtcNow.AddMonths(-1), DateTimeOffset.UtcNow.AddMonths(1));
\n if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
\n {
\n cert = new X509Certificate2(cert.Export(X509ContentType.Pfx));
\n }
\n }
\n return cert;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Handshake
\n.NET 8.0
\n2.652 ms
\n1.00
\n5.03 KB
\n1.00<\/p>\n

Handshake
\n.NET 9.0
\n2.581 ms
\n0.97
\n3.3 KB
\n0.66<\/p>\n

Of course, while driving down the costs of doing something is good, avoiding that thing altogether is even better. \u201cTLS resumption\u201d is a capability in the TLS protocol where, if a connection is closed and the same client later opens a new connection to the same server, the client may be able to effectively pick up where it left off with the previous TLS connection rather than starting a brand new one from scratch. Support for TLS resumption on Linux was added in .NET 7, but clients using client certificates weren\u2019t supported\u2026 now in .NET 9 thanks to dotnet\/runtime#102656<\/a>, even such clients can benefit from this significant time saver.<\/p>\n

TLS resumption is an optimization where information is stored to enable more efficient operation later. In some ways, it\u2019s not unlike pooling in that regard. We frequently talk about pooling as a way to optimize. Often our conversations are around avoiding allocations, where employing a pool is betting that you can be more efficient than the garbage collector. For small, cheap to create objects, that\u2019s often a bad bet. For larger objects, such as for larger arrays, it can be a good bet, which is why ArrayPool<T> exists and is used throughout the core libraries for getting temporary buffers (see this Deep .NET discussion on ArrayPool<\/a> for more info). But there\u2019s a much more impactful class where pooling is useful, and that\u2019s where the thing being pooled is really expensive to create. Such cases are no longer about memory management, they\u2019re about ammortizing the cost of that creation. And out of all of the pooling done throughout the core libraries, it\u2019s hard to imagine a more impactful case of that then the HTTP connection pool. The objects in this pool represent established connections to an HTTP server, and establishing such connections can be measured in microseconds, or even seconds in certain environments. If such costs had to be paid every single time you were making an HTTP request, it would add huge latency throughout the system. Instead, outgoing HTTP requests try to grab a connection from the connection pool, reusing that connection for the individual request\/response, and then putting the connection back into the pool when done.<\/p>\n

However, as with any pool, the pool itself has cost. In the case of the HTTP connection pool in a SocketsHttpHandler instance, the most important factor impacting performance is how quickly a connection can be rented and returned to the pool, especially when under load. That load aspect is important, because as a shared resource, access to this pool must be synchronized, in order to ensure the correctness of the system: it\u2019d be really bad, for example, if two requests went to rent a connection at the same time and ended up incorrectly being given the same connection to use, concurrently. \u201cReally bad\u201d in such a case could not only be corrupting data, but possibly even sending the wrong data to the wrong server. That obviously needs to be avoided. So, synchronization is used, but that synchronization creates a bottleneck, where under load lots of requests can end up being blocked just waiting to check whether a connection is even available. Over the years we\u2019ve whittled away at that cost, but it gets even lower in .NET 9, in particular for HTTP\/1.1 connections (we talk about \u201cthe\u201d pool, but in reality connections are only pooled together when they\u2019re interchangeable, so there are actually many groupings of connections, for example with HTTP\/1.1 connections separate from HTTP\/2 connections or HTTP\/3 connections, a separate pool for each endpoint, etc.). dotnet\/runtime#99364<\/a> changes the synchronization mechanism from using a pure lock-based scheme to a more opportunistic concurrency scheme that employs a first-layer of lockless synchronization. There\u2019s now still a lock, but for the hot path it\u2019s avoided as long as there are connections in the pool by using a ConcurrentStack<T>, such that renting is a TryPop and returning is a Push. ConcurrentStack<T> itself uses a lock-free algorithm, that\u2019s a lot more scalable than a lock. There is an interesting downside to ConcurrentStack<T>, which is that the algorithm it employs necessarily involves an allocation per pushed element, and for reasons related to the \u201cABA\u201d problem<\/a>, those allocations can\u2019t be pooled. That means that every time a connection is returned to the pool now, there\u2019s a small allocation. However, for an HTTP request\/response, even though we\u2019ve significantly reduced it over the years, there\u2019s still a non-trivial amount of allocation that occurs over the lifetime of the operation, so one more tiny one doesn\u2019t break the bank, and it\u2019s worth it for the reduced synchronization overheads. We\u2019ve experimented with other data structures, such as ConcurrentQueue<T> (which is able to avoid allocation per Enqueue at steady state), but they\u2019ve had other downsides. I expect we\u2019ll continue to push on this in the future, but for now, what\u2019s there now is a nice improvement.<\/p>\n

Of course once you\u2019ve got the connection, there\u2019s all of the costs associated with actually making the request and handling the response, and those have been whittled away at as well.<\/p>\n

Using vectorized helpers.<\/strong> dotnet\/runtime#93511<\/a> replaces a scalar loop for writing out bytes from an HTTP\/1.1 request header, instead using Ascii.FromUtf16, which is vectorized. In fact, that vectorization was further improved by dotnet\/runtime#102735<\/a>, which improved the 256-bit and 512-bit code paths by using better instructions possible due to not having to care about edge cases already weeded out.
\nAvoiding extra async state machines.<\/strong> dotnet\/runtime#100205<\/a> avoids an extra layer of async state machine that was incurred by most of the HTTP\/1.1 response streams; a method was async only to accommodate some rare logging being enabled, so the async wrapper is now only employed when that logging is enabled.
\nMore caching of very common data.<\/strong> dotnet\/runtime#100177<\/a> removes some more allocation and reduces some overheads by computing and caching some bytes that need to be written out on every request.
\nSpecial-casing main use cases.<\/strong> dotnet\/runtime#102859<\/a> and dotnet\/runtime#103008<\/a> made JsonContent and StringContent cheaper by special-casing the vastly most common media types used. dotnet\/runtime#93759<\/a> further improved JsonContent by reducing the number of async frames on the hot path. And dotnet\/runtime#102845<\/a> from @pedrobsaila<\/a> made TryAddWithoutValidation cheaper for multiple values by special-casing the most common case of the input being an IList<string>, which enables presizing arrays while also avoiding an enumerator allocation.
\nMore ArrayPool.<\/strong> dotnet\/runtime#103764<\/a> avoided a possibly large char[] allocation in the parsing of Alt-Svc headers by using ArrayPool rather than direct allocation.<\/p>\n

While this simple benchmark doesn\u2019t touch on all of these changes, it does highlight that the end-to-end performance of HTTP requests gets cheaper:<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Net.Sockets;
\nusing System.Net;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly Socket s_listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
\n private static readonly HttpMessageInvoker s_client = new(new SocketsHttpHandler());
\n private static Uri? s_uri;<\/p>\n

[Benchmark]
\n public async Task HttpGet()
\n {
\n var m = new HttpRequestMessage(HttpMethod.Get, s_uri) { Content = new StringContent(“Hello, there! How are you today?”) };
\n using (HttpResponseMessage r = await s_client.SendAsync(m, default))
\n using (Stream s = r.Content.ReadAsStream())
\n await s.CopyToAsync(Stream.Null);
\n }<\/p>\n

[GlobalSetup]
\n public void CreateSocketServer()
\n {
\n s_listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
\n s_listener.Listen(int.MaxValue);
\n var ep = (IPEndPoint)s_listener.LocalEndPoint!;
\n s_uri = new Uri($”http:\/\/{ep.Address}:{ep.Port}\/”);<\/p>\n

Task.Run(async () =>
\n {
\n while (true)
\n {
\n Socket s = await s_listener.AcceptAsync();
\n _ = Task.Run(() =>
\n {
\n using (var ns = new NetworkStream(s, true))
\n {
\n byte[] buffer = new byte[1024];
\n int totalRead = 0;
\n while (true)
\n {
\n int read = ns.Read(buffer, totalRead, buffer.Length – totalRead);
\n if (read == 0) return;
\n totalRead += read;
\n if (buffer.AsSpan(0, totalRead).IndexOf(“rnrn”u8) == -1)
\n {
\n if (totalRead == buffer.Length) Array.Resize(ref buffer, buffer.Length * 2);
\n continue;
\n }<\/p>\n

ns.Write(“HTTP\/1.1 200 OKrnDate: Sun, 05 Jul 2020 12:00:00 GMT rnServer: ExamplernContent-Length: 5rnrnHello”u8);<\/p>\n

totalRead = 0;
\n }
\n }
\n });
\n }
\n });
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

HttpGet
\n.NET 8.0
\n92.42 us
\n1.00
\n1.98 KB
\n1.00<\/p>\n

HttpGet
\n.NET 9.0
\n77.13 us
\n0.83
\n1.8 KB
\n0.91<\/p>\n

Related to HTTP, the WebUtility and HttpUtility types both got more efficient this release. dotnet\/runtime#103737<\/a>, in particular, made a variety of changes that have a measurable impact on HtmlEncode and UrlEncode:<\/p>\n

HtmlEncode had a scalar loop looking for characters that need to be encoded. That loop can instead be vectorized by using SearchValues<char>.
\nUrlEncode also had a simlar scalar loop as part of looking for the first non-safe character. SearchValues<char> can also solve this.
\nUrlEncode had a complicated scheme where it would UTF8-encode into a newly-allocated byte[], percent-encode in place in that (thanks to the ability to reinterpret cast with spans), and then use the resulting chars to create a new string. Instead, string.Create can be used, with all of the work done in-place in the buffer generated for that operation.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Net;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n [Arguments(“””
\n How much wood could a woodchuck chuck
\n If a woodchuck could chuck wood?
\n A woodchuck would chuck as much wood
\n As much wood as a woodchuck could chuck,
\n If a woodchuck could chuck wood.
\n “””)]
\n public string HtmlEncode(string input) => WebUtility.HtmlEncode(input);<\/p>\n

[Benchmark]
\n [Arguments(“short_name.txt”)]
\n public string UrlEncode(string input) => WebUtility.UrlEncode(input);
\n}<\/p>\n

Method
\nRuntime
\ninput
\nMean
\nRatio<\/p>\n

HtmlEncode
\n.NET 8.0
\nHow (\u2026)ood. [181]
\n102.607 ns
\n1.00<\/p>\n

HtmlEncode
\n.NET 9.0
\nHow (\u2026)ood. [181]
\n10.188 ns
\n0.10<\/p>\n

UrlEncode
\n.NET 8.0
\nshort_name.txt
\n8.656 ns
\n1.00<\/p>\n

UrlEncode
\n.NET 9.0
\nshort_name.txt
\n2.463 ns
\n0.28<\/p>\n

HttpUtility also received some attention. dotnet\/runtime#102805<\/a> from @TrayanZapryanov<\/a> updated UrlEncodeToBytes, using stack space instead of allocation for smaller inputs, and using SearchValues<byte> to optimize the search for invalid bytes. dotnet\/runtime#102753<\/a> from @TrayanZapryanov<\/a> did the same for UrlDecodeToBytes. dotnet\/runtime#102909<\/a> from @TrayanZapryanov<\/a> similarly reduced allocation in UrlPathEncode, but via the ArrayPool. dotnet\/runtime#102917<\/a> from @TrayanZapryanov<\/a> optimized JavaScriptStringEncode, in particular by using SearchValues. And dotnet\/runtime#102745<\/a> from @TrayanZapryanov<\/a> optimized ParseQueryString by using stackalloc instead of array allocation for smaller input lengths and by replacing string.Substrings with span slicing.<\/p>\n

There were also changes elsewhere in the networking stack that contribute to HTTP use cases. In dotnet\/runtime#98074<\/a>, for example, Uri gained new TryEscapeDataString and TryUnescapeDataString methods that store the output characters into a provided destination span rather than allocating new strings on each call. These methods were then used elsewhere in the networking stack, such as in FormUrlEncodedContent, to improve throughput and reduce allocation.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private KeyValuePair<string, string>[] _data =
\n [
\n new(“key1”, “value1”),
\n new(“key2”, “value2”),
\n new(“key3”, “value3”),
\n new(“key4”, “value4”)
\n ];<\/p>\n

[Benchmark]
\n public FormUrlEncodedContent Create() => new FormUrlEncodedContent(_data);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Create
\n.NET 8.0
\n311.5 ns
\n1.00
\n848 B
\n1.00<\/p>\n

Create
\n.NET 9.0
\n218.7 ns
\n0.70
\n384 B
\n0.45<\/p>\n

Beyond raw HTTP, there were also some new features for WebSocket in .NET 9, namely support for keep-alive pings and timeouts, though not many PRs focused solely on performance (though dotnet\/runtime#101953<\/a> from @PaulusParssinen<\/a> did utilize some newer APIs in ManagedWebSocket in a way that may have removed a bit of fat). There was one notable improvement, however, in dotnet\/runtime#104865<\/a>. The web sockets RFC 6455<\/a> specification requires that when a data frame\u2019s payload data has an opcode denoting it as text, that text must be checked to be valid UTF8-encoded bytes. The validation for that had been a hand-rolled scalar comparison loop. However, now that Utf8.IsValid exists (it was introduced in .NET 8), that accelerated method can be used here instead. It can\u2019t be used in all situations, which is probably why it wasn\u2019t immediately employed when the method was added in the first place. Web sockets payloads may be split across data frames, so it\u2019s possible that the frame being validated is actually the continuation of some previously-received data, and it\u2019s possible that this frame is not the end of the payload, either. But, we know those two pieces of information up-front: if it\u2019s a continuation from a previous frame, we would have already noted it as such, and if it\u2019s not complete, its end-of-message bit won\u2019t have been set. Thus, for the common case where the payload is complete, we can use the accelerated helper for UTF8 validation, and only fall back to the slower path for the corner cases. And this matters because even with the networking costs involved, that UTF8 validation shows up.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Net;
\nusing System.Net.Sockets;
\nusing System.Net.WebSockets;
\nusing System.Text;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private WebSocket _client, _server;
\n private Memory<byte> _buffer = Encoding.UTF8.GetBytes(“””
\n Shall I compare thee to a summer\u2019s day?
\n Thou art more lovely and more temperate:
\n Rough winds do shake the darling buds of May,
\n And summer\u2019s lease hath all too short a date;
\n Sometime too hot the eye of heaven shines,
\n And often is his gold complexion dimm’d;
\n And every fair from fair sometime declines,
\n By chance or nature\u2019s changing course untrimm’d;
\n But thy eternal summer shall not fade,
\n Nor lose possession of that fair thou ow\u2019st;
\n Nor shall death brag thou wander\u2019st in his shade,
\n When in eternal lines to time thou grow\u2019st:
\n So long as men can breathe or eyes can see,
\n So long lives this, and this gives life to thee.
\n “””);
\n private Memory<byte> _tmp = new byte[1024];<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n using var listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
\n listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
\n listener.Listen();<\/p>\n

var client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
\n client.Connect(listener.LocalEndPoint!);
\n Socket server = listener.Accept();<\/p>\n

_client = WebSocket.CreateFromStream(new NetworkStream(client, ownsSocket: true), new WebSocketCreationOptions { IsServer = false, });
\n _server = WebSocket.CreateFromStream(new NetworkStream(server, ownsSocket: true), new WebSocketCreationOptions { IsServer = true });
\n }<\/p>\n

[GlobalCleanup]
\n public void Cleanup()
\n {
\n _client.Dispose();
\n _server.Dispose();
\n }<\/p>\n

[Benchmark]
\n public async Task SendReceive()
\n {
\n await _client.SendAsync(_buffer, WebSocketMessageType.Text, true, default);
\n while (!(await _server.ReceiveAsync(_tmp, default)).EndOfMessage) ;
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

SendReceive
\n.NET 8.0
\n4.093 us
\n1.00<\/p>\n

SendReceive
\n.NET 9.0
\n3.438 us
\n0.84<\/p>\n

There are of course a variety of reasons that performance could have improved, e.g. maybe WebSockets is just exercising a code path that benefits from one of the other optimizations already discussed. How do we know it\u2019s connected to the validation? Let\u2019s profile. And since we already have a benchmark written, we can just use it. There\u2019s another very handy nuget package, Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers, which contains additional \u201cdiagnosers\u201d for BenchmarkDotNet. Diagnosers are one of the main extensibility points within BenchmarkDotNet, enabling developers to perform additional tracking and analyses over benchmarks. You\u2019ve already seen me use some, including the built-in [MemoryDiagnoser(false)] and [DisassemblyDiagnoser]; there are other built-in ones we haven\u2019t used in this post but that are helpful in various situations, like [ThreadingDiagnoser] and [ExceptionDiagnoser], but diagnosers can come from anywhere, and the aforementioned nuget package provides several more. The purpose of those diagnosers is to collect and export performance traces that Visual Studio\u2019s performance tools can then consume. In my case, I want to collect a CPU trace, so as to understand where CPU consumption is going, so I added a [CPUUsageDiagnoser] attribute to my Tests class:<\/p>\n

[CPUUsageDiagnoser]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests<\/p>\n

and then re-ran (on Windows). That\u2019s it. While the test is running, you\u2019ll see the same output as you\u2019re used to seeing, plus a little more. For example, at the end of the benchmarking, I now also see this:<\/p>\n

\/\/ * Diagnostic Output – VSDiagnosticsDiagnoser *
\nCollection result moved to ‘BenchmarkDotNet_Tests_20240804_081400.diagsession’.
\nSession : {a1671047-d6da-4a56-9c71-eadef6c1dd00}
\n Stopped
\nExported diagsession file: d:BenchmarksBenchmarkDotNet_Tests_20240804_081400.diagsession.<\/p>\n

I then simply opened that .diagsession file, just typing its name at the command-line, since that file extension is by default associated with Visual Studio, but you could also File->Open from within Visual Studio itself. That results in a view like the following:<\/p>\n

Notice this single trace covers both the .NET 8 test execution and the .NET 9 test execution, and each is represented by a different entry in the Benchmarks table (but both are on the same execution timeline). I can then double-click one of the tests to narrow the timeline down to just the relevant portion of activity, and then switch over to the CPU Usage tab. When I do, here\u2019s what I see for .NET 8 for the top impacting methods:<\/p>\n

and for .NET 9:<\/p>\n

Notice in the first trace that TryValidateUtf8 is taking up almost 8% of the CPU time, but it doesn\u2019t show up in the second trace at all, instead being replaced by Utf8Utility.GetPointerToFirstInvalidByte, which is the implementation of Utf8.IsValid and which is only half a percent. That ~8% correlates with the ~10% reduction we saw in benchmark execution time. Neat.<\/p>\n

JSON<\/h2>\n

System.Text.Json hit the scene in .NET Core 3.0, and every release since it\u2019s gotten more capable and more efficient. .NET 9 is no exception. In addition to new features like support for exporting JSON schema, deep semantic equality comparison of JsonElements, the ability to respect nullable reference type annotations, support for ordering JsonObject properties, new contract metadata APIs, and more, performance has also been a significant focus.<\/p>\n

One improvement comes from the integration of JsonSerializer with System.IO.Pipelines. Much of the .NET stack moves bytes around via Stream, however ASP.NET internally is implemented with System.IO.Pipelines. There are built-in bidirectional adapters between streams and pipes, but in some cases those adapters add some overhead. As JSON is so critical to modern services, it\u2019s important that JsonSerializer be able to work equally well with both streams and pipes. As such, dotnet\/runtime#101461<\/a> adds new JsonSerializer.SerializeAsync overloads that target PipeWriter in addition to the existing overloads that target Stream. That way, whether you have a Stream or a PipeWriter, JsonSerializer will natively work with either without requiring any indirection to adapt between them. Just use whichever you already have.<\/p>\n

JsonSerializer\u2018s handling of enums was also improved by dotnet\/runtime#105032<\/a>. In addition to adding support for the new [JsonEnumMemberName] attribute, it also moved to an allocation-free parsing solution for enums, utilizing the GetAlternateLookup support added to Dictionary<TKey, TValue> and ConcurrentDictionary<TKey, TValue> to enable a cache of enum information queryable via a ReadOnlySpan<char>.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.Json;
\nusing System.Reflection;
\nusing System.Text.Json.Serialization;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private static readonly JsonSerializerOptions s_options = new()
\n {
\n Converters = { new JsonStringEnumConverter() },
\n DictionaryKeyPolicy = JsonNamingPolicy.KebabCaseLower,
\n };<\/p>\n

[Params(BindingFlags.Default, BindingFlags.NonPublic | BindingFlags.Instance)]
\n public BindingFlags _value;<\/p>\n

private byte[] _jsonValue;
\n private Utf8JsonWriter _writer = new(Stream.Null);<\/p>\n

[GlobalSetup]
\n public void Setup() => _jsonValue = JsonSerializer.SerializeToUtf8Bytes(_value, s_options);<\/p>\n

[Benchmark]
\n public void Serialize()
\n {
\n _writer.Reset();
\n JsonSerializer.Serialize(_writer, _value, s_options);
\n }<\/p>\n

[Benchmark]
\n public BindingFlags Deserialize() =>
\n JsonSerializer.Deserialize<BindingFlags>(_jsonValue, s_options);
\n}<\/p>\n

Method
\nRuntime
\n_value
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

Serialize
\n.NET 8.0
\nDefault
\n38.67 ns
\n1.00
\n24 B
\n1.00<\/p>\n

Serialize
\n.NET 9.0
\nDefault
\n27.23 ns
\n0.70
\n\u2013
\n0.00<\/p>\n

Deserialize
\n.NET 8.0
\nDefault
\n73.86 ns
\n1.00
\n\u2013
\nNA<\/p>\n

Deserialize
\n.NET 9.0
\nDefault
\n70.48 ns
\n0.95
\n\u2013
\nNA<\/p>\n

Serialize
\n.NET 8.0
\nInstance, NonPublic
\n37.60 ns
\n1.00
\n24 B
\n1.00<\/p>\n

Serialize
\n.NET 9.0
\nInstance, NonPublic
\n26.82 ns
\n0.71
\n\u2013
\n0.00<\/p>\n

Deserialize
\n.NET 8.0
\nInstance, NonPublic
\n97.54 ns
\n1.00
\n\u2013
\nNA<\/p>\n

Deserialize
\n.NET 9.0
\nInstance, NonPublic
\n70.72 ns
\n0.73
\n\u2013
\nNA<\/p>\n

JsonSerializer relies on lots of other functionality from System.Text.Json, which has also improved. Here\u2019s a sampling:<\/p>\n

Direct use of UTF8.<\/strong> JsonProperty.WriteTo would always use writer.WritePropertyName(Name) to output the property name. However, that Name property might end up allocating a new string if the JsonProperty wasn\u2019t already caching one. dotnet\/runtime#90074<\/a> from @karakasa<\/a> tweaked the implementation to write out the string if it already had one, or else to directly write out a name based on the UTF8 bytes it would have used to create that string.
\nAvoiding unnecessary intermediate state.<\/strong> dotnet\/runtime#97687<\/a> from @habbes<\/a> is one of those lovely PRs that\u2019s a pure win. The primary change here is to a Base64EncodeAndWrite method that\u2019s Base64-encoding a source ReadOnlySpan<byte> to a destination Span<byte>. The implementation was either stackalloc\u2018ing a buffer or renting a buffer, then encoding into that temporary, and then copying the data into a buffer that is guaranteed to be large enough. Why wasn\u2019t it just encoding directly into that destination buffer rather than going through a temporary? Unclear. But thanks to this PR, that intermediate overhead was simply removed. Similarly, dotnet\/runtime#92284<\/a> removed some unnecessary intermediate state from JsonNode.GetPath. JsonNode.GetPath was doing a lot of allocation, creating a List<string> of all of the path segments which were then combined in reverse order into a StringBuilder. This changes the implementation to extract the path segments in the reverse order in the first place, then building up the resulting path in stack space or an array rented from the ArrayPool.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.Json.Nodes;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private JsonNode _json = JsonNode.Parse(“””
\n {
\n “staff”: {
\n “Elsa”: {
\n “age”: 21,
\n “position”: “queen”
\n }
\n }
\n }
\n “””)[“staff”][“Elsa”][“position”];<\/p>\n

[Benchmark]
\n public string GetPath() => _json.GetPath();
\n}<\/p>\n

Method
\nRuntime
\n_value
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

GetPath
\n.NET 8.0
\nDefault
\n176.68 ns
\n1.00
\n472 B
\n1.00<\/p>\n

GetPath
\n.NET 9.0
\nDefault
\n27.23 ns
\n0.30
\n64 B
\n0.14<\/p>\n

Using existing caches.<\/strong> JsonNode.ToString and JsonNode.ToJsonString were allocating a new PooledByteBufferWriter and Utf8JsonWriter, but the internal Utf8JsonWriterCache type already provides support for using cached versions of these same objects. dotnet\/runtime#92358<\/a> just updated these JsonNode methods to utilize the existing cache.
\nPre-sizing collections.<\/strong> JsonObject has a constructor that accepts an enumerable of properties to add to the object. For a lot of properties, as it\u2019s adding properties, the backing store may need to keep growing and growing, incurring the overhead of allocation and copies. dotnet\/runtime#96486<\/a> from @olo-ntaylor<\/a> tests to see whether a count can be retrieved from the enumerable, and if it can, it uses that count to pre-size the dictionary.
\nAllow fast paths to be fast.<\/strong> JsonValue has a niche feature that enables it to wrap an arbitrary .NET object. As JsonValue derives from JsonNode, JsonNode needs to take that capability into account. The current way it does so makes some common operations much more expensive than they\u2019d need to be. dotnet\/runtime#103733<\/a> refactors the implementation to optimize for the common cases.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.Json;
\nusing System.Text.Json.Nodes;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private JsonNode[] _nodes = [42, “I am a string”, false, DateTimeOffset.Now];<\/p>\n

[Benchmark]
\n [Arguments(JsonValueKind.String)]
\n public int Count(JsonValueKind kind)
\n {
\n var count = 0;
\n foreach (var node in _nodes)
\n {
\n if (node.GetValueKind() == kind)
\n {
\n count++;
\n }
\n }<\/p>\n

return count;
\n }
\n}<\/p>\n

Method
\nRuntime
\nkind
\nMean
\nRatio<\/p>\n

Count
\n.NET 8.0
\nString
\n729.26 ns
\n1.00<\/p>\n

Count
\n.NET 9.0
\nString
\n12.14 ns
\n0.02<\/p>\n

Deduplicating accesses.<\/strong> JsonValue.CreateFromElement accesses JsonElement.ValueKind to determine how to process the data, e.g.
\nif (element.ValueKind is JsonValueKind.Null) { … }
\nelse if (element.ValueKind is JsonValueKind.Object or JsonValueKind.Array) { … }
\nelse { … }<\/p>\n

If ValueKind were a simple field access, that\u2019d be fine. But it\u2019s a bit more complicated than that, involving a large switch to determine what kind to return. Rather than possibly reading from it twice, dotnet\/runtime#104108<\/a> from @andrewjsaid<\/a> just makes a small tweak to only access the property once. No point in doing that work twice.<\/p>\n

Spans over existing data.<\/strong> The JsonElement.GetRawText method is useful for extracting the original input backing the JsonElement, but the data is stored as UTF8 bytes and GetRawText returns a string, so every call allocates and transcodes to produce the result. From dotnet\/runtime#104595<\/a>, the new JsonMarshal.GetRawUtf8Value simply returns a span over the original data, no encoding, no allocation.
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Runtime.InteropServices;
\nusing System.Text.Json;
\nusing System.Text.Json.Nodes;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private JsonElement _json = JsonSerializer.Deserialize(“””
\n {
\n “staff”: {
\n “Elsa”: {
\n “age”: 21,
\n “position”: “queen”
\n }
\n }
\n }
\n “””);<\/p>\n

[Benchmark(Baseline = true)]
\n public string GetRawText() => _json.GetRawText();<\/p>\n

[Benchmark]
\n public ReadOnlySpan<byte> TryGetRawText() => JsonMarshal.GetRawUtf8Value(_json);
\n}<\/p>\n

Method
\nMean
\nRatio
\nAllocated
\nAlloc Ratio<\/p>\n

GetRawText
\n51.627 ns
\n1.00
\n192 B
\n1.00<\/p>\n

TryGetRawText
\n7.998 ns
\n0.15
\n\u2013
\n0.00<\/p>\n

Note that the new method is on the new JsonMarshal class because it\u2019s an API with safety concerns (in general, APIs on the Unsafe class or in the System.Runtime.InteropServices namespace are considered \u201cunsafe\u201d). The concern here is that the JsonElement might be backed by an array rented from the ArrayPool, if the JsonElement came from a JsonDocument. The ReadOnlySpan<byte> you get back is simply pointing into that array. If after getting the span, the JsonDocument is disposed, it\u2019ll return that array back to the pool, and now the span is referencing an array that someone else might rent. If they do and write into that array, the span will now contain whatever was written there, effectively yielding corrupted data. Try this:<\/p>\n

\/\/ dotnet run -c Release -f net9.0<\/p>\n

using System.Runtime.InteropServices;
\nusing System.Text.Json;
\nusing System.Text;<\/p>\n

ReadOnlySpan<byte> elsaUtf8;
\nusing (JsonDocument elsaJson = JsonDocument.Parse(“””
\n {
\n “staff”: {
\n “Elsa”: {
\n “age”: 21,
\n “position”: “queen”
\n }
\n }
\n }
\n “””))
\n{
\n elsaUtf8 = JsonMarshal.GetRawUtf8Value(elsaJson.RootElement);
\n}<\/p>\n

using (JsonDocument annaJson = JsonDocument.Parse(“””
\n {
\n “staff”: {
\n “Anna”: {
\n “age”: 18,
\n “position”: “princess”
\n }
\n }
\n }
\n “””))
\n{
\n Console.WriteLine(Encoding.UTF8.GetString(elsaUtf8)); \/\/ uh oh!
\n}<\/p>\n

When I run that, it prints out the information about \u201cAnna,\u201d even though I retrieved the raw text from the \u201cElsa\u201d JsonElement. Oops! As with anything in C# or .NET that\u2019s \u201cunsafe,\u201d you just need to make sure you hold it correctly.<\/p>\n

One last improvement I want to call out. The feature itself is not actually about performance, but the workarounds I\u2019ve seen folks employ for the lack of this capability do have a significant performance impact, and so having the feature built-in will be a net performance win. dotnet\/runtime#104328<\/a> adds support to both Utf8JsonReader and JsonSerializer for parsing out multiple top-level JSON objects from an input. Previously if any data was found after a JSON object in the input, that would be considered erroneous and fail to parse, and that means that if a particular data source served up multiple JSON objects one after the other, the data would need to be pre-parsed in order to feed only the relevant portions to System.Text.Json. This is particularly relevant with services that stream data, as some of them use such a format.<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Text.Json;
\nusing System.Text.Json.Nodes;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private MemoryStream _source = new MemoryStream(“””
\n {
\n “name”: “Alice”,
\n “age”: 30,
\n “city”: “New York”
\n }<\/p>\n

{
\n “name”: “Bob”,
\n “age”: 25,
\n “city”: “Los Angeles”
\n }<\/p>\n

{
\n “name”: “Charlie”,
\n “age”: 35,
\n “city”: “Chicago”
\n }
\n “””u8.ToArray());<\/p>\n

[Benchmark]
\n [Arguments(“Dave”)]
\n public async Task<Person?> FindAsync(string name)
\n {
\n _source.Position = 0;<\/p>\n

await foreach (var p in JsonSerializer.DeserializeAsyncEnumerable<Person>(_source, topLevelValues: true))
\n {
\n if (p?.Name == name)
\n {
\n return p;
\n }
\n }<\/p>\n

return null;
\n }<\/p>\n

public class Person
\n {
\n public string Name { get; set; }
\n public int Age { get; set; }
\n public string City { get; set; }
\n }
\n}<\/p>\n

Diagnostics<\/h2>\n

Being able to observe one\u2019s application in production is critical to the operation of modern services. System.Diagnostics.Metrics.Meter is .NET\u2019s recommended type for emitting metrics, and several improvements have gone into making it more efficient in .NET 9.<\/p>\n

Counter and UpDownCounter are often used for hot-path tracking of metrics like number of active or queued requests. In production environments, these instruments are frequently bombarded from multiple threads concurrently, which both means they need to be thread-safe but also that they need to be able to scale well. The thread-safety had been achieved by using a lock around updates (which were simply reading a value, adding to it, and storing it back), but under heavy load that could result in significant contention on the lock. To address this, dotnet\/runtime#91566<\/a> changed the implementation in a few ways. First, rather than using a lock to protect the state:<\/p>\n

lock (this)
\n{
\n _delta += value;
\n}<\/p>\n

it used an interlocked operation to perform the addition atomically. Here _delta is a double, and there\u2019s no Interlocked.Add that works with double values, so instead the standard approach of using a loop around an Interlocked.CompareExchange was employed.<\/p>\n

double currentValue;
\ndo
\n{
\n currentValue = _delta;
\n}
\nwhile (Interlocked.CompareExchange(ref _delta, currentValue + value, currentValue) != currentValue);<\/p>\n

That helps, but while this does reduce the overhead and improve scalability, it still represents a bottleneck under heavy contention. To address that, the change also split the single _delta into an array of values, one per core, and a thread picks one of them to update, typically the value associated with the core on which it\u2019s currently running. That way, contention is significantly reduced as it\u2019s both distributed across N values instead of 1 value, and because threads prefer the value for the core on which they\u2019re on, and because there\u2019s only ever one thread executing on a specific core at a given moment, chances for conflicts are significantly reduced. There is still some contention, both because a thread isn\u2019t guaranteed to use the associated value (e.g. the thread could migrate between the time it checks what core it\u2019s on and does the access) and because we actually cap the size of the array (so that it doesn\u2019t consume too much memory), but it still makes the system much more scalable.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Diagnostics.Metrics;
\nusing System.Diagnostics.Tracing;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private MetricsEventListener _listener = new MetricsEventListener();
\n private Meter _meter = new Meter(“Example”);
\n private Counter<int> _counter;<\/p>\n

[GlobalSetup]
\n public void Setup() => _counter = _meter.CreateCounter<int>(“counter”);<\/p>\n

[GlobalCleanup]
\n public void Cleanup()
\n {
\n _listener.Dispose();
\n _meter.Dispose();
\n }<\/p>\n

[Benchmark]
\n public void Counter_Parallel()
\n {
\n Parallel.For(0, 1_000_000, i =>
\n {
\n _counter.Add(1);
\n _counter.Add(1);
\n });
\n }<\/p>\n

private sealed class MetricsEventListener : EventListener
\n {
\n protected override void OnEventSourceCreated(EventSource eventSource)
\n {
\n if (eventSource.Name == “System.Diagnostics.Metrics”)
\n {
\n EnableEvents(eventSource, EventLevel.LogAlways, EventKeywords.All, new Dictionary<string, string?>() { { “Metrics”, “Example\\upDownCounter;Example\\counter” } });
\n }
\n }
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

Counter_Parallel
\n.NET 8.0
\n137.90 ms
\n1.00<\/p>\n

Counter_Parallel
\n.NET 9.0
\n30.65 ms
\n0.22<\/p>\n

There\u2019s another interesting aspect of the improvement worth mentioning, and that\u2019s the padding employed in the array. Going from a single double _delta to an array of deltas, you might imagine we\u2019d end up with:<\/p>\n

private readonly double[] _deltas;<\/p>\n

but if you look at the code, it\u2019s instead:<\/p>\n

private readonly PaddedDouble[] _deltas;<\/p>\n

where PaddedDouble is defined as:<\/p>\n

[StructLayout(LayoutKind.Explicit, Size = 64)]
\nprivate struct PaddedDouble
\n{
\n [FieldOffset(0)]
\n public double Value;
\n}<\/p>\n

This effectively increases the size of each value from 8 bytes to 64 bytes, where only the first 8 bytes of each value is used and the other 56 bytes are padding. That\u2019s odd, right? Normally we\u2019d jump at an opportunity to shrink 64 bytes down to 8 bytes in order to reduce allocation and memory consumption, but here we\u2019re purposefully going in the other direction.<\/p>\n

The reason for that is \u201cfalse sharing.\u201d Consider this benchmark, which I\u2019ve shamelessly borrowed from a conversation Scott Hanselman and I recently recorded for the Deep .NET series<\/a> but which hasn\u2019t yet posted online:<\/p>\n

\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _values = new int[32];<\/p>\n

[Params(1, 31)]
\n public int Index { get; set; }<\/p>\n

[Benchmark]
\n public void ParallelIncrement()
\n {
\n Parallel.Invoke(
\n () => IncrementLoop(ref _values[0]),
\n () => IncrementLoop(ref _values[Index]));<\/p>\n

static void IncrementLoop(ref int value)
\n {
\n for (int i = 0; i < 100_000_000; i++)
\n {
\n Interlocked.Increment(ref value);
\n }
\n }
\n }
\n}<\/p>\n

When I run that, I get results like this:<\/p>\n

Method
\nIndex
\nMean<\/p>\n

ParallelIncrement
\n1
\n1,779.9 ms<\/p>\n

ParallelIncrement
\n31
\n432.3 ms<\/p>\n

In this benchmark, one thread is incrementing _values[0] and the other thread is incrementing either _values[1] or _values[31]. That index is the only difference, yet the one accessing _values[31] is several times faster than the one accessing _values[1]. That\u2019s because there\u2019s contention here even if it\u2019s not obvious in the code. The contention comes from the fact that the hardware works with memory in groups of bytes called a \u201ccache line.\u201d Most hardware has caches lines of 64 bytes. In order to update a particular memory location, the hardware will acquire the whole cache line. If another core wants to update that same cache line, it\u2019ll need to acquire it. That back and forth results in a lot of overhead. It doesn\u2019t matter if one core is touching the first of those 64 bytes and another thread is touching the last, from the hardware\u2019s perspective there\u2019s still sharing happening. \u201cFalse sharing.\u201d Thus, the Counter fix is using padding around the double values to try to space them out more so as to minimize the sharing that limits scalability.<\/p>\n

As an aside, there are some additional BenchmarkDotNet diagnosers that can help to highlight the effects of false sharing. ETW on Windows enables collecting various CPU performances counters, such as for branch misses or instructions retired, and BenchmarkDotNet has a [HardwareCounters] diagnoser that\u2019s able to collect this ETW data. One such counter is for cache misses, which often reflect false sharing issues. If you\u2019re on Windows, you can try grabbing the separate BenchmarkDotNet.Diagnostics.Windows nuget package and using it as in this benchmark:<\/p>\n

\/\/ This benchmark only works on Windows.
\n\/\/ Add a <PackageReference Include=”BenchmarkDotNet.Diagnostics.Windows” Version=”0.14.0″ \/> to the csproj.
\n\/\/ dotnet run -c Release -f net9.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Diagnosers;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HardwareCounters(HardwareCounter.InstructionRetired, HardwareCounter.CacheMisses)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private int[] _values = new int[32];<\/p>\n

[Params(1, 31)]
\n public int Index { get; set; }<\/p>\n

[Benchmark]
\n public void ParallelIncrement()
\n {
\n Parallel.Invoke(
\n () => IncrementLoop(ref _values[0]),
\n () => IncrementLoop(ref _values[Index]));<\/p>\n

static void IncrementLoop(ref int value)
\n {
\n for (int i = 0; i < 100_000_000; i++)
\n {
\n Interlocked.Increment(ref value);
\n }
\n }
\n }
\n}<\/p>\n

Here I\u2019ve asked for both instructions retired, which reflects how much instructions were fully executed (this in and of itself can be a useful metric when analyzing performance, as it\u2019s not as prone to variation as wall-clock measurements), and cache misses, which reflects how many times data wasn\u2019t available in the CPU\u2019s cache.<\/p>\n

Method
\nIndex
\nMean
\nInstructionRetired\/Op
\nCacheMisses\/Op<\/p>\n

ParallelIncrement
\n1
\n1,846.2 ms
\n804,300,000
\n177,889<\/p>\n

ParallelIncrement
\n31
\n442.5 ms
\n824,333,333
\n52,429<\/p>\n

In the two benchmarks, we can see that the number of instructions executed is almost the same between when false sharing occurred (Index == 1) and didn\u2019t (Index == 31), but the number of cache misses is more than three times larger in the false sharing case, and reasonably well correlated with the time increase. When one core performs a write, that invalidates the corresponding cache line in the other core\u2019s cache, such that the other core then needs to reload the cache line, resulting in cache misses. But I digress\u2026<\/p>\n

Another nice improvement comes in dotnet\/runtime#105011<\/a> from @stevejgordon<\/a>, adding a new constructor to Measurement. Often when creating Measurements, you\u2019re also tagging them with additional key\/value pairs of information, for which the TagList type exists. TagList implements IList<KeyValuePair<string, object?>>, and Measurement has a constructor that takes an IEnumerable<KeyValuePair<string, object?>>, so you can pass a TagList to a Measurement and it \u201cjust works\u201d\u2026 albeit slower than it could. If you had code like:<\/p>\n

measurements.Add(new Measurement<long>(
\n snapshotV4.LastAckCount,
\n new TagList { tcpVersionFourTag, new(NetworkStateKey, “last_ack”) }));<\/p>\n

that would end up boxing the TagList struct as an enumerable, and then enumerating through it via the interface, which also entails an enumerator allocation. The new constructor this PR adds takes a TagList, avoiding those overheads. TagList is also a large struct, as common usage has it living only on the stack and so as an optimization it stores some of the contained key\/value pairs directly in fields on the struct rather than always forcing an array allocation. The net result is much less overhead in constructing these measurements.<\/p>\n

TagList itself was also improved by dotnet\/runtime#104132<\/a>, which re-implemented the type for .NET 8+ on top of [InlineArray]. TagList is effectively a list of key\/value pairs, but in order to avoid always allocating a backing store, it stores some of those pairs inline in itself. This previously was done with dedicated fields for each pair, and then code that directly accessed each field. Now, an [InlineArray] is used, cleaning up the code and enabling access via spans.<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Diagnostics;
\nusing System.Diagnostics.Metrics;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Counter<long> _counter;
\n private Meter _meter;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n this._meter = new Meter(“TestMeter”);
\n this._counter = this._meter.CreateCounter<long>(“counter”);
\n }<\/p>\n

[GlobalCleanup]
\n public void Cleanup() => this._meter.Dispose();<\/p>\n

[Benchmark]
\n public void CounterAdd()
\n {
\n this._counter?.Add(100, new TagList
\n {
\n { “Name1”, “Val1” },
\n { “Name2”, “Val2” },
\n { “Name3”, “Val3” },
\n { “Name4”, “Val4” },
\n { “Name5”, “Val5” },
\n { “Name6”, “Val6” },
\n { “Name7”, “Val7” },
\n });
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CounterAdd
\n.NET 8.0
\n31.88 ns
\n1.00<\/p>\n

CounterAdd
\n.NET 9.0
\n13.93 ns
\n0.44<\/p>\n

Peanut Butter<\/h2>\n

Throughout this post, I\u2019ve tried to group improvements by topic area in order to create a more fluid and interesting discussion. However, over the course of a year, with as vibrant a community as exists for .NET, and with the breadth of functionality that exists across the platform, there are invariably a large number of one-off PRs that improve this or that by a little. It\u2019s often challenging to imagine any one of these significantly \u201cmoving the needle,\u201d but altogether, such changes reduce the \u201cpeanut butter\u201d of performance overhead spread thinly across the libraries. In no particular order, here\u2019s a non-comprehensive look at some of these:<\/p>\n

StreamWriter.Null.<\/strong> StreamWriter exposes a static Null field. It stores a StreamWriter instance that\u2019s intended to be a \u201cbit bucket,\u201d a sink you can write to that just ignores all of the data, ala \/dev\/null on Unix, Stream.Null, and so on. Unfortunately, the way it was implemented had two problems, one of which I\u2019m incredibly surprised took us this long to discover (as it\u2019s been this way for as long as .NET has existed). It was implemented as new StreamWriter(Stream.Null, …). All of the state tracking done in StreamWriter is not thread-safe, yet here this instance is exposed from a public static member, which means it should be thread-safe, and if multiple threads hammered that StreamWriter instance, it could result in really strange exceptions occurring, like arithmetic overflow. Performance-wise, it\u2019s also problematic, because even though actual writes to the underlying Stream are ignored, all of the work actually done by StreamWriter is still done, even though it\u2019s useless. dotnet\/runtime#98473<\/a> fixes both of those problems by creating an internal NullStreamWriter : StreamWriter type that overrides everything to be nops, and then Null is initialized to an instance of that.
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n [Benchmark]
\n public void WriteLine() => StreamWriter.Null.WriteLine(“Hello, world!”);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

CounterAdd
\n.NET 8.0
\n7.5164 ns
\n1.00<\/p>\n

CounterAdd
\n.NET 9.0
\n0.0283 ns
\n0.004<\/p>\n

NonCryptographicHashAlgorithm.Append{Async}<\/strong> NonCryptographicHashAlgorithm is the base class in System.IO.Hashing for types like XxHash3 and Crc32. One nice feature it provides is the ability to append an entire Stream\u2018s contents to it in a single call, e.g.<\/p>\n

XxHash3 hash = new();
\nhash.Append(someStream);<\/p>\n

The implementation of Append was fairly straightforward: rent a buffer from the ArrayPool and then in a loop repeatedly Stream.Read (or Stream.ReadAsync in the case of AppendAsync) into that buffer and Append that filled portion of the buffer. This has a couple of performance downsides, however. First, the buffer being rented was 4096 bytes. That\u2019s not tiny, but using a larger buffer can reduce the number of calls to the source stream being appended, which in turn can reduce any I\/O performed by that Stream. Second, many streams have optimized implementations for pushing all of their contents to a sink like this: CopyTo. MemoryStream.CopyTo, for example, will just perform a single write of its entire internal buffer to the Stream passed to its CopyTo. But even if a Stream doesn\u2019t override CopyTo, the base CopyTo implementation already provides such a copying loop, and it does so by default using a much larger rented buffer. As such, dotnet\/runtime#103669<\/a> changes the implementation of Append to allocate a small temporary Stream object that wraps this NonCryptographicHashAlgorithm instance, and any calls to Write are just translated to be calls to Append. This is a neat example where sometimes we actually choose to pay for a small, short-lived allocation in exchange for significant throughput benefits.<\/p>\n

\/\/ dotnet run -c Release -f net8.0 –filter “*”<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Configs;
\nusing BenchmarkDotNet.Environments;
\nusing BenchmarkDotNet.Jobs;
\nusing BenchmarkDotNet.Running;
\nusing System.IO.Hashing;<\/p>\n

var config = DefaultConfig.Instance
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core80).WithNuGet(“System.IO.Hashing”, “8.0.0”).AsBaseline())
\n .AddJob(Job.Default.WithRuntime(CoreRuntime.Core90).WithNuGet(“System.IO.Hashing”, “9.0.0-rc.1.24431.7”));
\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args, config);<\/p>\n

[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”, “NuGetReferences”)]
\npublic class Tests
\n{
\n private Stream _stream;
\n private byte[] _bytes;<\/p>\n

[GlobalSetup]
\n public void Setup()
\n {
\n _bytes = new byte[1024 * 1024];
\n new Random(42).NextBytes(_bytes);<\/p>\n

string path = Path.GetRandomFileName();
\n File.WriteAllBytes(path, _bytes);
\n _stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 0, FileOptions.DeleteOnClose);
\n }<\/p>\n

[GlobalCleanup]
\n public void Cleanup() => _stream.Dispose();<\/p>\n

[Benchmark]
\n public ulong Hash()
\n {
\n _stream.Position = 0;
\n var hash = new XxHash3();
\n hash.Append(_stream);
\n return hash.GetCurrentHashAsUInt64();
\n }
\n}<\/p>\n

Method
\nRuntime
\nMean<\/p>\n

Hash
\n.NET 8.0
\n91.60 us<\/p>\n

Hash
\n.NET 9.0
\n61.26 us<\/p>\n

Unnecessary virtual.<\/strong> virtual methods have overhead. First, they\u2019re more expensive to invoke than non-virtual methods because it requires several indirections to find the actual target method to invoke (the actual target may differ based on the concrete type being used). And second, without a technology like dynamic PGO, virtual methods won\u2019t be inlined, because the compiler can\u2019t statically see which target should be inlined (and even if dynamic PGO makes such inlining possible for the most common type, there\u2019s still a check required to ensure it\u2019s ok to follow that path). As such, if things don\u2019t need<\/em> to be virtual, it\u2019s better performance-wise for them to not be. And if such things are internal, unless they\u2019re actively being overridden by something, there\u2019s no reason to keep them virtual. dotnet\/runtime#104453<\/a> from @xtqqczze<\/a>, dotnet\/runtime#104456<\/a> from @xtqqczze<\/a>, and dotnet\/runtime#104483<\/a> from @xtqqczze<\/a> all address exactly such cases, removing virtual from a smattering of internal members that weren\u2019t being overridden. It might only save a few instructions here and there, but there\u2019s effectively no downside to such a change (other than some minimal code churn), a pure win.
\nReadOnlySpan vs Span.<\/strong> We as developers like to protect ourselves from ourselves, for example making fields readonly to avoid accidentally changing them. Such changes can also have performance benefits, for example the JIT can better optimize static fields that are readonly than those that aren\u2019t. The same set of principles applies to Span<T> and ReadOnlySpan<T>. If a method doesn\u2019t need to mutate the contents of a collection being passed in, it\u2019s less accident prone to use a ReadOnlySpan<T> rather than a Span<T>. It also signals to the caller that they don\u2019t need to be concerned about the data changing out from under them. Interestingly, here, too, there\u2019s both a correctness and a performance benefit to using ReadOnlySpan<T> instead of Span<T>. The implementations of these two types is almost word-for-word identical, the critical difference being whether the indexer returns a ref T or a ref readonly T. There is one additional line in Span<T>, however, that doesn\u2019t exist in ReadOnlySpan<T>. Span<T>\u2018s constructor has this one extra check:
\nif (!typeof(T).IsValueType && array.GetType() != typeof(T[]))
\n ThrowHelper.ThrowArrayTypeMismatchException();<\/p>\n

This check exists because of array covariance. Let\u2019s say you have this:<\/p>\n

Base[] array = new Derived[3];<\/p>\n

class Base { }
\nclass Derived : Base { }<\/p>\n

That code compiles and runs successfully, because .NET supports array covariance, meaning an array of a derived type can be used as an array as the base type. But there\u2019s an important catch here. Let\u2019s augment the example slightly:<\/p>\n

Base[] array = new Derived[3];
\narray[0] = new AlsoDerived(); \/\/ uh oh!<\/p>\n

class Base { }
\nclass Derived : Base { }
\nclass AlsoDerived : Base { }<\/p>\n

This will compile successfully, but at run-time it\u2019ll fail with an ArrayTypeMismatchException. That\u2019s because it\u2019s trying to store an AlsoDerived instance into a Derived[], and there\u2019s no relationship between the two types that should permit that. The check required to enforce that comes at a cost, every single time you try to write into an array (except in cases where the compiler can prove it\u2019s safe and elide the costs). When Span<T> was introduced, the decision was made to hoist that check up to the span\u2019s constructor; that way, once you get a valid span, no such checking needs to be performed on every write, only once on construction. That\u2019s what that additional line of code is doing, checking to ensure that the specified T is the same as the provided array\u2019s element type. That means code like this will also throw an ArrayTypeMismatchException:<\/p>\n

Span<Object> span = new string[2]; \/\/ uh oh<\/p>\n

But that also means if you use Span<T> in situations where you could have used ReadOnlySpan<T>, there\u2019s a good chance you\u2019re unnecessarily incurring that check, which means you\u2019re both possibly going to hit unexpected exceptions depending on what arrays are passed in, and you\u2019re incurring a bit of peanut butter cost. dotnet\/runtime#104864<\/a> replaced a bunch of Span<T>s with ReadOnlySpan<T>s to reduce the chances we\u2019d incur such overhead, while also just improving the maintainability of the code.<\/p>\n

readonly and const.<\/strong> In the same vein, changing fields that could be const to be so, changing non-readonly fields that could be readonly to be so, and removing unnecessary property setters is all goodness for maintainability while also having the chance of improving performance. Making fields const avoids unnecessary memory accesses while also allowing the JIT to better employ constant propagation. And making static fields readonly enables the JIT to treat them as if they were const in tier 1 compilation. dotnet\/runtime#100728<\/a> updates hundreds of occurrences.
\nMemoryCache.<\/strong> dotnet\/runtime#103992<\/a> from @ADNewsom09<\/a> addresses an inefficiency in Microsoft.Extensions.Caching.Memory. If multiple concurrent operations end up triggering the cache\u2019s compaction operation, many of the involved threads can all end up duplicating each other\u2019s work. The fix is to simply have only one of the threads do the compaction operation.
\nBinaryReader.<\/strong> dotnet\/runtime#80331<\/a> from @teo-tsirpanis<\/a> made BinaryReader allocations relevant only to reading text be lazily allocated only when such reading occurs. If the reader is never used for reading text, the application won\u2019t need to pay for the allocation.
\nArrayBufferWriter.<\/strong> dotnet\/runtime#88009<\/a> from @AlexRadch<\/a> adds a new ResetWrittenCount method to ArrayBufferWriter. ArrayBufferWriter.Clear already exists, but in addition to setting the written count to 0, it also clears the underlying buffer. In many situations, that clearing is unnecessary overhead, so ResetWrittenCount allows it to be avoided. (There was an interesting debate about whether such a new method is even necessary, and whether Clear could just be changed to remove the zeroing. But concerns about stale data finding their way into consuming code as corrupted data led to the new method being added instead.)
\nSpan-based File methods.<\/strong> The static File class provides simple helpers for interacting with files, e.g. File.WriteAllText. Historically, these methods have worked with strings and arrays. That means, though, that if someone instead has a span, they either can\u2019t use these simple helpers or they need to pay to create a string or an array from the span. dotnet\/runtime#103308<\/a> adds new span-based overloads so that developers don\u2019t need to choose here between simplicity and performance.
\nstring concat vs Append.<\/strong> string concatenation inside of a loop is well-known no-no, as in the extreme it can lead to significant O(N^2) costs. Such a string concatenation was occurring, however, inside of MailAddressCollection, where an encoded version of every address in the collection was being appended onto a string using string concatenation. dotnet\/runtime#95760<\/a> from @YohDeadfall<\/a> changed that to use a builder instead.
\nClosures.<\/strong> The config source generator was introduced in .NET 8 to significantly improve the performance of configuration binding, while also making it friendlier to Native AOT. It achieved both. However, it can be improved further. There\u2019s an unanticipated extra allocation that occurs on success paths that\u2019s only relevant to failure paths, because of how the code is being generated. For a call site like this:
\npublic static void M(IConfiguration configuration, C1 c) => configuration.Bind(c);<\/p>\n

the source generator would emit a method like this:<\/p>\n

public static void BindCore(IConfiguration configuration, ref C1 obj, BinderOptions? binderOptions)
\n{
\n ValidateConfigurationKeys(typeof(C1), s_configKeys_C1, configuration, binderOptions);
\n if (configuration[“Value”] is string value15)
\n obj.Value = ParseInt(value15, () => configuration.GetSection(“Value”).Path);
\n}<\/p>\n

That lambda being passed to the ParseInt helper is accessing configuration, which is defined outside of the lambda as a parameter. To get that data into the lambda, the compiler allocates a \u201cdisplay class\u201d to store the information, with the body of the lambda translated into a method on that display class. That display class gets allocated at the beginning of the scope that contains the data, which in this case means it\u2019s allocated at the beginning of the BindCore method. That means it\u2019s allocated regardless of whether the if block is true, and even if ParseInt is called, the delegate passed to it is only ever invoked when there\u2019s a failure. dotnet\/runtime#100257<\/a> from @pedrobsaila<\/a> reworks the source generator code so that this allocation isn\u2019t incurred.<\/p>\n

Stream.Read\/Write Span Overrides.<\/strong> Streams that don\u2019t override the span-based Read\/Write methods end up utilizing the base implementations, which allocate. There are a ton of Stream implementations in dotnet\/runtime, and we\u2019ve overridden such methods almost everywhere, but now and again we find one that slipped through. dotnet\/runtime#86674<\/a> from @hrrrrustic<\/a> fixed one such case on the StreamOnSqlBytes type.
\nGlobalization Arrays.<\/strong> Every NumberFormatInfo object defaults its NumberGroupSizes, CurrentGroupSizes, and PercentGroupSizes to each be new instances of new int[] { 3 } (even if subsequent initialization overwrites them). And yet these arrays are never handed out to consumers: the properties that expose them make defensive copies. Which means all of these can just refer to the same shared singleton array. The same is true for NativeDigits, which is initialized first to a new array of the numbers 0 through 9. dotnet\/runtime#93117<\/a> addresses all of these by creating and using such singletons.
\nColorTranslator.ToWin32.<\/strong> There\u2019s a .NET design guideline<\/a> that says properties should be like smarter fields. The resulting expectation is that they should be cheap, like just accessing a field or doing a very simple calculation over a field. Unfortunately, we don\u2019t always follow our own guidance, and there exist some properties that really, really look like they should be trivial but are actually sometimes not. System.Drawing.Color is a good example. A very reasonable mental model for Color (which according to the docs \u201cRepresents an ARGB (alpha, red, green, blue) color\u201d) is that it\u2019d just be four byte values, one for each channel, either in their own fields or packed together into an int. Unfortunately, it\u2019s not quite as simple as that. Color can<\/em> be that, such as if it\u2019s constructed using Color.FromArgb(uint), but it can also be used to represent a \u201cknown colors,\u201d as is evident by the SystemColors type having a bunch of static properties (e.g. SystemColors.Control) that return a color for the underlying OS. And even there, you might think \u201coh, ok, well those properties must be what Stephen is referring to, they must call out to the OS to get the color, they probably do so and then use FromArgb.\u201d And again, that\u2019s a very intuitive mental model, and again it\u2019s not what actually happens. Those properties actually are cheap; all they do is construct a Color with an enum value corresponding to the system color. Then where is the actual OS color value retrieved, you ask? As part of the R, G, B, and A properties on Color!. That means if you access each of these properties, as ColorTranslator was doing in a variety of its methods, you\u2019re making three or four times as many P\/Invokes as you\u2019d otherwise need to. dotnet\/runtime#106042<\/a> fixes this for ColorTranslator, but it serves as a good reminder why such guidelines exist. (This benchmark is Windows-specific as SystemColor doesn\u2019t currently rely on OS information for Linux or macOS.)
\n\/\/ Windows-specific (it works on Linux and macOS, but doesn’t demonstrate the same thing.)
\n\/\/ dotnet run -c Release -f net8.0 –filter “*” –runtimes net8.0 net9.0<\/p>\n

using BenchmarkDotNet.Attributes;
\nusing BenchmarkDotNet.Running;
\nusing System.Drawing;<\/p>\n

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);<\/p>\n

[MemoryDiagnoser(false)]
\n[HideColumns(“Job”, “Error”, “StdDev”, “Median”, “RatioSD”)]
\npublic class Tests
\n{
\n private Color _color = SystemColors.Control;<\/p>\n

[Benchmark]
\n public int ColorToWin32() => ColorTranslator.ToWin32(_color);
\n}<\/p>\n

Method
\nRuntime
\nMean
\nRatio<\/p>\n

ColorToWin32
\n.NET 8.0
\n11.263 ns
\n1.00<\/p>\n

ColorToWin32
\n.NET 9.0
\n4.711 ns
\n0.42<\/p>\n

What\u2019s Next?<\/h2>\n

Maybe one more poem? An acrostic this time:<\/p>\n

Driving innovation with unmatched speed,
\nOpening doors to what developers need.
\nTurbocharged perf, breaking the mold,
\nNew benchmarks surpassed, metrics so bold.
\nEmpowering coders, dreams take flight,
\nTransforming visions with .NET might.<\/p>\n

Navigating challenges with precision and flair,
\nInspiring creativity, improvements everywhere.
\nNurturing growth, pushing limits high,
\nElevating success, reaching for the sky.<\/p>\n

Several hundred pages later and still not a poet. Oh well.<\/p>\n

I\u2019m asked from time to time why I invest in writing these \u201cPerformance Improvements in .NET\u201d posts. There\u2019s no one answer. In no particular order:<\/p>\n

Personal learning.<\/strong> I pay close attention throughout the year to all of the various performance improvements happening in the release, sometimes from a distance, sometimes as the one making the changes. Writing this post serves as a forcing-function for me to revisit them all and really internalize the changes that were made and their relevance to the broader picture. It\u2019s a learning opportunity for me.
\nTesting.<\/strong> As one of the developers on the team recently said to me, \u201cI like this time of the year when you give our optimizations a stress-test and uncover inefficiencies.\u201d Every year when I\u2019m going through the improvements, just the act of re-validating the improvements often highlights regressions, cases that were missed, or further opportunities that can be addressed in the future. Again, it\u2019s a forcing function to do more testing and with a fresh set of eyes.
\nThanks.<\/strong> Many of the performance improvements in each release aren\u2019t from the folks working on the .NET team or even at Microsoft. They\u2019re from passionate and talented individuals throughout the global .NET ecosystem, and I like to highlight their contributions. That\u2019s why throughout the post you see me calling out when PRs are from folks not employed by Microsoft as full-time employees. In this post, that accounts for ~20% of all the cited PRs. Amazing. Heartfelt thanks to everyone who\u2019s worked to make .NET better for everyone.
\nExcitement.<\/strong> Developers often have conflicting opinions about the speed at which .NET is advancing, some really appreciating the frequent introduction of new features, others concerned that they can\u2019t keep up with all of the newness. But the one thing everyone seems to agree on is the love of \u201cfree perf,\u201d and that\u2019s a lot of what these posts talk about. .NET gets faster and faster every release, and it\u2019s exciting to see a tour through the highlights collected all in one place.
\nEducation.<\/strong> There are multiple forms of performance improvements covered throughout the post. Some of the improvements you get completely for free just by upgrading the runtime; the implementations in the runtime are better, and so when you run on them, your code just gets better, too. Some of the improvements you get completely for free by upgrading the runtime and<\/em> recompiling; the C# compiler itself generates better code, often taking advantage of newer surface area exposed in the runtime. And other improvements are new features that, in addition to the runtime and compiler utilizing, you can utilize directly and make your code even faster. Educating about those capabilities and why and where you\u2019d want to utilize them is important to me. But beyond the new features, the techniques employed in making all of the rest of the optimizations throughout the runtime are often more broadly applicable. By learning how these optimizations are applied in the runtime, you can extrapolate and apply similar techniques to your own code, making it that much faster.<\/p>\n

If you\u2019ve read this far, I hope you indeed have learned something and are excited about the .NET 9 release. As is likely obvious from my enthusiastic ramblings and awkward poetry, I\u2019m incredibly excited about .NET, everything that\u2019s been achieved in .NET 9, and the future of the platform. If you\u2019re already using .NET 8, upgrading to .NET 9 should be a breeze (the .NET 9 Release Candidate<\/a> is available for download), and I\u2019d love it if you\u2019d do so and share with us any successes you achieve or issues you face along the way. We\u2019d love to learn from you. And if you have ideas about how to further improve the performance of .NET for .NET 10, please join us in dotnet\/runtime<\/a>.<\/p>\n

Happy coding!<\/p>\n

The post Performance Improvements in .NET 9<\/a> appeared first on .NET Blog<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"

Each year, summer arrives to find me daunted and excited to write about the performance improvements in the upcoming release […]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-1228","post","type-post","status-publish","format-standard","hentry","category-dotnet"],"_links":{"self":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/1228","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/comments?post=1228"}],"version-history":[{"count":0,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/1228\/revisions"}],"wp:attachment":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media?parent=1228"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/categories?post=1228"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/tags?post=1228"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}