Finding async anti-patterns<\/h3>\n
Asynchronous (async) programming has been around for several years on the .NET platform but can been difficult to do well. There has been some confusion on the best practices for async<\/a> and how to use it properly which has led to some antipatterns that may not reveal themselves until your service is under high load.<\/p>\n We talked to a bunch of support engineers and the \u201csync-over-async\u201d antipatterns have a set of negative performance characteristics:<\/p>\n Of course, at this point, there are still a wide range of underlying causes that might be at the heart of your problematic service behavior. Our team wanted to make it easier for developers to find these kinds of issues and also identify where you might remediate the problem.<\/p>\n In pursuit of this goal, we have started developing a new .NET Diagnostics Analyzer tool<\/strong><\/a> to help developers, new to dump debugging, quickly identify (or rule out) issues we know impact customers in production environments, let\u2019s look at an example.<\/p>\n Let\u2019s assume the service has been exhibiting the symptoms we noted earlier (timeouts, slow response times, etc.), how do you go about validating (or invalidating) the possible root cause?<\/p>\n First, let\u2019s open the memory dump in Visual Studio by using the\u00a0File ->Open -> File<\/em> menu and select your memory dump. You can also drag and drop the dump into the Visual Studio to open it.<\/p>\n Notice on the Memory Dump Summary page a new Action<\/strong> called Run Diagnostics Analysis<\/strong>.<\/p>\n Selecting this action will start the debugger and open the new Diagnostic Analysis<\/strong> page with a list of available analyzer options, organized by the underlying symptom.<\/p>\n In my example I am concerned with my \u201capp not responding to requests in a timely manner\u201d. <\/em>To investigate these symptoms, I am going to select all the options under Process Responsiveness<\/strong> as this best matches my app\u2019s issue.<\/p>\n Hitting the Analyze<\/strong> button will start the investigation and present results based on the combination of process info and CLR data captured in the memory dump.<\/p>\n In the image below the analyzer has found two error results, and upon selecting the first result (\u201cThread pool is out of threads\u201d) I get to see the Analysis Summary<\/strong> which is proposing that the \u201cCLR thread pool is experiencing starvation\u201d. This information is really important to an investigation, it suggests that the CLR has currently used all available thread pool threads, which in turn means we cannot respond to any new requests until a thread is released.<\/p>\n Selecting the second results \u201cThread pool is out of threads due to blocked async methods\u201d, reveals the heart of the problem more specifically. By looking at the dump, the analyzer was able to find specific locations where we have inadvertently called blocking code from an asynchronous thread context, and this is leading directly to an exhaustion of thread pools.<\/p>\n The call out in this case is \u201cDo not synchronously wait on Monitors, Events, Task, or any other objects that may block your thread. See if you can update the method to be asynchronous.\u201d. My next job is to find that problematic code.<\/p>\n By clicking on the Show call stack<\/strong> link Visual Studio will immediately switch to the threads that are exhibiting this behavior and the Call Stack<\/strong> window will show me all the methods that require further examination. I can quickly distinguish between my code (SyncOverAsyncExmple.*) from Framework code (System.*), and I can use this as the starting point of my investigation.<\/p>\n Each frame (or row) of a call stack corresponds to a method and by double-clicking on any of the stack frames I prompt Visual Studio to lead me to the code that led directly to this scenario on this thread. Unfortunately I do not have the symbols or the code associated with this application so on the Symbols not loaded<\/strong> page below I can select the Decompile Source code option<\/a>.<\/p>\n In the decompiled source below it is clear to me that I have an asynchronous Task<\/strong> (ConsumeThreadPoolThread) calling a function (DoSomething) that contains a WaitHandle.WaitOne method<\/a>, which is blocking the current thread pool thread until it receives a signal (exactly what I want to avoid!). So to improve the responsiveness of my app I have to find a way to remove this blocking code from all asynchronous contexts.<\/p>\n\n
Automatic analysis of a memory dump<\/h3>\n
Open the memory dump<\/h4>\n
![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/1-visualstudio-dump-summary-actions.png\")
<\/a><\/p>\nSelecting and executing analyzers against the dump<\/h4>\n
![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/2-visualstudio-diagnostics-analysis-window.png\")
<\/a><\/p>\nReviewing the results of the analyzers<\/h4>\n
![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/3-visualstudio-diagnostics-analysis-results.png\")
<\/a><\/p>\n![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/4-visualstudio-diagnostics-analysis-results2.png\")
<\/a><\/p>\nNavigating to the problematic code<\/h4>\n
![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/5-visualstudio-call-stack.png\")
<\/a><\/p>\n![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/6-visualstudio-decompilation.png\")
<\/a><\/p>\n
![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2021\/04\/7-visualstudio-decompiled-code.png\")
<\/a><\/p>\n