{"id":35477,"date":"2019-03-06T12:32:18","date_gmt":"2019-03-06T19:32:18","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/premier-developer\/?p=35477"},"modified":"2019-02-26T14:04:42","modified_gmt":"2019-02-26T21:04:42","slug":"gathering-real-time-perfmon-counters-in-a-cluster","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/gathering-real-time-perfmon-counters-in-a-cluster\/","title":{"rendered":"Gathering real-time Perfmon Counters in a cluster"},"content":{"rendered":"<p><span style=\"color: #333333;font-family: 'Segoe UI',sans-serif;font-size: 12pt\">Premier Development Consultant <a href=\"https:\/\/nam06.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fkurtschenk%2F&amp;data=02%7C01%7Creedr%40microsoft.com%7Ca7a2d2d3c9344e7fefb908d69c20f261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636868062669800500&amp;sdata=dLN%2BLuGaxmV7ufhnfM5RPmq7kH7YBq3hwkrRWwasFvI%3D&amp;reserved=0\">Kurt Schenk<\/a> demonstrates how to gather real-time performance metrics in a cluster.<\/span><\/p>\n<hr \/>\n<p>Performance (aka. Perfmon) Counters are critical to understanding the health of and diagnosing issues on Windows. In recent performance and scalability testing of a solution built on top of Azure Service Fabric, we collected Perfmon Counters across 15 VMs and sent them to Log Analytics. This was very helpful; however, there were times when we wanted them to be real time. In our case, we wanted to verify core OS Perfmon Counters like Disk, CPU, Memory, Exceptions, and since we were driving load over WebSockets, how many connections were established. In order to enable this, I created a PowerShell script that can easily be configured for the particular Perfmon Counters you want to collect, and machines you want monitor. \u00a0Ran this script in PowerShell ISE:<\/p>\n<pre class=\"lang:ps decode:true \"># uncomment the line below and run by itself to help you find the correct name for the Perfmon Counters you are interested in\r\n#(get-counter -listset \".NET CLR Memory\").paths\r\n \r\n$listOfMetrics = @(\r\n      \"\\LogicalDisk(c:)\\% Free Space\"\r\n      \"\\LogicalDisk(d:)\\% Free Space\",\r\n      \"\\LogicalDisk(d:)\\Free Megabytes\",\r\n      \"\\processor(_Total)\\% processor time\",\r\n      \"\\Memory\\Available MBytes\",\r\n      \"\\TCPv4\\Connections Established\",\r\n      \"\\.NET CLR Exceptions(_global_)\\# of Exceps Thrown \/ sec\",\r\n      \"\\.NET CLR Memory(_global_)\\% Time in GC\"\r\n)\r\n \r\n$vms2 = \"zero000000\", \"zero000001\", \"zero000002\", \"zero000003\", \"zero000004\", \"zero00000J\", \"zero00000K\", \"zero00000L\", \"zero00000M\", `\r\n\"zero00000N\", \"zero00000O\", \"zero00000X\", \"zero00000Y\", \"zero00000Z\", \"zero000010\"\r\n \r\nGet-counter -Counter $listOfMetrics -SampleInterval 1 -MaxSamples 1 -ComputerName $vms2 | select -expand countersamples | `\r\nOut-GridView #Format-Table -AutoSize # | Sort-Object Path | Tree\r\n<\/pre>\n<p>Which displays following.<\/p>\n<p><img decoding=\"async\" width=\"1920\" height=\"1200\" class=\"wp-image-35478\" src=\"http:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-out-gridview.png\" alt=\"Machine generated alternative text: Out-GridView #Format-Table -AutoSize # I Sort-Object Path I Tree Filter Add criteria Path \\\\zer000000k\\\\logicaldisk(d:)\\% free space \\\\zer0000001\\\\logicaldisk(d:)\\% free space \\\\zer000000m\\\\logicaldisk(d:)\\% free space \\\\zer000000n\\\\logicaldisk(d:)\\% free space \\\\zer0000000\\\\logicaldisk(d:)\\% free space \\\\zer000000x\\\\logicaldisk(d:)\\% free space \\\\zer000000y\\\\logicaldisk(d:)\\% free space \\\\zer000000z\\\\logicaldisk(d:)\\% free space \\\\zer0000010\\\\logicaldisk(d:)\\% free space \\\\zer0000000\\\\logicaldisk(d:)\\free megabytes \\\\zer0000001\\\\logicaldisk(d:)\\free megabytes \\\\zer0000002\\\\logicaldisk(d:)\\free megabytes \\\\zer0000003\\\\logicaldisk(d:)\\free megabytes \\\\zer0000004\\\\logicaldisk(d:)\\free megabytes \\\\zer000000j\\\\logicaldisk(d:)\\free megabytes \\\\zer000000k\\\\logicaldisk(d:)\\free megabytes \\\\zer0000001\\\\logicaldisk(d:)\\free megabytes \\\\zer000000m\\\\logicaldisk(d:)\\free megabytes \\\\zer000000n\\\\logicaldisk(d:)\\free megabytes \\\\zer0000000\\\\logicaldisk(d:)\\free megabytes \\\\zer000000x\\\\logicaldisk(d:)\\free megabytes \\\\zer000000y\\\\logicaldisk(d:)\\free megabytes \\\\zer000000z\\\\logicaldisk(d:)\\free megabytes \\\\zer0000010\\\\logicaldisk(d:)\\free megabytes \\\\zer0000000\\\\processorCtotal)\\% processor time \\\\zer0000001\\\\processorCtotal)\\% processor time \\\\zer0000002\\\\processorCtotal)\\% processor time \\\\zer0000003\\\\processorCtotal)\\% processor time \\\\zer0000004\\\\processorCtotal)\\% processor time \\\\zer000000j\\\\processor(_total)\\% processor time \\\\zer000000k\\\\processor(_total)\\% processor time x InstanceName d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: total total total total total total total CookedValue 22.49 43.73 34.66 42.91 44.19 34.86 44.89 45.97 14.70 16,938.00 30,005.00 28,565.00 28,081.00 22,742.00 20,806.00 14,736.00 28,655.00 22,717.00 28,121.00 28,960.00 22,846.00 29,417.00 30,123.00 9,631.00 1.61 0.33 0.71 1.01 0.79 0.57 1.53 10:02 PM 11\/21\/2018 \" srcset=\"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-out-gridview.png 1920w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-out-gridview-300x188.png 300w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-out-gridview-768x480.png 768w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-out-gridview-1024x640.png 1024w\" sizes=\"(max-width: 1920px) 100vw, 1920px\" \/><\/p>\n<p>And with the PowerShell grid, it is easy to sort and filter (for example by machine name below).<\/p>\n<p><img decoding=\"async\" width=\"1920\" height=\"1080\" class=\"wp-image-35479\" src=\"http:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-administrator.png\" alt=\"Machine generated alternative text: Administrator: Windows PowerShell ISE File Edit View Tools Debug Add-ons Help Get-Counte Out-GridView #Format-Table -AutoSize # I Sort-Object Path I Tree x 1 2 4 5 6 7 8 9 10 11 12 13 14 15 16 Svm 17 &quot; ze 18 19 \u2022 \\usel PS C. \u2022 \\usel PS C. \u2022 \\usel PS C. Completed zer000000m Add criteria Path \\\\zer000000m\\\\logicaldisk(c:)\\% free space \\\\zer000000m\\\\logicaldisk(d:)\\% free spa... \\\\zer000000m\\\\logicaldisk(d:)\\free mega.. \\\\zer000000m\\\\processor(_total)\\96 proc.. \\\\zer000000m\\\\memory\\available m bytes \\\\zer000000m\\\\tcpv4\\connections establ... \\\\zer000000m\\\\.net clr exceptions(_glob.. \\\\zer000000m\\\\.net clr InstanceName c: d: d: total global global CookedValue 77.45 60.96 39,951.00 2.67 10,670.00 135.00 0.00 0.07 Ln4 Col 1 100% \" srcset=\"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-administrator.png 1920w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-administrator-300x169.png 300w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-administrator-768x432.png 768w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/02\/machine-generated-alternative-text-administrator-1024x576.png 1024w\" sizes=\"(max-width: 1920px) 100vw, 1920px\" \/><\/p>\n<p>Keep in mind that this results are a snapshot. To update run the script again.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Performance (aka. Perfmon) Counters are critical to understanding the health of and diagnosing issues on Windows. In recent performance and scalability testing of a solution built on top of Azure Service Fabric, we collected Perfmon Counters across 15 VMs and sent them to Log Analytics. This was very helpful; however, there were times when we wanted them to be real time.<\/p>\n","protected":false},"author":582,"featured_media":35480,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[25,128],"tags":[24,2678,101,3],"class_list":["post-35477","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-performance","tag-azure","tag-azure-service-fabric","tag-performance","tag-team"],"acf":[],"blog_post_summary":"<p>Performance (aka. Perfmon) Counters are critical to understanding the health of and diagnosing issues on Windows. In recent performance and scalability testing of a solution built on top of Azure Service Fabric, we collected Perfmon Counters across 15 VMs and sent them to Log Analytics. This was very helpful; however, there were times when we wanted them to be real time.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/35477","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/582"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=35477"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/35477\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/35480"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=35477"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=35477"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=35477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}