{"@attributes":{"version":"2.0"},"channel":{"title":"Code Thoughts","description":"Thoughts on code","link":"https:\/\/jackmott.github.io\/\/","pubDate":"Tue, 14 Aug 2018 20:05:02 +0000","lastBuildDate":"Tue, 14 Aug 2018 20:05:02 +0000","generator":"Jekyll v3.7.3","item":[{"title":"Does Your Code Leave a Trail of Slowness?","description":"

The Trail of Slowness<\/h1>\n\n

Often I implore people to make better performing coding decisions when the downsides\nare small or nonexistent. A common response to this is that you should only worry\nabout performance when you have measured the code in question and found performance to \nbe an issue. There a couple of problems with that ethos, the first of which is that\nsometimes early decisions will be hard to reverse late in a project if performance turns\nout to be an issue. But there is a more insidious problem. Areas of your code for which\nperformance is not important, may be causing other<\/em> code, or even other programs<\/em> to slow\ndown. This trail of slowness left behind by uninformed performance decisions will not\nshow up in any particular place in a profiler, they will just slow down everything a bit.<\/p>\n\n

Why Does This Happen<\/h1>\n\n

Modern CPU performance is severely limited by RAM latency. A request to get data from RAM\ncan take over 100 CPU cycles to complete, and while your CPU waits, it just sits there, doing\nnothing useful. This problem is addressed by a series of caches in your CPU, smaller but much \nfaster regions of memory. Whenever you request data from RAM, you get back an entire cache line\nof contiguous memory into the CPU caches. Assuming that the next memory your request is in that\ncontiguous segment of memory in the cache line, you will get it very quickly.<\/p>\n\n

This is why iterating over an array in order (contiguous memory) is much, much faster than iterating over\na LinkedList, where each node is in a random(ish) location in the heap. Many of those requests\nfrom RAM while iterating over a LinkedList miss the cache, and cause long CPU stalls.<\/p>\n\n

But it is worse than just being slow. Every time you request memory that isn\u2019t in the cache\nalready, you have to pull down a whole cache line and replace data that is already in the cache. Any code running\nthat might have hoped to use that cached data won\u2019t have it, and will now have to get it from RAM\nagain. This is sometimes called \u201cthrashing the cache\u201d.<\/p>\n\n

How Bad Can It Be?<\/h1>\n\n

This can vary wildly based on the workloads involved. In the contrived example below, I do a \nfixed amount of work with a LinkedList, while at the same time I have other threads running\ndoing as much work as possible for 5 seconds. They are able to perform the ArraySumSquare function\nabout 3.5 million times in that 5 seconds.<\/p>\n\n

When I alter the RunLinkedList function to do the same fixed amount of work on arrays instead,\nthe original array loops are now able to perform ArraySumSquare about 4.4 million times.<\/p>\n\n

This could be similar to a situation where a server has a periodic computation it does, where\ntaking a few seconds isn\u2019t considered a problem at all. But that job is having a significant\nimpact on users using the live system, increasing latency by a ~20%. Or it could be similar to a code editor that\nis doing some parsing behind the scenes where the performance seems fine when you profile it, but it is \nslowing down UI response due to the cache thrashing.<\/p>\n\n

What To Do<\/h1>\n\n

Use arrays or array backed lists by default unless you have good reasons not to. Most languages have resize-able array backed \nstructures that are as convenient to use as LinkedLists:<\/p>\n\n