{"@attributes":{"version":"2.0"},"channel":{"title":"artificial::mind blog","description":"Articles about Graphics, C++, Optimization, Game Development, Programming Languages, and more. Personal blog of Philip Tretter writing under the alias of Artificial Mind.\n","link":"https:\/\/artificial-mind.net","item":[{"title":"Moves in Returns","description":"Mini guide to 'when is my return a move?'","pubDate":"Sat, 23 Oct 2021 02:00:00 +0000","link":"https:\/\/artificial-mind.net\/blog\/2021\/10\/23\/return-moves","guid":"https:\/\/artificial-mind.net\/blog\/2021\/10\/23\/return-moves","content":"

Today we\u2019ll discuss code of the form:<\/p>\n\n

T<\/span> work<\/span>(<\/span>\/* ... *\/<\/span>)<\/span>\n{<\/span>\n    \/* ... *\/<\/span>\n    return<\/span> x<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
This is a classical \u201creturn-by-value\u201d and (wrongfully) associated with copies and overhead.<\/p>\n\n
In many cases, this will actually move<\/code> the result instead of copying it.\nFor modern C++, one could even argue that this will move in most<\/em> cases (or, as we will see, completely elide<\/em> the copy and directly construct in the result memory).<\/p>\n\n
This post discusses several common patterns and if they are moved, copies, or elided.<\/p>\n\n
\n  
Side note: technically<\/em> a move is a type of copy.\nFor example, T x = <expr><\/code> performs a copy initialization<\/a>, which might select the move constructor during overload resolution.\nThe rest of this post will use the colloquial \u201cmove\u201d for \u201ccalls move ctor or assignment\u201d and \u201ccopy\u201d for \u201ccalls copy ctor or assignment\u201d.<\/p>\n<\/blockquote>\n\n
Tracking Construction and Assignment<\/h1>\n\n
Reading the C++ standard (or cppreference) to reason about your code is valuable, but given the flood of information, it can be difficult to draw the correct inferences.\nThus, in addition to this theoretical<\/em> understanding, I love to validate my findings on godbolt<\/a>.<\/p>\n\n
The following examples always use a type T<\/code>, defined as:<\/p>\n\n
struct<\/span> T<\/span>\n{<\/span>\n    T<\/span>();<\/span>                    \/\/ ctor<\/span>\n    ~<\/span>T<\/span>();<\/span>                   \/\/ dtor<\/span>\n    T<\/span>(<\/span>T<\/span>&&<\/span>);<\/span>                 \/\/ move ctor<\/span>\n    T<\/span>(<\/span>T<\/span> const<\/span>&<\/span>);<\/span>            \/\/ copy ctor<\/span>\n    T<\/span>&<\/span> operator<\/span>=<\/span>(<\/span>T<\/span>&&<\/span>);<\/span>      \/\/ move assignment<\/span>\n    T<\/span>&<\/span> operator<\/span>=<\/span>(<\/span>T<\/span> const<\/span>&<\/span>);<\/span> \/\/ copy assignment<\/span>\n};<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
These functions are not implemented on purpose, so we will see the corresponding calls in the assembly.\nFurthermore, I\u2019ll mark the work<\/code> functions as __attribute__((noinline))<\/code> so that we can see which special function calls belong where (caller or callee).<\/p>\n\n
Constructing Objects in Return<\/h1>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    return<\/span> T<\/span>();<\/span>\n}<\/span>\n\nvoid<\/span> use<\/span>()<\/span>\n{<\/span>\n    auto<\/span> obj<\/span> =<\/span> work<\/span>();<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Assembly<\/a>:<\/p>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\nuse<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> work<\/span>()<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
T<\/code> is constructed in work<\/code> in a memory location that is provided by the caller use<\/code>.\nAt the end of use<\/code>, T<\/code> is destructed.\nNo temporary copies are created, nothing is moved.<\/p>\n\n
This \u201cchains\u201d in the sense that it also applies to return other_work();<\/code> where other_work<\/code> also returns a T<\/code> by value.<\/p>\n\n
With return T();<\/code>, work<\/code> will always call exactly one constructor and nothing else from T<\/code>.\nHowever, use<\/code> is only this simple because we initialize obj<\/code> with the result of work()<\/code>.\nIf we assign it to an existing object, we get a temporary:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    return<\/span> T<\/span>();<\/span>\n}<\/span>\n\nvoid<\/span> use<\/span>(<\/span>T<\/span>&<\/span> obj<\/span>)<\/span>\n{<\/span>\n    obj<\/span> =<\/span> work<\/span>();<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\nuse<\/span>(<\/span>T<\/span>&<\/span>)<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> work<\/span>()<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>operator<\/span>=<\/span>(<\/span>T<\/span>&&<\/span>)<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
work<\/code> constructs a T<\/code> for which use<\/code> provides the stack space.\nThis temporary T<\/code> is then moved<\/em> into obj<\/code> using T::operator=(T&&)<\/code>.\nFinally, the temporary T<\/code> is destroyed.<\/p>\n\n
Before C++17, this type of optimization was allowed, but optional.\nIn particular, if your object is neither copyable nor movable, you could run into compile errors depending on if this optimization was applied or not (e.g. debug vs. release).<\/p>\n\n
\n  
Note: In C++17, this direct creation of the result in space provided by the caller has the fancy name \u201cunmaterialized value passing\u201d.<\/p>\n<\/blockquote>\n\n
Returning a Local Variable<\/h1>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    T<\/span> obj<\/span>;<\/span>\n    \/\/ ...<\/span>\n    return<\/span> obj<\/span>;<\/span>\n}<\/span>\n\nvoid<\/span> use<\/span>()<\/span>\n{<\/span>\n    auto<\/span> obj<\/span> =<\/span> work<\/span>();<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\nuse<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> work<\/span>()<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
It gets more interesting if we have other ways out of the function:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    if<\/span> (<\/span>some_condition<\/span>())<\/span>\n        return<\/span> T<\/span>();<\/span>\n\n    T<\/span> obj<\/span>;<\/span>\n    return<\/span> obj<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> some_condition<\/span>()<\/span>\n  if<\/span>:<\/span>\n    call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n    ...<\/span>\n    ret<\/span>\n  else<\/span>:<\/span>\n    ...<\/span>\n    call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n    ...<\/span>\n    call<\/span> T<\/span>::<\/span>T<\/span>(<\/span>T<\/span>&&<\/span>)<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n    ...<\/span>\n    call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n    ...<\/span>\n    ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
If the condition is true<\/code>, we construct the result directly as before.\nHowever, the second case now creates a temporary T<\/code>.\nThis temporary is then move-constructed into the return value.\nAfterwards, the temporary is destructed.<\/p>\n\n
\n  
Note: the actual rule has a few nuances, but this is a good first-order approximation.<\/p>\n<\/blockquote>\n\n
Moving from a Local Variable<\/h1>\n\n
You might have seen the following:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    T<\/span> obj<\/span>;<\/span>\n    return<\/span> std<\/span>::<\/span>move<\/span>(<\/span>obj<\/span>);<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>(<\/span>T<\/span>&&<\/span>)<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
A temporary that is move-constructed into the return value.\nWithout the std::move<\/code>, we had no move construction at all.<\/p>\n\n
Returning a Function Parameter<\/h1>\n\n
Local variables and function parameters have slightly different behavior.<\/p>\n\n
T<\/span> work<\/span>(<\/span>T<\/span> obj<\/span>)<\/span>\n{<\/span>\n    return<\/span> obj<\/span>;<\/span>\n}<\/span>\n\nvoid<\/span> use<\/span>()<\/span>\n{<\/span>\n    auto<\/span> obj<\/span> =<\/span> work<\/span>(<\/span>T<\/span>());<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
When obj<\/code> was a local variable, we had the freedom to change where<\/em> it is allocated.\nIf all paths through the function end in return obj<\/code>, the compiler could use the caller-provided space for the return value, thus eliding<\/em> any move into the result.<\/p>\n\n
work<\/span>(<\/span>T<\/span>)<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>(<\/span>T<\/span>&&<\/span>)<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\nuse<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> work<\/span>(<\/span>T<\/span>)<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::~<\/span>T<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Even passed-by-value, no T<\/code> is copied in this whole example.\nThe caller (use<\/code>) creates a T<\/code> where work<\/code> will expect it.\nwork<\/code> itself only move-constructs T<\/code> in the return value.\nuse<\/code> then destructs the argument T<\/code> (\u201cat the end of the statement\u201d), followed by destruction of the result of work<\/code> (\u201cat the end of the scope\u201d).<\/p>\n\n
Non-Matching Types<\/h1>\n\n
Copy elision only works if the result type matches what we want to return.\nThis might not always be the case.\nSomething I find myself writing with decent frequency:<\/p>\n\n
std<\/span>::<\/span>optional<\/span><<\/span>T<\/span>><\/span> work<\/span>()<\/span>\n{<\/span>\n    if<\/span> (<\/span>some_condition<\/span>())<\/span>\n        return<\/span> std<\/span>::<\/span>nullopt<\/span>;<\/span>\n\n    return<\/span> T<\/span>();<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
And the obvious question is if the second return<\/code> copies<\/em> or moves<\/em> the T<\/code> into the optional<T><\/code>.<\/p>\n\n
More general, let\u2019s say we have a second type U<\/code> and T<\/code> has implicit conversions:<\/p>\n\n
struct<\/span> T<\/span>\n{<\/span>\n    ...<\/span>\n    T<\/span>(<\/span>U<\/span>&&<\/span>)<\/span> noexcept<\/span>;<\/span>      \/\/ \"move-convert\"<\/span>\n    T<\/span>(<\/span>U<\/span> const<\/span>&<\/span>)<\/span> noexcept<\/span>;<\/span> \/\/ \"copy-convert\"<\/span>\n};<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Now we can ask the question what the following code will call:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    return<\/span> U<\/span>();<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
or<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    U<\/span> obj<\/span>;<\/span>\n    return<\/span> obj<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Both result in the same<\/a> assembly<\/a>:<\/p>\n\n
work<\/span>()<\/span>:<\/span>\n  ...<\/span>\n  call<\/span> U<\/span>::<\/span>U<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> T<\/span>::<\/span>T<\/span>(<\/span>U<\/span>&&<\/span>)<\/span> [<\/span>complete<\/span> object<\/span> constructor<\/span>]<\/span>\n  ...<\/span>\n  call<\/span> U<\/span>::~<\/span>U<\/span>()<\/span> [<\/span>complete<\/span> object<\/span> destructor<\/span>]<\/span>\n  ...<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
A temporary U<\/code> is created, move-\u201cconverted\u201d into the result T<\/code>, and then destructed.\nNo copy involved.<\/p>\n\n
\n  
Fun fact: in \u201cvanilla\u201d C++11, this created a copy.\nThe behavior was fixed in C++14 and back-ported via defect report<\/a>.\nThus, most compiler with C++14 support will emit the move, even if you explicitly compile for C++11.\nHowever, pre-C++14 compiler might emit the copy.<\/p>\n<\/blockquote>\n\n
Where Copy??<\/h1>\n\n
It delights me to see so many cases where the default (without any std::move<\/code> involved) will result in either move construction or even complete elision of copy or move.<\/p>\n\n
So, when will return-by-value actually copy?\nA small collection of patterns to look out for:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    struct<\/span> {<\/span> T<\/span> t<\/span>;<\/span> }<\/span> v<\/span>;<\/span>\n    return<\/span> v<\/span>.<\/span>t<\/span>;<\/span> \/\/ COPY! returning a member<\/span>\n}<\/span>\n\nT<\/span> work<\/span>()<\/span>\n{<\/span>\n    static<\/span> T<\/span> globalT<\/span>;<\/span>\n    return<\/span> globalT<\/span>;<\/span> \/\/ COPY! not a local var<\/span>\n}<\/span>\n\nT<\/span> work<\/span>(<\/span>T<\/span>&<\/span> obj<\/span>)<\/span>\n{<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ COPY! T& matches T const&, not T&& <\/span>\n}<\/span>\n\nT<\/span> work<\/span>(<\/span>T<\/span> const<\/span>&<\/span> obj<\/span>)<\/span>\n{<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ COPY!<\/span>\n}<\/span>\n\nT<\/span> work<\/span>(<\/span>T<\/span>&&<\/span> obj<\/span>)<\/span>\n{<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ COPY! \"inside\" work, obj is an lvalue<\/span>\n                \/\/ NOTE: careful with lifetimes here<\/span>\n                \/\/ NOTE: is a move in C++20<\/span>\n}<\/span>\n\nT<\/span> work<\/span>(<\/span>T<\/span> const<\/span> obj<\/span>)<\/span>\n{<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ COPY! T const cannot be moved<\/span>\n                \/\/ NOTE: const is ignored for the signature,<\/span>\n                \/\/       which is work(T) and not work(T const)<\/span>\n                \/\/       but has \"effect\" inside the function<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    T<\/span> const<\/span> obj<\/span>;<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ no copy or move involved<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Because while we cannot move T const<\/code>, we can still allocate it directly \u201cin\u201d the return value.\nStill, this is a bit brittle, as a slightly more complex function will create a copy:<\/p>\n\n
T<\/span> work<\/span>()<\/span>\n{<\/span>\n    if<\/span> (<\/span>some_condition<\/span>())<\/span>\n        return<\/span> T<\/span>();<\/span>\n\n    T<\/span> const<\/span> obj<\/span>;<\/span>\n    return<\/span> obj<\/span>;<\/span> \/\/ now it's a copy<\/span>\n}<\/span>\n\nT<\/span> work<\/span>()<\/span>\n{<\/span>\n    T<\/span> const<\/span> obj<\/span>;<\/span>\n    return<\/span> std<\/span>::<\/span>move<\/span>(<\/span>obj<\/span>);<\/span> \/\/ also a copy<\/span>\n                           \/\/ T const&& matches T const&, not T&&<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Example<\/h1>\n\n
Let us consider this simple test<\/code> function,<\/p>\n\n
\/\/ implemented in different TU or otherwise linked in<\/span>\nint<\/span> foo<\/span>(<\/span>int<\/span> n<\/span>);<\/span>\n\nint<\/span> test<\/span>(<\/span>int<\/span> const<\/span>&<\/span> n<\/span>)<\/span> \n{<\/span>\n    auto<\/span> sum<\/span> =<\/span> 0<\/span>;<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    return<\/span> sum<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
test<\/span>(<\/span>int<\/span> const<\/span>&<\/span>)<\/span>:<\/span>\n  push<\/span> rbp<\/span>\n  push<\/span> rbx<\/span>\n  push<\/span> rax<\/span>\n  mov<\/span> rbx<\/span>,<\/span> rdi<\/span>\n  mov<\/span> edi<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rdi<\/span>]<\/span> \/\/ load n from memory<\/span>\n  call<\/span> foo<\/span>(<\/span>int<\/span>)<\/span>            \/\/ call foo<\/span>\n  mov<\/span> ebp<\/span>,<\/span> eax<\/span>\n  mov<\/span> edi<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rbx<\/span>]<\/span> \/\/ load n from memory _again_<\/span>\n  call<\/span> foo<\/span>(<\/span>int<\/span>)<\/span>            \/\/ call foo<\/span>\n  add<\/span> eax<\/span>,<\/span> ebp<\/span>\n  add<\/span> rsp<\/span>,<\/span> 8<\/span>\n  pop<\/span> rbx<\/span>\n  pop<\/span> rbp<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Unsurprising, foo<\/code> is called twice.\nFor all we know, it has important side effects that have to be executed twice.\nMaybe a bit more subtle: n<\/code> had to be reloaded from memory, because the first call to foo<\/code> might have changed the memory.\n(For example, n<\/code> might be a reference to a global int<\/code> that foo<\/code> happens to increment.)<\/p>\n\n
This is really the worst case for the compiler.\nNothing about the inner workings of foo<\/code> is known.<\/p>\n\n
On the other end of the spectrum, we have a small, known foo<\/code>:<\/p>\n\n
int<\/span> foo<\/span>(<\/span>int<\/span> n<\/span>)<\/span>\n{<\/span>\n    return<\/span> n<\/span> *<\/span> n<\/span>;<\/span>\n}<\/span>\n\nint<\/span> test<\/span>(<\/span>int<\/span> const<\/span>&<\/span> n<\/span>)<\/span> \n{<\/span>\n    auto<\/span> sum<\/span> =<\/span> 0<\/span>;<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    return<\/span> sum<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
test<\/span>(<\/span>int<\/span> const<\/span>&<\/span>)<\/span>:<\/span>\n  mov<\/span> eax<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rdi<\/span>]<\/span> \/\/ load n from memory<\/span>\n  imul<\/span> eax<\/span>,<\/span> eax<\/span>            \/\/ tmp = n*n<\/span>\n  add<\/span> eax<\/span>,<\/span> eax<\/span>             \/\/ return tmp + tmp<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Not only is the call to foo<\/code> inlined, further optimizations made sure that n * n<\/code> is not computed twice.<\/p>\n\n
To Inline or not to Inline<\/h1>\n\n
If foo<\/code> is part of a different TU, the compiler can obviously not inline the call.\nShould you have enough patience, link-time optimization<\/a> might come to the rescue.\n(It\u2019s usually really expensive, so I can only really recommend it for CI building release versions.)<\/p>\n\n
However, there are many other reasons why foo<\/code> is not inlined.\nIn general, inlining is a double-edged sword.\nEach inlined function increases the pressure on the instruction cache<\/a> and makes the parent function harder to analyze.\nMore local variables mean more register spills<\/a>.\nAll these can negatively affect performance and compilers have various heuristic to try to make good decisions.<\/p>\n\n
Most small functions will get inlined by default.\nAs the complexity of the function body rises, this will stop at some point.\nRecursion usually<\/em> prevents inlining, though compilers are definitely able to inline recursive functions<\/a> via tail-call elimination<\/a>.<\/p>\n\n
We can artificially prevent inlining by declaring the function __declspec(noinline)<\/code> (for msvc) or __attribute__((noinline))<\/code> (for gcc\/clang):<\/p>\n\n
__attribute__<\/span>((<\/span>noinline<\/span>))<\/span> int<\/span> foo<\/span>(<\/span>int<\/span> n<\/span>)<\/span>\n{<\/span>\n    return<\/span> n<\/span> *<\/span> n<\/span>;<\/span>\n}<\/span>\n\nint<\/span> test<\/span>(<\/span>int<\/span> const<\/span>&<\/span> n<\/span>)<\/span> \n{<\/span>\n    auto<\/span> sum<\/span> =<\/span> 0<\/span>;<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    return<\/span> sum<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
test<\/span>(<\/span>int<\/span> const<\/span>&<\/span>)<\/span>:<\/span>\n  mov<\/span> edi<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rdi<\/span>]<\/span> \/\/ load n from memory<\/span>\n  call<\/span> foo<\/span>(<\/span>int<\/span>)<\/span>            \/\/ tmp = foo(n)<\/span>\n  add<\/span> eax<\/span>,<\/span> eax<\/span>             \/\/ return tmp + tmp<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
Pure functions are nice.\nPure functions return the same result if given the same input.\nPure functions do not modify global state, they work solely on their input values.\nThus, foo(n) + foo(n)<\/code> was simplified to tmp = foo(n); tmp + tmp<\/code>, even without inlining.<\/p>\n\n
Fun with Loops<\/h1>\n\n
The difference becomes even larger with loops:<\/p>\n\n
\/\/ different TU<\/span>\nint<\/span> foo<\/span>(<\/span>int<\/span> n<\/span>);<\/span>\n\nint<\/span> test<\/span>(<\/span>int<\/span> const<\/span>&<\/span> n<\/span>)<\/span> \n{<\/span>\n    auto<\/span> sum<\/span> =<\/span> 0<\/span>;<\/span>\n    for<\/span> (<\/span>auto<\/span> i<\/span> =<\/span> 0<\/span>;<\/span> i<\/span> <=<\/span> n<\/span>;<\/span> ++<\/span>i<\/span>)<\/span>\n        sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    return<\/span> sum<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
test<\/span>(<\/span>int<\/span> const<\/span>&<\/span>)<\/span>:<\/span>\n  push<\/span> rbp<\/span>\n  push<\/span> r14<\/span>\n  push<\/span> rbx<\/span>\n  mov<\/span> r14<\/span>,<\/span> rdi<\/span>\n  mov<\/span> edi<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rdi<\/span>]<\/span>\n  xor<\/span> ebp<\/span>,<\/span> ebp<\/span>\n  test<\/span> edi<\/span>,<\/span> edi<\/span> \/\/ n < 0 case<\/span>\n  js<\/span> .<\/span>LBB0_3<\/span>\n  mov<\/span> ebx<\/span>,<\/span> -<\/span>1<\/span>\n.<\/span>LBB0_2<\/span>:<\/span>         \/\/ loop body begin<\/span>\n  call<\/span> foo<\/span>(<\/span>int<\/span>)<\/span>            \/\/ call foo<\/span>\n  add<\/span> ebp<\/span>,<\/span> eax<\/span>\n  mov<\/span> edi<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>r14<\/span>]<\/span> \/\/ reload n from memory<\/span>\n  inc<\/span> ebx<\/span>\n  cmp<\/span> ebx<\/span>,<\/span> edi<\/span>\n  jl<\/span> .<\/span>LBB0_2<\/span>     \/\/ loop body end<\/span>\n.<\/span>LBB0_3<\/span>:<\/span>\n  mov<\/span> eax<\/span>,<\/span> ebp<\/span>\n  pop<\/span> rbx<\/span>\n  pop<\/span> r14<\/span>\n  pop<\/span> rbp<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
With no information, the compiler has to call foo<\/code> (and load n<\/code> from memory) in each iteration.\nContrast this with:<\/p>\n\n
__attribute__<\/span>((<\/span>noinline<\/span>))<\/span> int<\/span> foo<\/span>(<\/span>int<\/span> n<\/span>)<\/span>\n{<\/span>\n    return<\/span> n<\/span> *<\/span> n<\/span>;<\/span>\n}<\/span>\n\nint<\/span> test<\/span>(<\/span>int<\/span> const<\/span>&<\/span> n<\/span>)<\/span> \n{<\/span>\n    auto<\/span> sum<\/span> =<\/span> 0<\/span>;<\/span>\n    for<\/span> (<\/span>auto<\/span> i<\/span> =<\/span> 0<\/span>;<\/span> i<\/span> <=<\/span> n<\/span>;<\/span> ++<\/span>i<\/span>)<\/span>\n        sum<\/span> +=<\/span> foo<\/span>(<\/span>n<\/span>);<\/span>\n    return<\/span> sum<\/span>;<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
test<\/span>(<\/span>int<\/span> const<\/span>&<\/span>)<\/span>:<\/span>\n  mov<\/span> edx<\/span>,<\/span> dword<\/span> ptr<\/span> [<\/span>rdi<\/span>]<\/span>\n  test<\/span> edx<\/span>,<\/span> edx<\/span>\n  js<\/span> .<\/span>L5<\/span> \/\/ special case n < 0<\/span>\n  mov<\/span> edi<\/span>,<\/span> edx<\/span>\n  call<\/span> foo<\/span>(<\/span>int<\/span>)<\/span>  \/\/ tmp1 = foo(n)<\/span>\n  imul<\/span> edx<\/span>,<\/span> eax<\/span>  \/\/ tmp2 = tmp1 * n<\/span>\n  add<\/span> eax<\/span>,<\/span> edx<\/span>   \/\/ return tmp2 + n<\/span>\n  ret<\/span>\n.<\/span>L5<\/span>:<\/span>\n  xor<\/span> eax<\/span>,<\/span> eax<\/span> \/\/ return 0<\/span>\n  ret<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
So gcc is able to optimize the whole thing to basically foo(n) * (n + 1)<\/code>, without inlining.\nFunnily enough, clang tries (and fails) to be clever with lots of SIMD.<\/p>\n\n
Conclusion<\/h1>\n\n
This is not a long post, but it shows that while inlining is a very important optimization, a non-inlined function is not the end of the world<\/del> optimization.\nAs long as the function implementation is visible, compilers can and will analyze them so that they don\u2019t have to assume the worst case.\nThis, in turn, re-enables many optimizations such as value numbering<\/a>, common subexpression elimination<\/a>, and loop-invariant code motion<\/a>.<\/p>\n\n
PS: gcc and clang have a variety of function attributes<\/a> that can be used to enable optimizations, even if the implementation is in a different TU.\nThe two most important ones are __attribute__((const))<\/code> and __attribute__((pure))<\/code>.<\/p>\n\n
(Title image from unsplash<\/a><\/em>)<\/p>"},{"title":"Measuring std::unordered_map Badness","description":"When is a good hash a good hash?","pubDate":"Sat, 09 Oct 2021 02:00:00 +0000","link":"https:\/\/artificial-mind.net\/blog\/2021\/10\/09\/unordered-map-badness","guid":"https:\/\/artificial-mind.net\/blog\/2021\/10\/09\/unordered-map-badness","content":"
I probably could have generated more clicks by titling this \u201cHow I Improved std::unordered_map<\/code> Performance By 50x With This Little Trick\u201d.<\/p>\n\n
In reality, this post is mostly about my journey crafting the following snippet that allows you to measure the quality of your hash function on a concrete std::unordered_map<\/code>:<\/p>\n\n
template<\/span> <<\/span>class<\/span> Map<\/span>><\/span> \ndouble<\/span> unordered_map_badness<\/span>(<\/span>Map<\/span> const<\/span>&<\/span> map<\/span>)<\/span>\n{<\/span>\n    auto<\/span> const<\/span> lambda<\/span> =<\/span> map<\/span>.<\/span>size<\/span>()<\/span> \/<\/span> double<\/span>(<\/span>map<\/span>.<\/span>bucket_count<\/span>());<\/span>\n\n    auto<\/span> cost<\/span> =<\/span> 0.<\/span>;<\/span>\n    for<\/span> (<\/span>auto<\/span> const<\/span>&<\/span> [<\/span>k<\/span>,<\/span> _<\/span>]<\/span> :<\/span> map<\/span>)<\/span>\n        cost<\/span> +=<\/span> map<\/span>.<\/span>bucket_size<\/span>(<\/span>map<\/span>.<\/span>bucket<\/span>(<\/span>k<\/span>));<\/span>\n    cost<\/span> \/=<\/span> map<\/span>.<\/span>size<\/span>();<\/span>\n\n    return<\/span> std<\/span>::<\/span>max<\/span>(<\/span>0.<\/span>,<\/span> cost<\/span> \/<\/span> (<\/span>1<\/span> +<\/span> lambda<\/span>)<\/span> -<\/span> 1<\/span>);<\/span>\n}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
It measures the expected overhead when working with keys that are already in the map.\n0 means that your hash distribution is close to optimal.\nIn my case, the initial hash function had a badness of 550.<\/p>\n\n
Origin Story<\/h1>\n\n
Either from grad school or simply through years of osmosis, we all learned that hash maps are awesome associative containers that offer a staggering O(1) insert, delete, and lookup (albeit with some overhead).<\/p>\n\n
The design space of hash maps is quite large and depending on the use case, the trade-off space can change radically.\nstd::unordered_map<\/code> is (in)famous for having an API that basically forces implementers to use \u201cbuckets with linked lists\u201d, also known as separate chaining<\/em>.\nMany performance-critical applications swear on open addressing<\/em>, often storing keys and values directly in arrays (either together or separate).\nThese are often called flat_<\/code>maps.\nMany requirements and quality attributes influence which particular type is \u201cbest\u201d:<\/p>\n\n
Note: this post is not about perfect hashes<\/a>, which guarantee zero collisions.\nThose are also super interesting, but have different use cases.<\/p>\n\n
which results in the following assembly<\/a>:<\/p>\n\n
which results in the following assembly<\/a>:<\/p>\n\n
So, obviously foo<\/code> was not inlined.\nHowever, the compiler could still see and analyze it.\nIt came to the conclusion that foo<\/code> does not read or modify global state.\nfoo<\/code> was flagged as a pure function<\/a>.<\/p>\n\n
which results in the following assembly<\/a>:<\/p>\n\n
which results in the following assembly<\/a>:<\/p>\n\n
which results in the following assembly<\/a>:<\/p>\n\n
Conclusion<\/h1>\n\n
We saw many examples where modern C++ now naturally creates moves instead of copies or even elides them altogether and directly constructs the return value \u201cin the proper location\u201d.\nAs the last section showed, you still have to look out for references or non-local variables.\nThis is probably a good thing, because those tend to have multiple aliases, which might take offense if their data suddenly moved away.<\/p>\n\n
Most of the explanations in this post are somewhat simplified to make them palatable.\nIn particular, exceptions and volatile<\/code> variables can complicate the situation a lot.\nAlso, keep in mind that inside a lambda, captures are not considered local variables.<\/p>\n\n
The assembly shown in the examples can be considered a kind of worst case scenario, as the compiler has no access to the special member functions of T<\/code>.\nWhen these functions are visible, they can often be inlined and further optimized.<\/p>\n\n
(Title image from unsplash<\/a><\/em>)<\/p>"},{"title":"Optimization without Inlining","description":"Even without inlining, the compiler does not always has to assume the worst case.","pubDate":"Sun, 17 Oct 2021 02:00:00 +0000","link":"https:\/\/artificial-mind.net\/blog\/2021\/10\/17\/optimize-without-inline","guid":"https:\/\/artificial-mind.net\/blog\/2021\/10\/17\/optimize-without-inline","content":"
Inlining<\/a> is one of the most important compiler optimizations.\nWe can often write abstractions and thin wrapper functions without incurring any performance penalty, because the compiler will expand the method for us at call site.<\/p>\n\n
If a function is not inlined, conventional wisdom says that the compiler has to assume that the method can modify any global state and change the memory behind any pointer or reference that might have \u201cescaped\u201d.<\/p>\n\n
In this short post, I\u2019ll demonstrate exactly this effect.\nFurthermore, we will see that even if a function is not inlined, as long as the implementation is visible, some optimizations are still performed and sometimes to great effect.<\/p>\n\n
Maybe unexpectedly, the following is elided<\/a>:<\/p>\n\n
However, function parameters are already allocated by the caller and distinct from the return value.\nLuckily, the rules for return<\/code> statements still apply and we get a move in the assembly<\/a>:<\/p>\n\n
Many compilers, IDEs, and linters warn about this.\nGCC might say \u201cmoving a local object in a return statement prevents copy elision\u201d.\nAnd indeed, the assembly<\/a> is now:<\/p>\n\n
No copy elision was performed for obj<\/code> (though I am not 100% sure why. It should be allowed and possible here.)\nStill, the result is a move<\/em>, not a copy<\/em>.\nThis is a feature of the return statement<\/a>:\nSince C++11, return x<\/code> (or return (x)<\/code> or return ((x))<\/code> for that matter) will try to use the move constructor if x<\/code> is a local variable or a function parameter.<\/p>\n\n
Assembly<\/a>:<\/p>\n\n
No temporary object is created, nothing is moved or copied.\nHowever, this form of copy elision<\/a> is not mandatory.\nThis is also known as \u201cnamed return value optimization\u201d or NRVO.\nNote that in this case, in contrast to the previous case, T<\/code> must be copyable or movable, even if the actual copy or move constructor is not called in the end.<\/p>\n\n
Interestingly enough, this results in the same assembly<\/a> as our previous case:<\/p>\n\n
Assembly<\/a>:<\/p>\n
Since C++17, this invokes mandatory<\/strong> copy elision<\/a>.\nNo copy or move constructor is called, even if they have side effects.<\/p>\n\n