Keynote: Making C++ Standard Parallelism Multidimensional

Standard C++ currently exposes potential parallelism in a few different ways: explicit thread creation, the parallel algorithms, std::execution, and std::simd. We can think of natural ways to extend and improve each of these. Such efforts are worth continuing, but what is missing here is multidimensionality. This effectively excludes a wide class of algorithms that have been proven optimal in terms of parallel synchronization and memory movement. We suggest three topic areas for incorporating multidimensionality into the Standard's approaches to parallelism.

1. Locality: Constructing optimal multidimensional algorithms calls for exploiting locality multidimensionally, analogous to how std::simd does this for one dimension
2. Disjointness: Multidimensional algorithms can expose parallelism while helping users and implementations avoid complicated reasoning about whether possibly multidimensional arrays "overlap"
3. Coupling of low-level parallelism and memory operations: this leads to a preference for hiding layouts of local multidimensional arrays, and a proliferation of compile-time parameters that calls for just-in-time compilation

We will explain how Standard C++ already has the seeds of these three properties, and suggest ways to extend the Standard so that users can write portable code that performs well on a variety of hardware.

View Slides

Mark Hoemmen

Programming models architectNVIDIA

Mark Hoemmen is a programming models architect with a background in parallel computing and numerical linear algebra. He joined NVIDIA in 2022, and his current role there is to bridge between the C++ Standard and his company's parallel programming models.

Mark finished his PhD on "communication-avoiding" linear algebra algorithms in 2010. After that, he worked for ten years for Sandia National Laboratories, where he did research on communication-avoiding and fault-tolerant algorithms, and contributed to several internal and open-source scientific computing software projects. He then took a position at Stellar Science (a small Albuquerque company) for two years, focusing on C++ modernization and performance measurement, before moving to NVIDIA.

Mark's preferred programming language is C++. He has been writing it professionally for over 25 years, and has been contributing to the C++ Standard (WG21) process since 2017. Mark is main author of std::linalg, a linear algebra library that was added to C++26. He is also coauthor of several proposals relating to multidimensional arrays (such as mdspan (in C++23) and submdspan (in C++26)) and parallel algorithms.