Nice, thanks for taking the time to look into this. I think we should update doc/user/install.md too, which has the CMake options in it:

• We can just remove the DEBUG option from that list.
• If you wanted, we could remove PROFILE from that list too and mark it deprecated also. (Probably the better way to do this is just to specify cmake -DCMAKE_CXX_FLAGS="-pg" ../, and only hardcore developers are going to be doing that anyway, not users.)
• We should add a new option under the 'general configuration' section, maybe at the top, up to you:

| `-DCMAKE_BUILD_TYPE="build type"` | Specify the build configuration: `"Debug"`, `"Release"`, `"RelWithDebInfo"`, `"MinSizeRel"`.  See the [CMake documentation](https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html#variable:CMAKE_BUILD_TYPE). | `"Release"` |

Another possible discussion point... right now if you don't specify DEBUG, or you set it to OFF, we do a release build. But if you don't specify CMAKE_BUILD_TYPE="Release", then CMake will just compile without optimizations or any special CXXFLAGS.

So, maybe it makes sense to add an extra block of CMake code like:

if (NOT CMAKE_BUILD_TYPE)
  set(CMAKE_BUILD_TYPE "Release")
endif ()

That line of code is actually done first thing in the ensmallen CMake configuration.

Looks good to me. I also grepped through the other documentation to see if DEBUG or PROFILE were used in any documentation examples and just found one instance:

cmake -G "Visual Studio 16 2019" -A x64 -DBLAS_LIBRARIES:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DLAPACK_LIBRARIES:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DARMADILLO_INCLUDE_DIR="C:/mlpack/armadillo/include" -DARMADILLO_LIBRARY:FILEPATH="C:/mlpack/armadillo/build/Debug/armadillo.lib" -DDEBUG=OFF -DPROFILE=OFF ..

That's in doc/user/build_windows.md. You could just remove the -DDEBUG=OFF -DPROFILE=OFF since it will now default to a release build.

Might also be a nice thing to put in HISTORY.md too, like Use CMAKE_BUILD_TYPE to specify build type instead of DEBUG and PROFILE options..

Anyway if you can handle those two things (plus my tiny typo comment) before merge I think this is good to go.

Second approval provided automatically after 24 hours. 👍