gcc8+ memory usage regression for compiling indexing_op.o #18501
Description
Description
Hi there, I try to build MXNet2.0 (only cpu) in my laptop with 16GB memory. I found that it takes over 16GB memory to compile a single file src/operator/tensor/index_op.o. I need to create extra 8GB virtual memory for building this file.
Is it possible to divide indexing_op into multiple small files to reduce the memory cost?
Environment
The latest code of MXNet 2.0
Arch Linux
Conclusion
The issue has been solved.
The cost of memory depends on the compiler and the building method (ninja or make)
I build indexing_op.o by ninja with different version of gcc.
| Compiler | The cost of memory(Child high-water RSS) |
|---|---|
| g++ 6.4.1 | 1.95 GB |
| g++ 7.4.1 | 1.78 GB |
| g++ 10.1.0 | 11 GB |
Besides, since the compiler flags is different in different building ways (for example Makefile enable -funroll-loops, it will takes more memory), the cost of memory is different.
The solution is to build MXNet with g++-6 or g++-7.