-
Notifications
You must be signed in to change notification settings - Fork 18.5k
syncedmem.cpp:78 Check failed: error == cudaSuccess (2 vs. 0) out of memory #5353
Description
Issue summary
GPU Cuda crash when trying basic setup with train.py.
Crash occurs within a few seconds of launching python script
Machine has Nvidia with 8GB VRAM and 64GB RAM.
python train.py --solver test_solver.prototxt
....
I0303 20:08:31.785871 42944 layer_factory.cpp:58] Creating layer data
I0303 20:08:32.127293 42944 db_lmdb.cpp:40] Opened lmdb test_lmbd
I0303 20:08:32.148350 42944 net.cpp:86] Creating Layer data
I0303 20:08:32.148350 42944 net.cpp:382] data -> data
I0303 20:08:32.149355 42944 net.cpp:382] data -> label
I0303 20:08:32.149355 42944 data_transformer.cpp:25] Loading mean file from: test_mean_image.binaryproto
I0303 20:08:32.176491 42944 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
I0303 20:08:32.177494 42944 data_layer.cpp:45] output data size: 5000,3,200,200
F0303 20:08:36.003052 42944 syncedmem.cpp:78] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
Steps to reproduce
python train.py --solver test_solver.prototxt
Your system configuration
Operating system: Windows 10
Compiler: VS 2015
CUDA version (if applicable): 8.0
CUDNN version (if applicable): 5.1
BLAS: ?
Python or MATLAB version (for pycaffe and matcaffe respectively): Python 3.5.2