-
Notifications
You must be signed in to change notification settings - Fork 18.5k
NCCL PyCaffe build errors #5541
Copy link
Copy link
Closed
Description
Issue summary
Building latest PyCaffe with NCCL not successful.
Steps to reproduce
CMakeCache.txt file: CMakeCache.txt
cmake -D USE_NCCL=ON ..
Output:
tranlaman@VIML2:~/Desktop/caffe-workspace/BLVC-caffe/cmake-build$ make all -j8
-- Boost version: 1.55.0
-- Found the following Boost libraries:
-- system
-- thread
-- filesystem
-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found PROTOBUF Compiler: /usr/bin/protoc
-- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
-- CUDA detected: 8.0
-- Found cuDNN: ver. 5.1.5 found (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- Added CUDA NVCC flags for: sm_61
-- Cuda + Boost 1.55: Applying noinline work around
-- Found NCCL (include: /usr/local/include, library: /usr/local/lib/libnccl.so)
-- OpenCV found (/usr/share/OpenCV)
-- Found Atlas (include: /usr/include library: /usr/lib/libatlas.so lapack: /usr/lib/liblapack.so
-- NumPy ver. 1.11.1 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
-- Boost version: 1.55.0
-- Found the following Boost libraries:
-- python
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
--
-- ******************* Caffe Configuration Summary *******************
-- General:
-- Version : 1.0.0
-- Git : 1.0-dirty
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Build type : Release
--
-- BUILD_SHARED_LIBS : ON
-- BUILD_python : ON
-- BUILD_matlab : OFF
-- BUILD_docs : ON
-- CPU_ONLY : OFF
-- USE_OPENCV : ON
-- USE_LEVELDB : ON
-- USE_LMDB : ON
-- USE_NCCL : ON
-- ALLOW_LMDB_NOLOCK : OFF
--
-- Dependencies:
-- BLAS : Yes (Atlas)
-- Boost : Yes (ver. 1.55)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 2.5.0)
-- lmdb : Yes (ver. 0.9.10)
-- LevelDB : Yes (ver. 1.15)
-- Snappy : Yes (ver. 1.1.0)
-- OpenCV : Yes (ver. 2.4.8)
-- CUDA : Yes (ver. 8.0)
--
-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_61
-- cuDNN : Yes (ver. 5.1.5)
--
-- Python:
-- Interpreter : /usr/bin/python2.7 (ver. 2.7.6)
-- Libraries : /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.6)
-- NumPy : /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.11.1)
--
-- Documentaion:
-- Doxygen : No
-- config_file :
--
-- Install:
-- Install path : /home/tranlaman/Desktop/caffe-workspace/BLVC-caffe/cmake-build/install
--
CMake Warning (dev) in src/caffe/CMakeLists.txt:
Policy CMP0022 is not set: INTERFACE_LINK_LIBRARIES defines the link
interface. Run "cmake --help-policy CMP0022" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Target "caffe" has an INTERFACE_LINK_LIBRARIES property which differs from
its LINK_INTERFACE_LIBRARIES properties.
INTERFACE_LINK_LIBRARIES:
proto;/usr/lib/x86_64-linux-gnu/libboost_system.so;/usr/lib/x86_64-linux-gnu/libboost_thread.so;/usr/lib/x86_64-linux-gnu/libboost_filesystem.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libglog.so;/usr/lib/x86_64-linux-gnu/libgflags.so;$<$<NOT:$<CONFIG:DEBUG>>:/usr/lib/x86_64-linux-gnu/libprotobuf.so>;$<$<CONFIG:DEBUG>:/usr/lib/x86_64-linux-gnu/libprotobuf.so>;-lpthread;/usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so;/usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so;/usr/lib/x86_64-linux-gnu/liblmdb.so;/usr/lib/x86_64-linux-gnu/libleveldb.so;/usr/local/cuda/lib64/libcudart.so;/usr/local/cuda/lib64/libcurand.so;/usr/local/cuda/lib64/libcublas.so;/usr/local/cuda/lib64/libcudnn.so;/usr/local/lib/libnccl.so;opencv_core;opencv_highgui;opencv_imgproc;/usr/lib/liblapack.so;/usr/lib/libcblas.so;/usr/lib/libatlas.so;/usr/lib/x86_64-linux-gnu/libboost_python.so
LINK_INTERFACE_LIBRARIES:
proto;/usr/lib/x86_64-linux-gnu/libboost_system.so;/usr/lib/x86_64-linux-gnu/libboost_thread.so;/usr/lib/x86_64-linux-gnu/libboost_filesystem.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libglog.so;/usr/lib/x86_64-linux-gnu/libgflags.so;/usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread;/usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so;/usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so;/usr/lib/x86_64-linux-gnu/liblmdb.so;/usr/lib/x86_64-linux-gnu/libleveldb.so;/usr/local/cuda/lib64/libcudart.so;/usr/local/cuda/lib64/libcurand.so;/usr/local/cuda/lib64/libcublas.so;/usr/local/cuda/lib64/libcudnn.so;/usr/local/lib/libnccl.so;opencv_core;opencv_highgui;opencv_imgproc;/usr/lib/liblapack.so;/usr/lib/libcblas.so;/usr/lib/libatlas.so;/usr/lib/x86_64-linux-gnu/libboost_python.so
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tranlaman/Desktop/caffe-workspace/BLVC-caffe/cmake-build
[ 1%] Built target proto
Scanning dependencies of target caffe
Linking CXX shared library ../../lib/libcaffe.so
[ 90%] Built target caffe
Linking CXX executable finetune_net
Linking CXX executable compute_image_mean
Linking CXX executable caffe
Linking CXX executable convert_imageset
Linking CXX executable extract_features
Linking CXX executable device_query
Linking CXX executable test_net
Linking CXX executable net_speed_benchmark
[ 91%] Built target finetune_net
[ 93%] Built target device_query
[ 93%] Built target net_speed_benchmark
Linking CXX executable upgrade_net_proto_binary
[ 93%] Linking CXX executable train_net
Built target test_net
Linking CXX executable upgrade_net_proto_text
Linking CXX executable upgrade_solver_proto_text
[ 93%] Built target train_net
Linking CXX executable cpp_classification/classification
[ 93%] Built target caffe.bin
Linking CXX executable cifar10/convert_cifar_data
[ 94%] Built target compute_image_mean
[ 95%] Built target convert_imageset
Linking CXX executable mnist/convert_mnist_data
Linking CXX executable siamese/convert_mnist_siamese_data
[ 95%] Built target extract_features
[ 97%] Building CXX object python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o
[ 98%] [ 98%] Built target upgrade_net_proto_binary
Built target upgrade_net_proto_text
[100%] Built target upgrade_solver_proto_text
[100%] Built target classification
[100%] Built target convert_cifar_data
[100%] Built target convert_mnist_siamese_data
[100%] Built target convert_mnist_data
/home/tranlaman/Desktop/caffe-workspace/BLVC-caffe/python/caffe/_caffe.cpp: In function ‘boost::python::api::object caffe::NCCL_New_Uid()’:
/home/tranlaman/Desktop/caffe-workspace/BLVC-caffe/python/caffe/_caffe.cpp:374:10: error: could not convert ‘uid’ from ‘std::string {aka std::basic_string<char>}’ to ‘boost::python::api::object’
return uid;
^
/home/tranlaman/Desktop/caffe-workspace/BLVC-caffe/python/caffe/_caffe.cpp:376:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
make[2]: *** [python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o] Error 1
make[1]: *** [python/CMakeFiles/pycaffe.dir/all] Error 2
make: *** [all] Error 2
If you are having difficulty building Caffe or training a model, please ask the caffe-users mailing list. If you are reporting a build error that seems to be due to a bug in Caffe, please attach your build configuration (either Makefile.config or CMakeCache.txt) and the output of the make (or cmake) command.
Your system configuration
Operating system: Ubuntu 14.04
Compiler: gcc484
CUDA version (if applicable): 8.0
CUDNN version (if applicable): 5.0
BLAS:
Python or MATLAB version (for pycaffe and matcaffe respectively):2.7
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels