Skip to content

Conv2d is missing CUDA/CPU type checks #1607

New issue
New issue
Closed
Closed
Conv2d is missing CUDA/CPU type checks#1607
Labels
awaiting response (this tag is deprecated)This tag is deprecated while we figure out what to do with it
@zklgame

Description

@zklgame
zklgame
opened on May 22, 2017

When I run the GPU version of pytorch, some error occurs, and kill the ipython notebook kernel.
some code is listed as bellow:

dtype = torch.cuda.FloatTensor
cnn = torchvision.models.squeezenet1_1(pretrained=True).features
cnn.type(dtype)

print type(prev_feat)
print len(cnn._modules.values())
for i, module in enumerate(cnn._modules.values()):
        print i, module
        next_feat = module(prev_feat)
        print 'done module'
        features.append(next_feat)
        prev_feat = next_feat
print 'done all'

output is:

<class 'torch.autograd.variable.Variable'>
13
0 Conv2d(3, 64, kernel_size=(3, 3), stride=(2, 2))

Then, ipython notebook kernel stoped, infomation(too long to list them all):

*** Error in `/home/zkl/anaconda2/bin/python': free(): invalid pointer: 0x00007f790d2deae0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f79417f37e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7f79417fbe0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f79417ff98c]
/home/zkl/anaconda2/lib/python2.7/site-packages/zmq/backend/cython/../../../../.././libstdc++.so.6(_ZNSt15basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x13b)[0x7f793ab09f9b]
/home/zkl/anaconda2/lib/python2.7/site-packages/zmq/backend/cython/../../../../.././libstdc++.so.6(_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+0x36)[0x7f793ab0e106]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/lib/libshm.so(_ZSt16__ostream_insertIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_PKS3_l+0x1c5)[0x7f790d053235]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(+0x5d2842)[0x7f790d8d4842]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(+0x5d34ae)[0x7f790d8d54ae]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(_ZN5torch2nn33SpatialConvolutionMM_updateOutputEPN4thpp6TensorES3_S3_S3_S3_S3_iiiiii+0xb3)[0x7f790d8e91a3]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(+0x5caf27)[0x7f790d8ccf27]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(_ZN5torch8autograd11ConvForward5applyERKSt6vectorISt10shared_ptrINS0_8VariableEESaIS5_EE+0x17bf)[0x7f790d8d165f]
/home/zkl/anaconda2/lib/python2.7/site-packages/torch/_C.so(+0x5c191b)[0x7f790d8c391b]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x715d)[0x7f794256e80d]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79b68)[0x7f79424ebb68]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x61d6)[0x7f794256d886]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79a61)[0x7f79424eba61]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x5c64f)[0x7f79424ce64f]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0xba2ac)[0x7f794252c2ac]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x715d)[0x7f794256e80d]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8c95)[0x7f7942570345]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8c95)[0x7f7942570345]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f7942570d52]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8158)[0x7f794256f808]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79b68)[0x7f79424ebb68]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x61d6)[0x7f794256d886]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79b68)[0x7f79424ebb68]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x61d6)[0x7f794256d886]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79b68)[0x7f79424ebb68]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f79424bbe93]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x61d6)[0x7f794256d886]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8b47)[0x7f79425701f7]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e)[0x7f7942570c3e]
/home/zkl/anaconda2/bin/../lib/libpython2.7.so.1.0(+0x79b68)[0x7f79424ebb68]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:04 4853203                            /home/zkl/anaconda2/bin/python2.7
00600000-00601000 rw-p 00000000 08:04 4853203                            /home/zkl/anaconda2/bin/python2.7
012c3000-3bc9d000 rw-p 00000000 00:00 0                                  [heap]
200000000-200100000 rw-s 1cc537000 00:06 575                             /dev/nvidiactl
200100000-200104000 rw-s 1cc63a000 00:06 575                             /dev/nvidiactl
200104000-200120000 ---p 00000000 00:00 0 
200120000-200520000 rw-s 1cc646000 00:06 575                             /dev/nvidiactl
200520000-200524000 rw-s 8f058000 00:06 575                              /dev/nvidiactl
200524000-200540000 ---p 00000000 00:00 0 
200540000-200940000 rw-s 8f064000 00:06 575                              /dev/nvidiactl
200940000-200a40000 rw-s 8e4ba000 00:06 575                              /dev/nvidiactl
200a40000-200b40000 rw-s 8e5bd000 00:06 575                              /dev/nvidiactl
200b40000-200c40000 rw-s 1cb61e000 00:06 575                             /dev/nvidiactl
200c40000-200d40000 rw-s 1cb721000 00:06 575                             /dev/nvidiactl
200d40000-200e40000 rw-s 1cb024000 00:06 575                             /dev/nvidiactl
200e40000-200ec2000 rw-s 1cb127000 00:06 575                             /dev/nvidiactl
200ec2000-200ee0000 ---p 00000000 00:00 0 
200ee0000-200fe0000 rw-s 00000000 00:05 22857                            /dev/zero (deleted)
200fe0000-700000000 ---p 00000000 00:00 0 
7f78b8000000-7f78b8021000 rw-p 00000000 00:00 0 
7f78b8021000-7f78bc000000 ---p 00000000 00:00 0 
7f78bc000000-7f78bc021000 rw-p 00000000 00:00 0 
7f78bc021000-7f78c0000000 ---p 00000000 00:00 0 
7f78c0000000-7f78c0021000 rw-p 00000000 00:00 0 
7f78c0021000-7f78c4000000 ---p 00000000 00:00 0 
7f78c8000000-7f78c8021000 rw-p 00000000 00:00 0 
7f78c8021000-7f78cc000000 ---p 00000000 00:00 0 
7f78cc000000-7f78cc021000 rw-p 00000000 00:00 0 
7f78cc021000-7f78d0000000 ---p 00000000 00:00 0 
7f78d0b34000-7f78d15f9000 rw-p 00000000 00:00 0 
7f78d20be000-7f78d2992000 rw-p 00000000 00:00 0 
7f78d2a64000-7f78d2a65000 ---p 00000000 00:00 0 
7f78d2a65000-7f78d2e65000 rw-p 00000000 00:00 0 
7f78d2e65000-7f78d2e66000 ---p 00000000 00:00 0 
7f78d2e66000-7f78d3933000 rw-p 00000000 00:00 0 
7f78d3bff000-7f78d3c00000 ---p 00000000 00:00 0 
7f78d3c00000-7f78d4000000 rw-p 00000000 00:00 0 
7f78d4000000-7f78d4021000 rw-p 00000000 00:00 0 
7f78d4021000-7f78d8000000 ---p 00000000 00:00 0 
7f78d821b000-7f78d8568000 rw-p 00000000 00:00 0 
7f78d88b5000-7f78d8bff000 rw-p 00000000 00:00 0 
7f78d8c3c000-7f78d8d39000 r-xp 00000000 08:04 5642187                    /home/zkl/anaconda2/lib/python2.7/site-packages/torch/_thnn/_THCUNN.so
7f78d8d39000-7f78d8f38000 ---p 000fd000 08:04 5642187                    /home/zkl/anaconda2/lib/python2.7/site-packages/torch/_thnn/_THCUNN.so
7f78d8f38000-7f78d8f3d000 rw-p 000fc000 08:04 5642187                    /home/zkl/anaconda2/lib/python2.7/site-packages/torch/_thnn/_THCUNN.so
7f78d8f3d000-7f78d8f49000 rw-p 0035f000 08:04 5642187                    /home/zkl/anaconda2/lib/python2.7/site-packages/torch/_thnn/_THCUNN.so
7f78d8f49000-7f78d9199000 rw-p 00000000 00:00 0 
7f78d930e000-7f78d940e000 rw-p 00000000 00:00 0 
7f78d940e000-7f78d9450000 r-xp 00000000 08:04 7605248                    /usr/lib/nvidia-375/libnvidia-fatbinaryloader.so.375.39
7f78d9450000-7f78d964f000 ---p 00042000 08:04 7605248                    /usr/lib/nvidia-375/libnvidia-fatbinaryloader.so.375.39
7f78d964f000-7f78d9659000 rw-p 00041000 08:04 7605248                    /usr/lib/nvidia-375/libnvidia-fatbinaryloader.so.375.39
7f78d9659000-7f78d965a000 rw-p 00000000 00:00 0 
7f78d965a000-7f78d9d28000 r-xp 00000000 08:04 400258                     /usr/lib/x86_64-linux-gnu/libcuda.so.375.39
7f78d9d28000-7f78d9f27000 ---p 006ce000 08:04 400258                     /usr/lib/x86_64-linux-gnu/libcuda.so.375.39
7f78d9f27000-7f78da043000 rw-p 006cd000 08:04 400258                     /usr/lib/x86_64-linux-gnu/libcuda.so.375.39
7f78da043000-7f78da04f000 rw-p 00000000 00:00 0 
7f78da04f000-7f78da059000 r-xp 00000000 08:04 5507160                    /home/zkl/anaconda2/lib/python2.7/site-packages/scipy/optimize/_nnls.so
7f78da059000-7f78da259000 ---p 0000a000 08:04 5507160                    /home/zkl/anaconda2/lib/python2.7/site-packages/scipy/optimize/_nnls.so

if I change the dtype to CPU version, everything is OK:

dtype = torch.FloatTensor

Well, I found some reason:
If I change the prev_feat to .cuda() version, then everything works well. Now I really do not want the kernel to be crashed when this kind of mistake happened, can anyone help? Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    awaiting response (this tag is deprecated)This tag is deprecated while we figure out what to do with it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions