This repository was archived by the owner on Nov 17, 2023. It is now read-only.
Multiplatform docker based builds #7792
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
please add apache license header to docker/tool.py |
Contributor
|
This pull removes dockers for perl,R,julia, can we keep them in ? |
Contributor
Author
|
@sergeykolychev I think we should have a separate set of dockerfiles for running MxNet / building MxNet. I can refine the PR tomorrow. |
larroy
force-pushed
the
edge_ci
branch
3 times, most recently
from
7366056 to
25e3b68
Compare
Contributor
Author
|
@piiswrong ping |
larroy
force-pushed
the
edge_ci
branch
2 times, most recently
from
24c92d8 to
3b61257
Compare
Contributor
Author
Contributor
|
This is not how our jenkins setup works. Builds and tests are defined in jenkins. Dockerfiles only define the environment |
Contributor
Author
|
The goal of these dockerfiles is not to replace CI but to create multi-arch binary artifacts. Also when you use the docker build with these kind of dockerfile you get the benefit of command caching. But I get your point with regards environment vs execution. |
Contributor
|
Then you can probably add these to a separate folder without removing existing code |
Contributor
|
@larroy The existing dockerfiles in the docker folder as well as the tool.sh script are being used to release mxnet docker images to dockerhub. I am in the process of automating this process and updating these scripts. |
|
Build triggered. sha1 is merged. |
|
Build started sha1 is merged. |
Contributor
Author
|
@mbaijal @piiswrong Done, separate folder. |
|
Build triggered. sha1 is merged. |
|
Build finished. |
|
Build started sha1 is merged. |
Contributor
Author
|
skip ci |
Contributor
Author
larroy
force-pushed
the
edge_ci
branch
2 times, most recently
from
f427c74 to
4599580
Compare
Contributor
Author
|
skip ci |
Contributor
|
Is this good to merge? |
Contributor
Author
|
Yes please merge it, we will refine and improve with further PRs, but this is already useful & usable for us and the community. Thank you! |
Contributor
Author
|
can we merge this and close? |
cjolivier01
pushed a commit
to cjolivier01/mxnet
that referenced
this pull request
Oct 13, 2017
* Add dockerized multi-architecture build files * Add android arm64 build
Contributor
|
@larroy when you'll be improving this please add Hash::Ordered after Function::Parameters for perl image ? |
piiswrong
pushed a commit
that referenced
this pull request
Oct 14, 2017
#8232) * GPROF update, also include include/mxnet/*.h as sources for CLionwq * Added FindGperftools.cmake * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * USE_GPERFTOOLS off by default for now * Add Apache license to FindGperftools.cmake * Update CMakeLists.txt Try to use GPerftools or JEmalloc by default * Update CMakeLists.txt Off by default for now * internal labeling * gperftools and jemalloc * gperftools and jemalloc on by default * Fixing the Caught error (#8199) * Temporarily disable some unit tests to fix the build (#8253) * Temporarily disable the following unit tests that have been causing build failures: test_rms: This can be re-enabled once #8230 is fixed. test_autograd_save_memory: This can be re-enabled once #8211 is fixed. * OMP num threads 0->1 * remove check * Update documentation links to point to mxnet.incubator.apache.org Update documentation links to point to mxnet.incubator.apache.org * add export to gluon (#8212) * add export * fix * add test * fix nnvm * fix * ReleaseFeedback: License Files (#8247) * Updating license Headers * License changes * Sequential aug (#8243) * add sequentialAug * add type for castaug * modify docs * Basic CPU Kernel OMP selection based upon whether GPU has been used (#7854) * Basic CPU Kernel OMP selection based upon whether GPU has been used * lint * Disabling the test_CSVIter for now (#7829) * Disabling the test_CSVIter for now This test causing random failure while running on windows. Disabling it for now till we fix it. An git hub issue has been created to track it. * Update test_io.py * Update test_io.py * Use OMP thread count as test in Kernel, set count for Kernel loop * lint * removed * Remove assert * Adjust DefaultOMPThreadsPerWorker * remove -1 from omp_cores * Trigger build * It is not clear why pylint claims that this is re-imported. It is not. This is not changed from master branch. Trying a different format. * lint * lint * Change getter/setter naming style * allow env override * check environment directly, since OMP_NUM_THREADS mnay have odd formatting (i.e. 3, 2"). * CR comments * Squashed commit of the following: commit ec704f1 Author: Olivier <[email protected]> Date: Mon Sep 25 12:29:25 2017 -0700 Fix formatting commit 0218c49 Author: Olivier <[email protected]> Date: Mon Sep 25 12:21:48 2017 -0700 Splitting unary ops commit 9abbba1 Author: Olivier <[email protected]> Date: Mon Sep 25 11:38:04 2017 -0700 split unary * Update mxnet_predict0.cc * Update mxnet_predict0.cc * fix oversight with bracket * Binary scatter working on CPU and GPU * return unchanged * This test case is BS. I can't even tell what's wrong on the CI build because so many errors coming from this test. * inconsequential cleanup * Update test_kvstore.py * Update CMakeLists.txt * Update CMakeLists.txt trigger build * force fail * remove forced error * test clean every make * Test * Copy Jenkinsfile from upstream/master to fix the build. * logic was reversed * Update threaded_engine.h Trigger build * Trigger rebuild * Trigger build * Trigger build * Multiplatform docker based builds (#7792) * Add dockerized multi-architecture build files * Add android arm64 build * Operators for sum(csr, axis=0) and sum(csr, axis=1) (#8174) * Add Infer storage for sparse slice operator * Remove unused files * Indentation fix and add gpu test for fallback * Change sum builtin to py_sum * Add sum_axis(csr,axis=0)=dense and sum(csr,axis=1)=dense operator * Documentation changes for sparse * Add fallback unittest for keepdims and exclude * PR review based changes : * Fix CHECK_NE * Change in_stype to int * Using const int instead of int * Initialize mid with the start * Generalizing * OMP num threads 0->1 * remove check
piiswrong
pushed a commit
to piiswrong/mxnet
that referenced
this pull request
Oct 14, 2017
apache#8232) * GPROF update, also include include/mxnet/*.h as sources for CLionwq * Added FindGperftools.cmake * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * USE_GPERFTOOLS off by default for now * Add Apache license to FindGperftools.cmake * Update CMakeLists.txt Try to use GPerftools or JEmalloc by default * Update CMakeLists.txt Off by default for now * internal labeling * gperftools and jemalloc * gperftools and jemalloc on by default * Fixing the Caught error (apache#8199) * Temporarily disable some unit tests to fix the build (apache#8253) * Temporarily disable the following unit tests that have been causing build failures: test_rms: This can be re-enabled once apache#8230 is fixed. test_autograd_save_memory: This can be re-enabled once apache#8211 is fixed. * OMP num threads 0->1 * remove check * Update documentation links to point to mxnet.incubator.apache.org Update documentation links to point to mxnet.incubator.apache.org * add export to gluon (apache#8212) * add export * fix * add test * fix nnvm * fix * ReleaseFeedback: License Files (apache#8247) * Updating license Headers * License changes * Sequential aug (apache#8243) * add sequentialAug * add type for castaug * modify docs * Basic CPU Kernel OMP selection based upon whether GPU has been used (apache#7854) * Basic CPU Kernel OMP selection based upon whether GPU has been used * lint * Disabling the test_CSVIter for now (apache#7829) * Disabling the test_CSVIter for now This test causing random failure while running on windows. Disabling it for now till we fix it. An git hub issue has been created to track it. * Update test_io.py * Update test_io.py * Use OMP thread count as test in Kernel, set count for Kernel loop * lint * removed * Remove assert * Adjust DefaultOMPThreadsPerWorker * remove -1 from omp_cores * Trigger build * It is not clear why pylint claims that this is re-imported. It is not. This is not changed from master branch. Trying a different format. * lint * lint * Change getter/setter naming style * allow env override * check environment directly, since OMP_NUM_THREADS mnay have odd formatting (i.e. 3, 2"). * CR comments * Squashed commit of the following: commit ec704f1 Author: Olivier <[email protected]> Date: Mon Sep 25 12:29:25 2017 -0700 Fix formatting commit 0218c49 Author: Olivier <[email protected]> Date: Mon Sep 25 12:21:48 2017 -0700 Splitting unary ops commit 9abbba1 Author: Olivier <[email protected]> Date: Mon Sep 25 11:38:04 2017 -0700 split unary * Update mxnet_predict0.cc * Update mxnet_predict0.cc * fix oversight with bracket * Binary scatter working on CPU and GPU * return unchanged * This test case is BS. I can't even tell what's wrong on the CI build because so many errors coming from this test. * inconsequential cleanup * Update test_kvstore.py * Update CMakeLists.txt * Update CMakeLists.txt trigger build * force fail * remove forced error * test clean every make * Test * Copy Jenkinsfile from upstream/master to fix the build. * logic was reversed * Update threaded_engine.h Trigger build * Trigger rebuild * Trigger build * Trigger build * Multiplatform docker based builds (apache#7792) * Add dockerized multi-architecture build files * Add android arm64 build * Operators for sum(csr, axis=0) and sum(csr, axis=1) (apache#8174) * Add Infer storage for sparse slice operator * Remove unused files * Indentation fix and add gpu test for fallback * Change sum builtin to py_sum * Add sum_axis(csr,axis=0)=dense and sum(csr,axis=1)=dense operator * Documentation changes for sparse * Add fallback unittest for keepdims and exclude * PR review based changes : * Fix CHECK_NE * Change in_stype to int * Using const int instead of int * Initialize mid with the start * Generalizing * OMP num threads 0->1 * remove check
piiswrong
pushed a commit
that referenced
this pull request
Oct 14, 2017
#8232) * GPROF update, also include include/mxnet/*.h as sources for CLionwq * Added FindGperftools.cmake * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * USE_GPERFTOOLS off by default for now * Add Apache license to FindGperftools.cmake * Update CMakeLists.txt Try to use GPerftools or JEmalloc by default * Update CMakeLists.txt Off by default for now * internal labeling * gperftools and jemalloc * gperftools and jemalloc on by default * Fixing the Caught error (#8199) * Temporarily disable some unit tests to fix the build (#8253) * Temporarily disable the following unit tests that have been causing build failures: test_rms: This can be re-enabled once #8230 is fixed. test_autograd_save_memory: This can be re-enabled once #8211 is fixed. * OMP num threads 0->1 * remove check * Update documentation links to point to mxnet.incubator.apache.org Update documentation links to point to mxnet.incubator.apache.org * add export to gluon (#8212) * add export * fix * add test * fix nnvm * fix * ReleaseFeedback: License Files (#8247) * Updating license Headers * License changes * Sequential aug (#8243) * add sequentialAug * add type for castaug * modify docs * Basic CPU Kernel OMP selection based upon whether GPU has been used (#7854) * Basic CPU Kernel OMP selection based upon whether GPU has been used * lint * Disabling the test_CSVIter for now (#7829) * Disabling the test_CSVIter for now This test causing random failure while running on windows. Disabling it for now till we fix it. An git hub issue has been created to track it. * Update test_io.py * Update test_io.py * Use OMP thread count as test in Kernel, set count for Kernel loop * lint * removed * Remove assert * Adjust DefaultOMPThreadsPerWorker * remove -1 from omp_cores * Trigger build * It is not clear why pylint claims that this is re-imported. It is not. This is not changed from master branch. Trying a different format. * lint * lint * Change getter/setter naming style * allow env override * check environment directly, since OMP_NUM_THREADS mnay have odd formatting (i.e. 3, 2"). * CR comments * Squashed commit of the following: commit ec704f1 Author: Olivier <[email protected]> Date: Mon Sep 25 12:29:25 2017 -0700 Fix formatting commit 0218c49 Author: Olivier <[email protected]> Date: Mon Sep 25 12:21:48 2017 -0700 Splitting unary ops commit 9abbba1 Author: Olivier <[email protected]> Date: Mon Sep 25 11:38:04 2017 -0700 split unary * Update mxnet_predict0.cc * Update mxnet_predict0.cc * fix oversight with bracket * Binary scatter working on CPU and GPU * return unchanged * This test case is BS. I can't even tell what's wrong on the CI build because so many errors coming from this test. * inconsequential cleanup * Update test_kvstore.py * Update CMakeLists.txt * Update CMakeLists.txt trigger build * force fail * remove forced error * test clean every make * Test * Copy Jenkinsfile from upstream/master to fix the build. * logic was reversed * Update threaded_engine.h Trigger build * Trigger rebuild * Trigger build * Trigger build * Multiplatform docker based builds (#7792) * Add dockerized multi-architecture build files * Add android arm64 build * Operators for sum(csr, axis=0) and sum(csr, axis=1) (#8174) * Add Infer storage for sparse slice operator * Remove unused files * Indentation fix and add gpu test for fallback * Change sum builtin to py_sum * Add sum_axis(csr,axis=0)=dense and sum(csr,axis=1)=dense operator * Documentation changes for sparse * Add fallback unittest for keepdims and exclude * PR review based changes : * Fix CHECK_NE * Change in_stype to int * Using const int instead of int * Initialize mid with the start * Generalizing * OMP num threads 0->1 * remove check
crazy-cat
pushed a commit
to crazy-cat/incubator-mxnet
that referenced
this pull request
Oct 26, 2017
* Add dockerized multi-architecture build files * Add android arm64 build
crazy-cat
pushed a commit
to crazy-cat/incubator-mxnet
that referenced
this pull request
Oct 26, 2017
apache#8232) * GPROF update, also include include/mxnet/*.h as sources for CLionwq * Added FindGperftools.cmake * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * Add option USE_GPERFTOOLS * USE_GPERFTOOLS off by default for now * Add Apache license to FindGperftools.cmake * Update CMakeLists.txt Try to use GPerftools or JEmalloc by default * Update CMakeLists.txt Off by default for now * internal labeling * gperftools and jemalloc * gperftools and jemalloc on by default * Fixing the Caught error (apache#8199) * Temporarily disable some unit tests to fix the build (apache#8253) * Temporarily disable the following unit tests that have been causing build failures: test_rms: This can be re-enabled once apache#8230 is fixed. test_autograd_save_memory: This can be re-enabled once apache#8211 is fixed. * OMP num threads 0->1 * remove check * Update documentation links to point to mxnet.incubator.apache.org Update documentation links to point to mxnet.incubator.apache.org * add export to gluon (apache#8212) * add export * fix * add test * fix nnvm * fix * ReleaseFeedback: License Files (apache#8247) * Updating license Headers * License changes * Sequential aug (apache#8243) * add sequentialAug * add type for castaug * modify docs * Basic CPU Kernel OMP selection based upon whether GPU has been used (apache#7854) * Basic CPU Kernel OMP selection based upon whether GPU has been used * lint * Disabling the test_CSVIter for now (apache#7829) * Disabling the test_CSVIter for now This test causing random failure while running on windows. Disabling it for now till we fix it. An git hub issue has been created to track it. * Update test_io.py * Update test_io.py * Use OMP thread count as test in Kernel, set count for Kernel loop * lint * removed * Remove assert * Adjust DefaultOMPThreadsPerWorker * remove -1 from omp_cores * Trigger build * It is not clear why pylint claims that this is re-imported. It is not. This is not changed from master branch. Trying a different format. * lint * lint * Change getter/setter naming style * allow env override * check environment directly, since OMP_NUM_THREADS mnay have odd formatting (i.e. 3, 2"). * CR comments * Squashed commit of the following: commit ec704f1 Author: Olivier <[email protected]> Date: Mon Sep 25 12:29:25 2017 -0700 Fix formatting commit 0218c49 Author: Olivier <[email protected]> Date: Mon Sep 25 12:21:48 2017 -0700 Splitting unary ops commit 9abbba1 Author: Olivier <[email protected]> Date: Mon Sep 25 11:38:04 2017 -0700 split unary * Update mxnet_predict0.cc * Update mxnet_predict0.cc * fix oversight with bracket * Binary scatter working on CPU and GPU * return unchanged * This test case is BS. I can't even tell what's wrong on the CI build because so many errors coming from this test. * inconsequential cleanup * Update test_kvstore.py * Update CMakeLists.txt * Update CMakeLists.txt trigger build * force fail * remove forced error * test clean every make * Test * Copy Jenkinsfile from upstream/master to fix the build. * logic was reversed * Update threaded_engine.h Trigger build * Trigger rebuild * Trigger build * Trigger build * Multiplatform docker based builds (apache#7792) * Add dockerized multi-architecture build files * Add android arm64 build * Operators for sum(csr, axis=0) and sum(csr, axis=1) (apache#8174) * Add Infer storage for sparse slice operator * Remove unused files * Indentation fix and add gpu test for fallback * Change sum builtin to py_sum * Add sum_axis(csr,axis=0)=dense and sum(csr,axis=1)=dense operator * Documentation changes for sparse * Add fallback unittest for keepdims and exclude * PR review based changes : * Fix CHECK_NE * Change in_stype to int * Using const int instead of int * Initialize mid with the start * Generalizing * OMP num threads 0->1 * remove check
|
Has anyone successfully amalgamated mxnet for Android x86-64? I need to run mxnet on an Android Emulator on Jenkins. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Set of docker files + tool to build MXNet for different architectures and collect the artifacts.