This repository was archived by the owner on Nov 17, 2023. It is now read-only.
RowSparse pull/push #7015
Closed
RowSparse pull/push #7015
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Same test failure happened here. It is using the source code of this PR #6955. Such a strange behavior... |
eric-haibin-lin
force-pushed
the
pull-rsp-rebase
branch
from
135e755 to
a0c786e
Compare
eric-haibin-lin
force-pushed
the
pull-rsp-rebase
branch
from
285b383 to
0e87159
Compare
szha
reviewed
Jul 21, 2017
| zippath = os.path.join(data_dir, data_origin_name) | ||
| urllib.urlretrieve(url, zippath) | ||
| os.system("bzip2 -d %r" % data_origin_name) | ||
| os.chdir("..") |
Member
There was a problem hiding this comment.
Consider using test_utils.download
Member
Author
fix bug in rsp add rsp sync push race condition for push fix bug in rsp pull. refactor test cleanup comments refactor dist server fix lint fix storage shape issue with the new ndarray constructor data sharding draft; fix lint. add comment add support for zeros gradients use std::upper_bound/lower_bound remove special init function for rowsparse dist kvstore temporary support for inplace operators for sparse add test. fix return type store kRowSparseNDArray in kv server remove fcomp_ex sgd with dns weight and rsp gradient bug fix in sparse retain sparse pull c_api revise rowsparse pull api use engine to compute unique to ensure thread safety add rowsparse pull to dist-kv fix lint add example for rsp_pull remove name2idx; add sparse_pull_dict param to module fix unit test and c rowid conversion
* update kvstore unit test * update model/module.py * fix lint * remove int keys in kvstore * update cast to str function * remove _cast_to_str_keys * fix lint * always cast to str Conflicts: include/mxnet/c_api.h include/mxnet/kvstore.h python/mxnet/kvstore.py python/mxnet/model.py python/mxnet/module/module.py src/c_api/c_api.cc src/kvstore/kvstore_local.h tests/python/unittest/test_kvstore.py
update stypes in kvstore after refactoring change type of size from size_t to int64_t add sparse linear regression example remove sparse_pull_dict from module fix init_optim for seq_module. update sparse example resolve conflict for binary add rsp rsp
eric-haibin-lin
force-pushed
the
pull-rsp-rebase
branch
2 times, most recently
from
60cac0b to
1c3895a
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Opening this PR for interface review:
row_sparse_pullpython API to kvstore and c_api, which requires providingrow_idfor each row_sparsevalueto pull. Therow_ids are sorted in backend to contain only unique elements. The oldpullAPI will not accept pulling avaluestored in row_sparse storagekRowSparsePushPulltype to kvstore server. When received data request of this type, keys are encoded askey, row_id1, row_id2 ... row_idn.row_sparse_pullin regression_example.pyNote:
TODOs for next PR:
@mli @reminisce @piiswrong @madjam @stefanhenneking