Merged
Conversation
The RawAccounting changes are to be ported on top. Signed-off-by: Kir Kolyshkin <[email protected]>
kolyshkin
changed the title
This is a port of commits adding RawHeader() to go-1.11 archive/tar. In addition: * simplify the rawBytes.Write() code in readHeader() * ignore errors from rawBytes.Write(), as (at least for go-1.11) it never returns an error, only panics (if the buffer grew too large) Also, remove the internal/testenv from tar_tar.go to enable go test. As working symlink detection is non-trivial on Windows, just skip the test on that platform. In addition to `go test`, I did some minimal manual testing, and it seems this code creates tar-data.json.gz which is identical to the one made by the old version. Signed-off-by: Kir Kolyshkin <[email protected]>
The fork of archive/tar is now from go-1.11. Signed-off-by: Kir Kolyshkin <[email protected]>
Now when golang 1.11 is out, 1.9 and older versions are no longer supported. More to say, since the archive/tar is from go-1.11, it uses some features from new Go versions (strings.Builder and sync.Map) not supported by anything older than Go 1.10. Signed-off-by: Kir Kolyshkin <[email protected]>
Contributor
Author
This was referenced Sep 6, 2018
Contributor
Author
Yes it does; see moby/moby#37771 |
Owner
|
oh cool! you beat me to it. |
Owner
|
nicely nicely [vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ go build ./cmd/tar-split/
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ head -c 8589934592 </dev/urandom >file
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ ls -lsh file
8.0G -rw-rw-r-- 1 vbatts vbatts 8.0G Sep 6 08:26 file
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ bsdtar -c --format pax -f ./test.tar ./file
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ ./tar-split d --no-stdout ./test.tar
INFO[0014] created tar-data.json.gz from ./test.tar (read 8589937152 bytes)
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ mv test.tar{,.bak}
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ ./tar-split a --input ./tar-data.json.gz --path . --output test.tar
INFO[0045] created test.tar from . and ./tar-data.json.gz (wrote 8589937152 bytes)
[vbatts@getdown] {(3a386a2...)} ~/src/github.com/vbatts/tar-split$ openssl sha512 ./test.tar{,.bak}
SHA512(./test.tar)= 6056f068c52b581dbce3944fc70584761a35b82dc567d8a80af3c5d0411a8fa815547b8b062d4d5214dd04e6897efa851d7c5d858885b7fa6f82a0830b6762da
SHA512(./test.tar.bak)= 6056f068c52b581dbce3944fc70584761a35b82dc567d8a80af3c5d0411a8fa815547b8b062d4d5214dd04e6897efa851d7c5d858885b7fa6f82a0830b6762daLGTM |
kolyshkin
added a commit
to kolyshkin/moby
that referenced
this pull request
Sep 6, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]>
kolyshkin
added a commit
to kolyshkin/moby
that referenced
this pull request
Sep 7, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 92e7543)
kolyshkin
added a commit
to kolyshkin/moby
that referenced
this pull request
Sep 7, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 92e7543)
docker-jenkins
pushed a commit
to docker-archive/docker-ce
that referenced
this pull request
Sep 7, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> Upstream-commit: 92e75439037d205c218208945c70cfb3633e87aa Component: engine
docker-jenkins
pushed a commit
to docker-archive/docker-ce
that referenced
this pull request
Sep 13, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 92e75439037d205c218208945c70cfb3633e87aa) Upstream-commit: d7085abec2e445630bedd3e79782c5ec33f62682 Component: engine
cyphar
added a commit
to SUSE/docker-ce
that referenced
this pull request
Dec 11, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> SUSE-Bugs: bsc#1118990 Signed-off-by: Aleksa Sarai <[email protected]>
cyphar
added a commit
to SUSE/docker-ce
that referenced
this pull request
Dec 11, 2018
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> SUSE-Bugs: bsc#1118990 Signed-off-by: Aleksa Sarai <[email protected]>
jfilak
pushed a commit
to jfilak/docker
that referenced
this pull request
Jan 13, 2019
To include vbatts/tar-split#48 which fixes the issue of creating an image with >8GB file in it. Signed-off-by: Kir Kolyshkin <[email protected]> --- Original commit: moby@92e7543 Signed-off-by: Jakub Filak <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a port of commits adding
RawHeader()to go-1.11 archive/tar.In addition:
rawBytes.Write()code inreadHeader()rawBytes.Write(), as (at least for go-1.11)it never returns an error, only panics (if the buffer grew too large)
Also, remove the
internal/testenvfromtar_tar.goto enablego test.As working symlink detection is non-trivial on Windows, just skip
the test on that platform.
In the absence of any tests, I did some minimal manual testing, and
it seems this code creates
tar-data.json.gzwhich is identicalto the one made by the old version.
Should fix #47
Obsoletes #35