tests: fix _get_container_ip for docker 29.0#4606
tests: fix _get_container_ip for docker 29.0#4606georglauterbach merged 3 commits intodocker-mailserver:masterfrom
Conversation
polarathene
left a comment
There was a problem hiding this comment.
Thanks for spotting this and contributing a fix ❤️
I think we'd be fine avoiding the conditional as the information still seems to be there in older releases for some time? Since we already use jq elsewhere, might be better to adopt that here than all the extra commentary explaining the Go template syntax 😅 (I think that's what it is?)
| # use .NetworkSettings.IPAddress when available (docker < 29 and container uses bridge) | ||
| # else use .NetworkSettings.Networks.bridge.IPAddress (docker > 29 and container uses bride) | ||
| # else empty string | ||
| # with newline at the end | ||
| docker inspect --format '{{with (index .NetworkSettings "IPAddress")}}{{.}}{{else}}{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}{{end}}' "${TARGET_CONTAINER_NAME}" |
There was a problem hiding this comment.
We could do it with the --format template, but since it's JSON we might be better off with just using jq?
| # use .NetworkSettings.IPAddress when available (docker < 29 and container uses bridge) | |
| # else use .NetworkSettings.Networks.bridge.IPAddress (docker > 29 and container uses bride) | |
| # else empty string | |
| # with newline at the end | |
| docker inspect --format '{{with (index .NetworkSettings "IPAddress")}}{{.}}{{else}}{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}{{end}}' "${TARGET_CONTAINER_NAME}" | |
| docker inspect --format '{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}' "${TARGET_CONTAINER_NAME}" |
vs via jq:
| # use .NetworkSettings.IPAddress when available (docker < 29 and container uses bridge) | |
| # else use .NetworkSettings.Networks.bridge.IPAddress (docker > 29 and container uses bride) | |
| # else empty string | |
| # with newline at the end | |
| docker inspect --format '{{with (index .NetworkSettings "IPAddress")}}{{.}}{{else}}{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}{{end}}' "${TARGET_CONTAINER_NAME}" | |
| docker inspect "${TARGET_CONTAINER_NAME}" | jq '.[].NetworkSettings.Networks["bridge"].IPAddress' |
Latter looks simpler to grok to me 🤔
There was a problem hiding this comment.
well, jq could work, the comment tries actually explaining of the fallbacks:
- jq needs
-rand// ""and that would need the adequate comments for explaining the fallback (as i can understand that most people would find that hard to read). i just documented the cases i could think of were the new codes output is the same as the old code with docker < 29.docker inspect container_with_bridge | jq '.[].NetworkSettings.Networks["bridge"].IPAddress' # "172.17.0.2" docker inspect container_with_bridge | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress' # 172.17.0.2 docker inspect container_without_bridge | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress' # null docker inspect container_without_bridge | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress // ""' #
Should i also change _container_is_running to jq?
There was a problem hiding this comment.
And also introducing a pipe would mean we should check the error handling:
docker inspect unknown_container | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress // ""' || echo "FAIL"
#
#error: no such object: unknown_container
docker inspect --format '{{with (index .NetworkSettings "IPAddress")}}{{.}}{{else}}{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}{{end}}' unknown_container || echo "FAIL"
#
#error: no such object: unknown_container
#FAILThere was a problem hiding this comment.
Regarding the jq fallback to empty with // "", while the reasoning is good our tests don't seem to use such a fallback when encountering a failure.
# With blank line included (stdout):
$ docker inspect --format '{{ .NetworkSettings.IPAddress }}'
Error: No such object: unknown_containerI'm not sure why your output example differs there with all lowercase 🤔 (Oh, just compared to v29+ where it is lowercase, prior to that it was as my output shows)
that would need the adequate comments for explaining the fallback (as i can understand that most people would find that hard to read).
I'd agree if we had the jq fallback you suggest it would need clarification.
From the looks of it the blank value to stdout probably isn't meaningful to our tests (fail2ban.bats + postscreen.bats that use this utility method), they'd still fail as your shared error output demonstrated with stderr being reported instead.
And also introducing a pipe would mean we should check the error handling:
set -o pipefail will handle that:
$ set -o pipefail
$ docker inspect unknown_container | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress // ""' || echo "FAIL"
error: no such object: unknown_container
FAIL
$ echo $?
1That said there's quite a bit of discouragement online for using pipefail broadly/globally, so that's a bit of a bummer.
jq does have an option to output an exit code for invalid input though:
-e/--exit-status: Sets the exit status ofjqto
0if the last output values was neitherfalsenornull1if the last output value was eitherfalseornull4if no valid result was ever produced.
$ docker inspect unknown_container | jq -r --exit-status '.[].NetworkSettings.Networks["bridge"].IPAddress'
error: no such object: unknown_container
$ echo $?
4Should i also change
_container_is_running()tojq?
I don't think that's necessary other than for consistency. My main gripe was the template syntax for docker inspect --format is less intuitive vs the JQ equivalent:
docker inspect --format '{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}' "${TARGET_CONTAINER_NAME}"vs
docker inspect "${TARGET_CONTAINER_NAME}" | jq -r --exit-status '.[].NetworkSettings.Networks["bridge"].IPAddress'At least regarding the response is an array of objects, with some nested field access. In this case with jq I've added a bit of extra complexity for the bridge key since it won't work well if there's a hypen in the network name should this change in future (I assume that might be a similar case with your template equivalent syntax having a simpler form?), the jq query could just as easily be .[].NetworkSettings.Networks.bridge.IPAddress.
_container_is_running() is simple enough to grok:
docker-mailserver/test/helper/common.bash
Lines 377 to 380 in b410e56
vs
[[ $(docker inspect "${TARGET_CONTAINER_NAME}" | jq -re '.[].State.Running') == 'true' ]]There was a problem hiding this comment.
After reading through the changes and this discussion, I am in favor of not using jq but keeping the changes as they are. They are mostly similar in quality IMHO but what jq gains in terms of lower complexity for its query it loses IMO in terms of error handling and cases with "" etc.
I think the query that this PR introduces is easy enough to understand.
There was a problem hiding this comment.
what
jqgains in terms of lower complexity for its query it loses IMO in terms of error handling and cases with""etc.
There is no need to emit a blank output, the default null is fine since the error still can be output via --exit-status (causing the test to fail, while the earlier docker inspect stderr should be captured for the error message).
I am in favor of not using
jqbut keeping the changes as they are.
I disagree with "changes as they are", but I'll compromise.
We do not need the much longer docker inspect proposed here with two branches based on field detection when the latter is sufficient.
There was a problem hiding this comment.
I personally prefer the jq approach with:
docker inspect "${TARGET_CONTAINER_NAME}" | jq -r --exit-status '.[].NetworkSettings.Networks["bridge"].IPAddress'I do appreciate the concerns and effort towards replicating the existing behaviour prior to Docker v29 though 👍
I think for the most part it's largely myself and @georglauterbach that work on the test suite, so he may want to weigh in here too, otherwise I think jq approach is preferred unless I've missed addressing any other concern?
| # use .NetworkSettings.IPAddress when available (docker < 29 and container uses bridge) | ||
| # else use .NetworkSettings.Networks.bridge.IPAddress (docker > 29 and container uses bride) | ||
| # else empty string | ||
| # with newline at the end | ||
| docker inspect --format '{{with (index .NetworkSettings "IPAddress")}}{{.}}{{else}}{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}{{end}}' "${TARGET_CONTAINER_NAME}" |
There was a problem hiding this comment.
Regarding the jq fallback to empty with // "", while the reasoning is good our tests don't seem to use such a fallback when encountering a failure.
# With blank line included (stdout):
$ docker inspect --format '{{ .NetworkSettings.IPAddress }}'
Error: No such object: unknown_containerI'm not sure why your output example differs there with all lowercase 🤔 (Oh, just compared to v29+ where it is lowercase, prior to that it was as my output shows)
that would need the adequate comments for explaining the fallback (as i can understand that most people would find that hard to read).
I'd agree if we had the jq fallback you suggest it would need clarification.
From the looks of it the blank value to stdout probably isn't meaningful to our tests (fail2ban.bats + postscreen.bats that use this utility method), they'd still fail as your shared error output demonstrated with stderr being reported instead.
And also introducing a pipe would mean we should check the error handling:
set -o pipefail will handle that:
$ set -o pipefail
$ docker inspect unknown_container | jq -r '.[].NetworkSettings.Networks["bridge"].IPAddress // ""' || echo "FAIL"
error: no such object: unknown_container
FAIL
$ echo $?
1That said there's quite a bit of discouragement online for using pipefail broadly/globally, so that's a bit of a bummer.
jq does have an option to output an exit code for invalid input though:
-e/--exit-status: Sets the exit status ofjqto
0if the last output values was neitherfalsenornull1if the last output value was eitherfalseornull4if no valid result was ever produced.
$ docker inspect unknown_container | jq -r --exit-status '.[].NetworkSettings.Networks["bridge"].IPAddress'
error: no such object: unknown_container
$ echo $?
4Should i also change
_container_is_running()tojq?
I don't think that's necessary other than for consistency. My main gripe was the template syntax for docker inspect --format is less intuitive vs the JQ equivalent:
docker inspect --format '{{with (index .NetworkSettings.Networks "bridge")}}{{.IPAddress}}{{end}}' "${TARGET_CONTAINER_NAME}"vs
docker inspect "${TARGET_CONTAINER_NAME}" | jq -r --exit-status '.[].NetworkSettings.Networks["bridge"].IPAddress'At least regarding the response is an array of objects, with some nested field access. In this case with jq I've added a bit of extra complexity for the bridge key since it won't work well if there's a hypen in the network name should this change in future (I assume that might be a similar case with your template equivalent syntax having a simpler form?), the jq query could just as easily be .[].NetworkSettings.Networks.bridge.IPAddress.
_container_is_running() is simple enough to grok:
docker-mailserver/test/helper/common.bash
Lines 377 to 380 in b410e56
vs
[[ $(docker inspect "${TARGET_CONTAINER_NAME}" | jq -re '.[].State.Running') == 'true' ]]|
Thank you for spottend this @ap-wtioit! It seems like the latest changes with Docker cause havoc wherever I look.. I'll also review this today, please wait with changes (jq vs no jq) until I took a loo :) |
georglauterbach
left a comment
There was a problem hiding this comment.
These changes work for me. But I'll leave it up to @polarathene to decide whether jq is really necessary.
Description
Docker 29 removes NetworkSettings.IPAddress
Docker 28.4.0 inspect of container using bridge
docker inspect ...Docker 29.0.0 inspect of container using bridge
docker inspect ...Test error on our CI when having a docker 29.0.0 runner:
Type of change
Checklist
docs/)CHANGELOG.mdInfo @wt-io-it