Skip to content

Revert internal resolver for default bridge#48020

Merged
robmry merged 2 commits intomoby:masterfrom
robmry:revert_internal_resolver_for_default_bridge
Jun 18, 2024
Merged

Revert internal resolver for default bridge#48020
robmry merged 2 commits intomoby:masterfrom
robmry:revert_internal_resolver_for_default_bridge

Conversation

@robmry
Copy link
Contributor

@robmry robmry commented Jun 17, 2024

- What I did

Because buildkit doesn't run an internal resolver, and it bases its /etc/resolv.conf on the host's ... when buildkit is run in a container that has 'nameserver 127.0.0.11', its build containers will use Google's DNS servers as a fallback (unless the build container uses host networking).

Before, when the 127.0.0.11 resolver was not used for the default network, the buildkit container would have inherited a site-local nameserver. So, the build containers it created would also have inherited that DNS server - and they'd be able to resolve site-local hostnames.

By replacing the site-local nameserver with Google's, we broke access to local DNS and its hostnames.

Note that buildkit's build containers currently use host networking, although that may change. But, until it does, the only problem is that (without --network=host) buildkit replaces localhost resolvers with the Google defaults. If it didn't do that, and kept 127.0.0.11, it'd work as expected.

- How I did it

- How to verify it

- Description for the changelog

n/a - but check the notes for https://github.com/moby/moby/pull/47602 have been removed

@robmry robmry added kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. area/networking Networking area/networking/dns Networking labels Jun 17, 2024
@robmry robmry added this to the 28.0.0 milestone Jun 17, 2024
@robmry robmry self-assigned this Jun 17, 2024
robmry added 2 commits June 17, 2024 20:19
This reverts commit d365702.

Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).

Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.

By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.

Signed-off-by: Rob Murray <[email protected]>
This reverts commit 18f4f77.

Because buildkit doesn't run an internal resolver, and it bases its
/etc/resolv.conf on the host's ... when buildkit is run in a container
that has 'nameserver 127.0.0.11', its build containers will use Google's
DNS servers as a fallback (unless the build container uses host
networking).

Before, when the 127.0.0.11 resolver was not used for the default network,
the buildkit container would have inherited a site-local nameserver. So,
the build containers it created would also have inherited that DNS
server - and they'd be able to resolve site-local hostnames.

By replacing the site-local nameserver with Google's, we broke access
to local DNS and its hostnames.

Signed-off-by: Rob Murray <[email protected]>
@robmry robmry force-pushed the revert_internal_resolver_for_default_bridge branch from 0864768 to 74d77d8 Compare June 17, 2024 19:19
@robmry robmry modified the milestones: 28.0.0, 27.0.0 Jun 17, 2024
@robmry robmry marked this pull request as ready for review June 17, 2024 19:43
@corhere
Copy link
Contributor

corhere commented Jun 17, 2024

By replacing the site-local nameserver with Google's, we broke access to local DNS and its hostnames.

Note that buildkit's build containers currently use host networking, although that may change. But, until it does, the only problem is that (without --network=host) buildkit replaces localhost resolvers with the Google defaults. If it didn't do that, and kept 127.0.0.11, it'd work as expected.

Buildkit is replacing the site-local DNS server with Google's, not us. If that's the only root cause, why can't it be fixed in Buildkit by having Buildkit stop shooting itself in the foot?

@robmry
Copy link
Contributor Author

robmry commented Jun 17, 2024

Buildkit is replacing the site-local DNS server with Google's, not us. If that's the only root cause, why can't it be fixed in Buildkit by having Buildkit stop shooting itself in the foot?

I think it could be ... if our understanding of the reported issue is right, and we haven't missed anything else.

As 27.0 will ship tomorrow, the sensible option is reverting - to give ourselves some thinking time, and to deal with buildkit's DNS properly (hopefully getting rid of the Google fallback once and for all).

@corhere
Copy link
Contributor

corhere commented Jun 17, 2024

According to moby/buildkit#4524 (comment) there is no bug. Buildkit is behaving as intended, apparently: when the network mode for the build is unspecified, containers are only guaranteed to have public internet access.

@corhere
Copy link
Contributor

corhere commented Jun 17, 2024

It also sounds like there is a viable workaround of projecting /run/systemd/resolve/resolv.conf into the buildkit container. moby/buildkit#5009

Copy link
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking for more discussion. I want to hear from @tonistiigi

@tonistiigi
Copy link
Member

@corhere The issue is not (only) related to running buildkit container with host networking. If extra mounts should be added to container when the user specifies buildx create --driver-opt network=host can be discussed separately and potentially added to next version of buildx if desirable.

The main case seems to be that all current buildkit releases make a distinction if RUN --network=default containers get the host-based resolv.conf or default resolv.conf based on whether the DNS is only detected to have localhost DNS addresses. This changes so that all containers have localhost DNS (as detected by libnetwork/resolveconf) where they previously had not. This changes the resolv.conf redirection condition and potentially breaks configuration where the default 8.8.8.8 DNS does not work.

@robmry robmry merged commit 9e389b1 into moby:master Jun 18, 2024
@robmry robmry deleted the revert_internal_resolver_for_default_bridge branch June 18, 2024 08:05
@robmry robmry added kind/bugfix PR's that fix bugs and removed kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. labels Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/networking/dns Networking area/networking Networking kind/bugfix PR's that fix bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docker 27.0.0~rc.1 does not use host dns

4 participants