Bug: in DinD, Since 4.0.0 postgres.get_connection_url() within another container gives broken connection url #475

Activity

1. totallyzen

   added

   🐛 bug

   📦 package: core

   on Mar 15, 2024
2. totallyzen commented on Mar 15, 2024

   

   totallyzen

   on Mar 15, 2024

   Contributor

   More actions

   Hi!

   Thanks for reporting, you are welcome to add a test case for it in the postgres module.
   We're a bit short on energy and hands to deal with everything on the list of issues and PRs.
   If you add a test case, it might be even easy to fix it - contributions are welcome! :)

   Cheers,
   B
3. totallyzen

   added

   🛟 help wanted

   on Mar 15, 2024
4. bearrito commented on Mar 15, 2024

   

   bearrito

   on Mar 15, 2024

   Contributor

   More actions

   @Tanguy-LeFloch

   How are you running that code in a container? I could have a look if I had a clean way to look at this in a debugger.

   Presumably this PR - #388 is the cause/regression. You might try setting the TC_HOST envar to your external host.
5. Tanguy-LeFloch commented on Mar 15, 2024

   

   Tanguy-LeFloch

   on Mar 15, 2024

   Last edited by Tanguy-LeFloch

   Author

   More actions

   Hello @totallyzen @bearrito thanks for the quick answers!

   On your questions:

   1. I don't mind at all adding a test, but to reproduce means adding some more setup that you don't have right now (see below).
      I could give it a shot but I'd need pointers as to 1) You mean to add it in the CI (it seems the dind tests are not in it right now) 2) You have specific prerequisites about how docker tests should be organized and which part of the codebase they should target.
      If you don't have much preferences for now I could do something sticking as much as possible to the existing.

   2. I'm running it inside a devcontainer, but it's different from yours as I'm using the external docker daemon instead of dind.
      Basically I do:

   {
       "mounts": [
           "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
       ],
   }

   Instead of:

   {
       "features": {
           "ghcr.io/devcontainers/features/docker-in-docker:2": {
   	    "version": "latest",
   	    "dockerDashComposeVersion": "v2"
   	}
       },
   }

   I was able to reproduce with this change, your devcontainer and the test code piece above.

   3. Indeed setting the TC_HOST works but I'd rather avoid setting it manually if possible as I don't hardcode this IP anywhere currently. The tests that break run in our devcontainers as dev environments as well as in our CI (with docker run ... and the mount from above).
6. bearrito commented on Mar 15, 2024

   

   bearrito

   on Mar 15, 2024

   Last edited by bearrito

   Contributor

   More actions

   I can repro this. But it looks largely intentional based of the PR and some of the related TC projects. A maintainer could comment more.

   👍React with 👍1Reacted by Tanguy-LeFloch
7. totallyzen commented on Mar 19, 2024

   

   totallyzen

   on Mar 19, 2024

   Contributor

   More actions

   Heyo!

   It is somewhat intentional, but I wouldn't say that I personally have no interest in supporting dind or devcontainers.
   I think it's important to keep improving and apologies for it breaking in the first place!
   Better to ask for forgiveness than permission in this case, so apologies!

   The main matter is energy in reverse-engineering what the original code was doing.
   It was

   • poorly understood what it was trying to do,
   • conflicting with the tc.host intent from the owners of TC
   • and wasn't covered with tests the same way as some other things out there

   it seems the dind tests are not in it right now

   Yeah the dind tests were non-existent when I arrived so didn't know what to include in the first place.
   I think at this point I'll fiddle around with restarting the notion of dind tests by simply replicating the socket mount like you are doing!
8. Tanguy-LeFloch commented on Mar 20, 2024

   

   Tanguy-LeFloch

   on Mar 20, 2024

   Author

   More actions

   Hello @totallyzen no worries, breaking things along the way happens!

   1. Just to clarify, dind and mounting the socket are two different ways of running docker from within a docker. The former uses within-docker daemon while the second uses the host docker daemon (the same that spawned the current docker). The postgres get_connection_url with dind isn't broken. With the socket mount it is.
   2. If that helps I can add tests for one of them, or even both. I was just wondering above if you already had opinions about a few things:
      1. Should they run in the CI
      2. Should they be on core / community / both / scope?

   Otherwise I can design them out-of-CI first and document how to run that.
   I'm asking that because they'll be much slower than your existing tests (assuming we don't setup any cache) and we'll need to build an image.

   Let me know what you think
9. totallyzen commented on Mar 20, 2024

   

   totallyzen

   on Mar 20, 2024

   Contributor

   More actions

   Hey!

   Thanks for clarifying, I wouldn't call myself a docker expert for sure. I did know about the two ways, but never bothered to understand the true difference. Life is short 😅

   I'm actually keen on getting the socket mount version of the testing going anyway.

   I think it's valuable for catching future regressions and I see what you mean with test slowness and I have a few tricks up my sleeve for building the image once and then using it in a follow-up matrix (may not be super effective).

   Scope is definintely only core for now, I don't want to explode the costs of the GHA. Kevin wouldn't be happy with me, haha!
   Let me handle the new addition for the matrix and I'll ask you for some review comments. :)

   I think it's important to boil down the issue to the host+port inference issue because I think this will be an issue with other containers that make use of the same inference.

   It's also worth noting there was another recent PR with #368 that should help.
   Can you run some tests with the main branch to see if that changed your situation? I think inferring the correct host network solves a lot of underlying issues.

   Correct me if I'm wrong, I'm still grasping networking issues on this matter
10. bearrito commented on Mar 20, 2024

    

    bearrito

    on Mar 20, 2024

    Contributor

    More actions

    I don't believe with Managed Runners you can run DIND in Github Actions? I know with self-hosted runners you can, but don't believe I've seen it with managed. Love to be proven wrong though.
11. Tanguy-LeFloch commented on Mar 20, 2024

    

    Tanguy-LeFloch

    on Mar 20, 2024

    Author

    More actions

    Hello,

    @totallyzen you're welcome :)

    So I just tested on main and unfortunately:

    1. The new ryuk feature from d019874 breaks completely in the setting of using the external daemon as I would expect due to the incorrect host (it breaks here https://github.com/testcontainers/testcontainers-python/blob/main/core/testcontainers/core/container.py#L205)
    2. Without ryuk the problem persists even if the port seems okay now, the host is still localhost (which is not okay for connections as we are within a container and the container we want to reach runs outside, on the host machine)

    @bearrito I'd have to test with a dind image, I'm not sure why it wouldn't work.
    However I'm sure docker run -v /var/run/docker.sock:/var/run/docker.sock ... works as it is what we are doing right now with Github managed runners.

    👍React with 👍2Reacted by Barrett Strausser and David Ankin
12. alexanderankin commented on Mar 29, 2024

    

    alexanderankin

    on Mar 29, 2024

    Member

    More actions

    sorry, reading this for the first time - not really a DinD expert, but bear with me, so to run dind, i think im going to:

    docker run --rm -it --name dind --privileged docker
    

    then i open a new terminal, and set up my python dev env?

    Details

    $ docker exec -it dind sh
    $ apk add bash python3 && bash # install python, bash shell (and launch it)
    $ python3 -m venv .pe && . .pe/bin/activate && pip install -U pip && pip install poetry && ln -s $(which poetry) /usr/bin && deactivate # install poetry in the poetry venv
    $ apk add git && git clone https://github.com/testcontainers/testcontainers-python
    $ cd testcontainers-python
    $ poetry install --with dev

    now i am running testcontainers on dind? and i can write a test like:

    cat > core/tests/test_issue_475.py <<EOF
    from testcontainers.postgres import PostgresContainer
    def test_issue_475():
        with PostgresContainer("postgres:alpine") as postgres:
            print(postgres.get_connection_url())
    EOF

    and it (poetry run pytest -s core/tests/test_issue_475.py) prints postgresql+psycopg2://test:test@localhost:32774/test

    but it should print:

    1. "dind" - the name of the container
    2. $(hostname)"? (in my case "hostname" command printed: 8a51aa4b9616)
    3. one of the ip addresses?

    ifconfig output

    8a51aa4b9616:/testcontainers-python# ifconfig
    docker0   Link encap:Ethernet  HWaddr 02:42:5A:E1:AC:CD  
              inet addr:172.18.0.1  Bcast:172.18.255.255  Mask:255.255.0.0
              UP BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:21 errors:0 dropped:0 overruns:0 frame:0
              TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:1020 (1020.0 B)  TX bytes:2122 (2.0 KiB)
    
    eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:02  
              inet addr:172.17.0.2  Bcast:172.17.255.255  Mask:255.255.0.0
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:15635 errors:0 dropped:0 overruns:0 frame:0
              TX packets:14403 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:209109459 (199.4 MiB)  TX bytes:1290642 (1.2 MiB)
    
    lo        Link encap:Local Loopback  
              inet addr:127.0.0.1  Mask:255.0.0.0
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:32 errors:0 dropped:0 overruns:0 frame:0
              TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000 
              RX bytes:2040 (1.9 KiB)  TX bytes:2040 (1.9 KiB)

    the default route's gateway ip address is what is provided in the issue, but does that really make sense

    # route | grep ^default | awk '{ print $2 }'
    172.17.0.1

    when i do that outside of docker, i get my router so i think what we actually want is the ip address of the interface which is our outbound route:

    # # inside dind container:
    # apk add iproute2 # because busybox doesn't support "brief"
    # route | grep ^default | awk '{ print $NF }' | xargs ip -brief addr show  | awk '{ print $3 }' | cut -d'/' -f1
    172.17.0.2
    
    # outside docker:
    $ route | grep ^default | awk '{ print $NF }' | xargs ip -brief addr show  | awk '{ print $3 }' | cut -d'/' -f1
    192.168.<omitted>.<omitted>

    and i thought that this brings back the terribleness of the ipv6 localhost address that macs report, but maybe not:

    $ # on macos:
    $ route -n get default | sed -n '/interface:/{s/.*: //;p;}' | xargs ipconfig getifaddr
    192.168.<omitted>.<omitted>
13. alexanderankin commented on Mar 29, 2024

    

    alexanderankin

    on Mar 29, 2024

    Member

    More actions

    interestingly enough, this one seems to give me the 172 address inside docker and a localhost address in linux, the LAN address in macos(??? - i mean 192.168), and then on windows, it gives two fe80's, an ipv6, and the LAN address also:

    python -c 'import socket; print(set([a[4][0] for a in socket.getaddrinfo(socket.gethostname(), None)]))'

    anyways, since all the other discussions were about docker abstraction layer, i figured maybe we can get somewhere with just basic networking command chat gpt prompting research
14. alexanderankin commented on Mar 29, 2024

    

    alexanderankin

    on Mar 29, 2024

    Last edited by alexanderankin

    Member

    More actions

    what im really hoping to understand is folks' dind setup - are the commands i sent realistic? i think the answer is no, somehow there are multiple containers involved.

    taking the "two scenarios" from above - docker engine in container, docker engine on host:

    1. docker engine on host - i dont see how this is distinguishable from not dind (except of compose i guess)
    2. docker engine in docker - how do you even connect to containers running side a container if you are outside the container, don't you have to set up the port forwarding ahead of time?

    are you supposed to interact with the dind container from outside of it? e.g. if you use devcontainers:

    ___________________________
    | inside dind             |        integration tests (outside)
    |                         |
    |    ___________          |
    |   | in D.E.   |         |
    |   |           |         |
    |   |  CPsql    |  CDinD--|-------------------CDev
    |   |           |         |
    |   |  CKafka   |         |
    |   |           |         |
    |   |  CValKey  |         |
    |   |___________|         |
    |                         |
    |_________________________|
    
15. 24 remaining items

    Load more

16. massdosage commented on Sep 16, 2024

    

    massdosage

    on Sep 16, 2024

    More actions

    Here are the results of running the test script.

    GitLab CI/CD pipeline:

    ### Run 
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.3 (172.17.0.3)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  '172.17.0.1' (mapped port) → gateway_ip
     ✅  '172.17.0.3' (original port) → brigde_ip
    Found in_container_id='b7768bff63105ce06e589ce2fa15d5b661968d12208afe7211c9b5834cd32250'
    Pulling image testcontainers/ryuk:0.8.1
    INFO:testcontainers.core.container:Pulling image testcontainers/ryuk:0.8.1
    Container started: f2c99af15f32
    INFO:testcontainers.core.container:Container started: f2c99af15f32
    Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: f2c99af15f32> with image testcontainers/ryuk:0.8.1 to be ready ...
    ### Run bridge
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.3 (172.17.0.3)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for bridge:
     ✅  '172.17.0.1' (mapped port) → gateway_ip
     ✅  '172.17.0.3' (original port) → brigde_ip
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.02    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    ▶️ Start Ryuk, connect to 172.17.0.1:32768
    🌈  0.01   → ✅ (('172.17.0.1', True)) ryuk ok (gateway_ip)
    ▶️ Start Ryuk, connect to 172.17.0.3:8080
    🌈  0.01   → ✅ (('172.17.0.3', False)) ryuk ok (brigde_ip)
    ▶️ Start Ryuk, connect to 172.17.0.3:32768
    🌈  6.02    🔥 (('172.17.0.3', True)) ryuk failed (brigde_ip)
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.02    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    ▶️ Start Ryuk, connect to localhost:32768
    🌈  6.02    🔥 (('localhost', True)) ryuk failed (container_host_ip, docker_host)
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.02    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    ▶️ Start Ryuk, connect to 172.17.0.1:32769
    🌈  0.01   → ✅ (('172.17.0.1', True)) ryuk ok (gateway_ip)
    ▶️ Start Ryuk, connect to 172.17.0.3:8080
    🌈  0.01   → ✅ (('172.17.0.3', False)) ryuk ok (brigde_ip)
    ▶️ Start Ryuk, connect to 172.17.0.3:32769
    🌈  6.02    🔥 (('172.17.0.3', True)) ryuk failed (brigde_ip)
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.02    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    ▶️ Start Ryuk, connect to localhost:32769
    🌈  6.02    🔥 (('localhost', True)) ryuk failed (container_host_ip, docker_host)
    

    Local developer run on a macbook:

    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.04    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: a0120b66f16c> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.1:49973
    🌈  6.06    🔥 (('172.17.0.1', True)) ryuk failed (gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.2:8080
    🌈  6.04    🔥 (('172.17.0.2', False)) ryuk failed (brigde_ip)
    
    Waiting for container <Container: a0120b66f16c> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.2:49973
    🌈  6.04    🔥 (('172.17.0.2', True)) ryuk failed (brigde_ip)
    
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.05    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    
    Waiting for container <Container: a0120b66f16c> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to localhost:49973
    🌈  0.04   → ✅ (('localhost', True)) ryuk ok (container_host_ip, docker_host)
    
    ### Run
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.2 (172.17.0.2)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  'localhost' (mapped port) → container_host_ip, docker_host
    Could not determine network running in
    
17. Mithmi commented on Sep 17, 2024

    

    Mithmi

    on Sep 17, 2024

    More actions

    Run inside my sandbox docker container (DinD)

    inside_container()=True
    os_name()='linux'
    Pulling image testcontainers/ryuk:0.8.1
    Container started: 0ac4a19a5062
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  7.02    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: 0ac4a19a5062> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.1:32773
    🌈  0.02   → ✅ (('172.17.0.1', True)) ryuk ok (gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.2:8080
    🌈  6.01    🔥 (('172.17.0.2', False)) ryuk failed (brigde_ip)
    
    Waiting for container <Container: 0ac4a19a5062> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.2:32773
    🌈  6.03    🔥 (('172.17.0.2', True)) ryuk failed (brigde_ip)
    
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.01    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    
    Waiting for container <Container: 0ac4a19a5062> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to localhost:32773
    🌈  6.02    🔥 (('localhost', True)) ryuk failed (container_host_ip, docker_host)
    
    ### Run 
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.2 (172.17.0.2)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  '172.17.0.1' (mapped port) → gateway_ip
    Could not determine network running in
    

    local just parallel dockers.

    inside_container()=False
    os_name()='linux'
    Pulling image testcontainers/ryuk:0.8.1
    Container started: c6d57140fe85
    ▶️ Start Ryuk, connect to 172.16.139.1:8080
    🌈  6.01    🔥 (('172.16.139.1', False)) ryuk failed (default_gateway_ip)
    
    Waiting for container <Container: c6d57140fe85> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.16.139.1:32782
    🌈  6.02    🔥 (('172.16.139.1', True)) ryuk failed (default_gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.02    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: c6d57140fe85> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.1:32782
    🌈  0.03   → ✅ (('172.17.0.1', True)) ryuk ok (gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.2:8080
    🌈  0.01   → ✅ (('172.17.0.2', False)) ryuk ok (brigde_ip)
    
    Waiting for container <Container: c6d57140fe85> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.2:32782
    🌈  6.01    🔥 (('172.17.0.2', True)) ryuk failed (brigde_ip)
    
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.03    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    
    Waiting for container <Container: c6d57140fe85> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to localhost:32782
    🌈  0.01   → ✅ (('localhost', True)) ryuk ok (container_host_ip, docker_host)
    
    ### Run 
     - default_gateway_ip: '172.16.139.1 (172.16.139.1)'
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.2 (172.17.0.2)'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  '172.17.0.1' (mapped port) → gateway_ip
     ✅  '172.17.0.2' (original port) → brigde_ip
     ✅  'localhost' (mapped port) → container_host_ip, docker_host
    Could not determine network running in
    
18. wlipski commented on Sep 17, 2024

    

    wlipski

    on Sep 17, 2024

    Last edited by wlipski

    More actions

    This is a run from our pipeline. We are running a Jenkins pipeline with DinD.

    The only problem for us is that we need to pass ip/host/url of a container to other containers during pipeline, so they will be able to talk. I think I've mistakenly assumed that get_container_host_ip should be used to pick ip to the container. I think in our case we should use internal ip of a container inside of a bridge network (since we don't create any other network) and that works. Example:

    kafka_docker_ip = kafka_container.get_docker_client().bridge_ip(kafka_container.get_wrapped_container().id)

    In order to run it on the pipeline I need to add --network=host for the Jenkins pipeline. So I'm no sure if I need any changes at that point, but below are the results.

    That is the run from the pipeline (docker socket is mounted):

    # --network=host
    
    inside_container()=True
    os_name()='linux'
    WARNING:root:DOCKER_AUTH_CONFIG is experimental, see testcontainers/testcontainers-python#566
    Pulling image testcontainers/ryuk:0.8.1
    INFO:testcontainers.core.container:Pulling image testcontainers/ryuk:0.8.1
    Container started: 072c200fd44d
    INFO:testcontainers.core.container:Container started: 072c200fd44d
    
    Start Ryuk, connect to 172.17.0.1:8080
     6.01    🔥(('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    Start Ryuk, connect to 172.17.0.1:32768
     0.01    ✅(('172.17.0.1', True)) ryuk ok (gateway_ip)
    
    Start Ryuk, connect to 172.17.0.2:8080
     0.01    ✅(('172.17.0.2', False)) ryuk ok (brigde_ip)
    
    Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    Start Ryuk, connect to 172.17.0.2:32768
     6.02    🔥(('172.17.0.2', True)) ryuk failed (brigde_ip)
    
    Start Ryuk, connect to localhost:8080
     6.01    🔥(('localhost', False)) ryuk failed (container_host_ip, docker_host)
    
    Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 072c200fd44d> with image testcontainers/ryuk:0.8.1 to be ready ...
    Start Ryuk, connect to localhost:32768
     0.01    ✅(('localhost', True)) ryuk ok (container_host_ip, docker_host)
    
    ### Run 
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.2 (172.17.0.2)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
       '172.17.0.1' (mapped port)  gateway_ip
       '172.17.0.2' (original port)  brigde_ip
       'localhost' (mapped port)  container_host_ip, docker_host
    Could not determine network running in
    

    While developing locally I'm just packing all the test code into the docker container and running it, so technically not DinD I think (I do run that container with mounted docker socket).

     inside_container()=True
    os_name()='linux'
    WARNING:root:DOCKER_AUTH_CONFIG is experimental, see testcontainers/testcontainers-python#566
    Pulling image testcontainers/ryuk:0.8.1
    INFO:testcontainers.core.container:Pulling image testcontainers/ryuk:0.8.1
    Container started: 30797ba38d42
    INFO:testcontainers.core.container:Container started: 30797ba38d42
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.03    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.1:60864
    🌈  6.03    🔥 (('172.17.0.1', True)) ryuk failed (gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.3:8080
    🌈  0.01   → ✅ (('172.17.0.3', False)) ryuk ok (brigde_ip)
    
    Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.3:60864
    🌈  6.03    🔥 (('172.17.0.3', True)) ryuk failed (brigde_ip)
    
    ▶️ Start Ryuk, connect to localhost:8080
    🌈  6.03    🔥 (('localhost', False)) ryuk failed (container_host_ip, docker_host)
    
    Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: 30797ba38d42> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to localhost:60864
    🌈  6.05    🔥 (('localhost', True)) ryuk failed (container_host_ip, docker_host)
    
    ### Run 
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.3 (172.17.0.3)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'localhost (127.0.0.1)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  '172.17.0.3' (original port) → brigde_ip
    Could not determine network running in
    

    #  -e TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal
    
    inside_container()=True
    os_name()='linux'
    WARNING:root:DOCKER_AUTH_CONFIG is experimental, see testcontainers/testcontainers-python#566
    Pulling image testcontainers/ryuk:0.8.1
    INFO:testcontainers.core.container:Pulling image testcontainers/ryuk:0.8.1
    Container started: b0d5d1d1beb2
    INFO:testcontainers.core.container:Container started: b0d5d1d1beb2
    ▶️ Start Ryuk, connect to 172.17.0.1:8080
    🌈  6.03    🔥 (('172.17.0.1', False)) ryuk failed (gateway_ip)
    
    Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.1:60916
    🌈  6.05    🔥 (('172.17.0.1', True)) ryuk failed (gateway_ip)
    
    ▶️ Start Ryuk, connect to 172.17.0.3:8080
    🌈  0.03   → ✅ (('172.17.0.3', False)) ryuk ok (brigde_ip)
    
    Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to 172.17.0.3:60916
    🌈  6.04    🔥 (('172.17.0.3', True)) ryuk failed (brigde_ip)
    
    ▶️ Start Ryuk, connect to host.docker.internal:8080
    🌈  0.04   → ✅ (('host.docker.internal', False)) ryuk ok (container_host_ip, docker_host)
    
    Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    INFO:testcontainers.core.waiting_utils:Waiting for container <Container: b0d5d1d1beb2> with image testcontainers/ryuk:0.8.1 to be ready ...
    ▶️ Start Ryuk, connect to host.docker.internal:60916
    🌈  0.03   → ✅ (('host.docker.internal', True)) ryuk ok (container_host_ip, docker_host)
    
    ### Run 
     - gateway_ip: '172.17.0.1 (172.17.0.1)'
     - brigde_ip: '172.17.0.3 (172.17.0.3)'
     - default_gateway_ip: '?'
     - container_host_ip, docker_host: 'host.docker.internal (192.168.65.254)'
    find_host_network=None
    Docker Client points to client_base_url='http+docker://localhost' (localhost (127.0.0.1))
    network_name='bridge'
    Successfully connections for default network:
     ✅  '172.17.0.3' (original port) → brigde_ip
     ✅  'host.docker.internal' (original port) → container_host_ip, docker_host
     ✅  'host.docker.internal' (mapped port) → container_host_ip, docker_host
    Could not determine network running in
    
19. CarliJoy commented on Oct 8, 2024

    

    CarliJoy

    on Oct 8, 2024

    Contributor

    More actions

    @wlipski your local setup seems strange. In DooD settings (mounted socket) I was always able to determine the container_id the current container was running in. For you this doesn't work.
    Could you investigate that?

    Because this seemed to be the best way to determine if we running a DinD or DooD.

    What is your local setup?
20. CarliJoy commented on Oct 11, 2024

    

    CarliJoy

    on Oct 11, 2024

    Contributor

    More actions

    I performed a summary:

    

    In green I marked the determined configuration that clustered a possible connection mode.

    To summarize:

    • If not running in a container or if the docker_host is not localhost, connect to docker_host:mapped_port.

    • Otherwise try to determine the network the current container is running in. If found use it and use bridge ip:port to connect to the container.
      This is rather a special case.

    • In any other cases use gateway_ip:mapped_port to connect to the container.

    This covers all cases besides the one of @wlipski without TESTCONTAINERS_HOST_OVERRIDE. As setting this variable fixes it, it seems to be okay. Also as this is not a CI, I don't consider it as important.

    If you see another way to determine the correct connection mode from the given settings, feel free to adopt the given
    Sheet and post you solution here.

    I will start working on a PR now.

    In my humble option the determination of the connection mode should be done already when settings are loaded with the possibility to overwrite it. I will implement it that way.
21. CarliJoy commented on Oct 11, 2024

    

    CarliJoy

    on Oct 11, 2024

    Contributor

    More actions

    @Mithmi and @wlipski it would be helpful to know how you setup you local DinD / DooD containers.
    The exact command you ran and what kind of docker you are using.
22. wlipski commented on Oct 11, 2024

    

    wlipski

    on Oct 11, 2024

    Last edited by wlipski

    More actions

    Hi @CarliJoy

    Sorry for a delay and thank you for the summary!

    Results above where archived by packing test script into a docker (assuming testc.py exists in the folder):

    # Dockerfile
    FROM python:3.10
    COPY ./testc.py /
    RUN pip install testcontainers==4.8.0
    CMD ["python3", "testc.py"]
    

    Env:

    • M1 Mac, MacOS 15.0.1
    • Docker version 27.3.1, build ce1223035a
    • Docker desktop 4.32.0

    Commands:

    docker build -f ./Dockerfile -t testc . 
    docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -t testc
    docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -e TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal -t testc
23. CarliJoy commented on Oct 11, 2024

    

    CarliJoy

    on Oct 11, 2024

    Contributor

    More actions

    Hmm okay the problem lies within Docker Desktop.

    I would suggest that at a later point in time add a detection if DockerDestkop is running somehow (let's dig into that in another issue as this issue is already getting to big [not all anwers are loaded]) and set the docker_host correctly in this case.

    That would eliminate the need for the `TESTCONTAINERS_HOST_OVERRIDE´ which a is valid workaround in the meanwhile.

    🎉React with 🎉1Reacted by Giel D
24. CarliJoy

    added 2 commits that reference this issue on Oct 11, 2024

    fix running testcontainer inside a container

    ...

    bcfb770

    fix running testcontainer inside a container

    ...

    51ec378
25. RafalSkolasinski commented on Oct 11, 2024

    

    RafalSkolasinski

    on Oct 11, 2024

    More actions

    Hmm okay the problem lies within Docker Desktop.

    FWIW, my reported results where also from using Docker Desktop. Just don't remember if I had the /var/run/docker.sock setup or did I rely fully on docker context.
26. CarliJoy

    added a commit that references this issue on Oct 18, 2024

    fix running testcontainer inside a container

    ...

    72c7f86
27. alexanderankin

    closed this as completedin #714on Oct 23, 2024
28. alexanderankin

    closed this as completedin 85a6666on Oct 23, 2024
29. alexanderankin

    added a commit that references this issue on Oct 23, 2024

    fix(core): running testcontainer inside container (#714)

    ...

    This issue will close once commit 85a6666 is merged into the 'main' branch.

    85a6666