Skip to content

https authentication support#3068

Merged
vieux merged 1 commit intomoby:masterfrom
discordianfish:https_client
Mar 19, 2014
Merged

https authentication support#3068
vieux merged 1 commit intomoby:masterfrom
discordianfish:https_client

Conversation

@discordianfish
Copy link
Contributor

This addeds two-way https authentication. If the flag -tlscacert is used, docker client will authenticate the server based on this. CA as well as docker daemon will authenticate clients based on this CA.

This closes #1745

This does not support unauthenticated HTTPS because I don't really see the need and it makes the flag/user interface much harder to understand. So -tlscacert is what enables https and docker will accept remotes only if signed by that CA.

This PR also includes a basic https integration test. I'm happy to add more test cases (at least some invalid certificate for both server and client), but since I struggled hard with the tests and I'm not very happy with them, I would like to get some input on what I have so far.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I would prefer to avoid "includes". Normally I would suggest to use a link instead, but in this case, I think these flags should be included in the list of flags at the top of the file (under daemon).

@shykes
Copy link
Contributor

shykes commented Dec 6, 2013

Hey @discordianfish, the current preferred path for authentication is to use S3's hmac request signature scheme.

I'm not necessarily against also supporting certs, but they will be a 2nd-class citizen once hmac is introduced. Which by the way we're looking for an owner to implement that :)

Do you see any way having both hmac and client certs would be a problem?

@discordianfish
Copy link
Contributor Author

@shykes For HMAC would still need (non client authenticated) https right? Because we care not only about integrity but also secrecy. Then introduce hmac in another PR?

No problem, I can add a flag to not authenticate the client. But I still think tls client auth is a very useful feature. It has the great advantage, that you get user management for free. Every client can has it's own keys, they can be revoked and the ca keys can be stored safely somewhere completely else. To achieve the same with hmac, docker needs to know each user and their keys and if you have several docker nodes, you need to distribute those.

@jpetazzo Since you came up with the idea of using tls client certs for authenticate, maybe you want to chip in as well?

@bodenr
Copy link

bodenr commented Dec 6, 2013

+1 for this support, but a quick clarification.... @discordianfish -- so this client side SSL support would extend to all aspects of the docker client CLI correct? i.e. using https with docker push <private_ssl_repo_tag>?

please see my comment at the bottom of #2996

thanks

@discordianfish
Copy link
Contributor Author

@bodenr: This is for authenticating client <-> daemon connctions, not daemon <-> registry. But there is PR for that here: #3070

@discordianfish
Copy link
Contributor Author

@shykes What do you think about that? Now all TLS modes are possible (and the interface a bit more complex), but I think that's fine.

@discordianfish
Copy link
Contributor Author

I would still like to get some input on the tests. Does it make sense to spawn up the https daemon like this? Or is there a better/easier way? I will need to commit some more keys and certs for testing other test cases, is the fixtures directory I create fine like that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add:

.. warning::

  As shown in the example above, you don't have to run the client
  ``docker`` with ``sudo`` or the ``docker`` group when you use
  certificate authentication. That means anyone with the certificates
  can give any instructions to your Docker daemon, giving them root
  access to the machine hosting the daemon. Guard these certificates
  as you would a root password!

@discordianfish
Copy link
Contributor Author

@metalivedev I've added it but changed certificates to keys, because the certificates are public data. Hope that's okay!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, docker runs via a non-networked unix socket. it can also optionally communicate using a http socket.

(or something like that?)

@discordianfish
Copy link
Contributor Author

@SvenDowideit Thanks! I've fixed that. I also removed that 'sudo docker -d' example, which doesn't make much sense imo.

@SvenDowideit
Copy link
Contributor

I really wonder if we can start adding default values for parameters to the cfg file - but that should probably go in a separate issue :)

@discordianfish
Copy link
Contributor Author

@SvenDowideit What do you mean by config file? The auth conf or is there another config file I'm not aware of? But I agree, the more complex the configuration gets, the more reasonable would be using a config file.

@SvenDowideit
Copy link
Contributor

Heya @discordianfish for merging, you'll be asked to squash the commits into one

documentation LGTM

@shykes
Copy link
Contributor

shykes commented Dec 11, 2013

Actually we changed to allow multiple commits as long as they map discrete logical changes. It makes merges easier.

I asked @discordianfish to make a case on docker-dev in favor of client certs as tge primary auth method. I think it's a viable option, I just have practical questions.

On Tue, Dec 10, 2013 at 9:32 PM, Sven Dowideit [email protected]
wrote:

Heya @discordianfish for merging, you'll be asked to squash the commits into one

documentation LGTM

Reply to this email directly or view it on GitHub:
#3068 (comment)

@SvenDowideit SvenDowideit mentioned this pull request Dec 14, 2013
@creack
Copy link
Contributor

creack commented Jan 3, 2014

@discordianfish Can you rebase? as said in #3070, tentatively scheduling this for 0.7.4

@discordianfish
Copy link
Contributor Author

@creack Oh cool! It's rebased now!

@shykes
Copy link
Contributor

shykes commented Jan 3, 2014

Don't mean to rain on the parade @creack but you don't get to decide this on your own. You just risk disappointing everyone if it doesn't actually get merged.

@metalivedev
Copy link
Contributor

docs LGTM (thanks for the update!)

@vieux
Copy link
Contributor

vieux commented Jan 10, 2014

@discordianfish I always launch docker using:
sudo -E docker -d -H unix:///var/run/docker.sock -H tcp://0.0.0.0:4243

With your PR, doing a make gives me

$>make
docker build -rm -t "docker:discordianfish-https_client" .
Uploading context 111.7 MB
Uploading context
# Skipping unknown instruction DOCKER-VERSION
Step 1 : FROM stackbrew/ubuntu:12.04
2014/01/10 11:59:11 use of closed network connection
make: *** [build] Error 1

@tianon
Copy link
Member

tianon commented Jan 11, 2014

FWIW, I am definitely +1 on having this as an option, even though I don't see myself using it in the immediate future.

@discordianfish
Copy link
Contributor Author

@vieux Can't reproduce that. What exactly did you do?
Just ran 'make' on a copy of https_client? This works fine for me:

git branch
..
* https_client
..
fish@apollon:~/dev/go/src/github.com/dotcloud/docker$ make
docker build -t docker .
Uploading context 53.64 MB
Uploading context 
# Skipping unknown instruction DOCKER-VERSION
Step 1 : FROM stackbrew/ubuntu:12.04
....

After building the binary from this branch (make binary), I can run it just fine and running make against the new binary also works as expected.

@vieux
Copy link
Contributor

vieux commented Jan 13, 2014

@discordianfish
Copy link
Contributor Author

@vieux I've tried to reproduce it in vagrant but it worked fine for me. I did a vagrant up, vagrant ssh. Then checkout out docker to the same path as you, ran make, stopped the old docker, ran the binary with my changes (sudo -E ./bundles/0.7.5-dev/binary/docker-0.7.5-dev -d -H unix:///var/run/docker.sock -H tcp://0.0.0.0:4243) and finally ran make which ran just fine.

But I had to increase the memory of my VM, without that it crashed already when building the binary the first time. Maybe you ran into that issue as well?

@unclejack
Copy link
Contributor

This PR is ready for testing. Anyone who'd like to use this feature and can try it out, please feel free to do so and provide feedback.

@denibertovic
Copy link
Contributor

I seem to be getting "handshake failure" whatever I do. This is what I used for generating the certificates: http://pastebin.com/Qa4JtHBE

And I've taken care to set CN to * for the server.crt.

@crosbymichael
Copy link
Contributor

I get the same error as @denibertovic when using the instructions in this PR.

@discordianfish
Copy link
Contributor Author

Thanks @jamtur01 - Guess I'm really not very good at writing docs :)
@crosbymichael That's odd.. The reason @denibertovic had this issue was due to the client key missing the 'extendedKeyUsage' attribute, easy-rsa should take care of that. Anyways! As discussed with @denibertovic in IRC it make sense to use openssl directly, so I updated the docs. Could you please try it with those instructions again?

@denibertovic
Copy link
Contributor

I can confirm it's working now, following the examples in the docs.

@bfirsh
Copy link
Contributor

bfirsh commented Mar 6, 2014

I can confirm this works great with the cert/keys we're generating for Orchard!

We're missing a crucial bit for the server side though: we need to verify that a client is using a specific certificate, not just one that has been signed by the CA. We use this for authentication – to authenticate that client has access to the host.

This is how the stunnel verify=3 option works (see their docs for a detailed explanation). Docker is currently behaving as if it were stunnel in verify=2 mode, which is not useful for access control.

This feature is definitely not necessary for this PR though – it can be added later. And, in fact, the client-side stuff will work great for Orchard. It doesn't need to verify a specific server-side cert on the client-side, so we can use Docker's TLS stuff against our stunnel servers.

Looking forward to using this – thanks @discordianfish. :)

@discordianfish
Copy link
Contributor Author

@bfirsh Right, for now authorization happens just based on the CA: Everything signed by that CA is allowed to connect to the server (or it's a valid server, if the client validates that).

@denibertovic
Copy link
Contributor

FYI I tested everything from scratch one more time with the latest changes. LGTM.

the tests seem to fail because of travis/github issue (ie the worker failed to clone the repo)

@tianon
Copy link
Member

tianon commented Mar 6, 2014

They've been having lots of issues like that recently. I've gone and restarted the worker manually, so hopefully it'll complete properly this time.

@crosbymichael
Copy link
Contributor

@discordianfish this needs rebased

@discordianfish
Copy link
Contributor Author

@crosbymichael Rebased & tests passed

@crosbymichael
Copy link
Contributor

@discordianfish

build error

---> Making bundle: binary (in bundles/0.8.1-dev/binary)
# github.com/dotcloud/docker/api
api/server.go:1192: undefined: socketGroup
api/server.go:1192: cannot assign to socketGroup
api/server.go:1193: undefined: socketGroup
api/server.go:1194: undefined: socketGroup
api/server.go:1195: undefined: socketGroup

@denibertovic
Copy link
Contributor

I can confirm I get the same error when running "make binary".

Docker-DCO-1.1-Signed-off-by: Johannes 'fish' Ziemke <[email protected]> (github: discordianfish)
@discordianfish
Copy link
Contributor Author

@crosbymichael Sorry about that, missed to stage a change. It's pushed now!

@denibertovic
Copy link
Contributor

Tested again from scratch. The error is gone now.

@crosbymichael
Copy link
Contributor

@discordianfish this needs rebased but we will take this from here. thanks and everything is looking good.

@crosbymichael
Copy link
Contributor

meaning one of us will do the rebase when we merge ;)

@crosbymichael crosbymichael modified the milestones: 1.0, 0.9.0 Mar 11, 2014
@crosbymichael
Copy link
Contributor

@vieux I tried to do a quick merge of master for this but there was a lot of changes in the api. Do you think you could take a look?

@vieux
Copy link
Contributor

vieux commented Mar 19, 2014

@crosbymichael Yes I'll

@vieux vieux merged commit c000cb6 into moby:master Mar 19, 2014
@jamtur01 jamtur01 mentioned this pull request Apr 2, 2014
@paturuv
Copy link

paturuv commented Jan 19, 2015

The basic authentication on docker private registry using nginx is real tricky one.
Not sure if that really works with docker client.

I have docker registry running on a ssl enabled server. I am using nginx to enable ssl and configured the basic authentication too.

The curl works find to connect to registry and get response. Where as the docker client can't go through with the following error message :
Error response from daemon: Server Error: Post https://example.domain.com/v1/users/:
x509: certificate signed by unknown authority

I have figured out that the docker won't work with self signed certificates and hence created a CA certificate and used the same, still the same issue.

Am I missing anything here or is it a bug?

@thaJeztah
Copy link
Member

@paturuv have you looked in current open issues? A quick search for "x509" in the open issues showed some other people having problems getting it to work; https://github.com/docker/docker/issues?q=is%3Aissue+is%3Aopen+x509 perhaps those contain useful information to get it working.

@paturuv
Copy link

paturuv commented Jan 21, 2015

Hi-
Thank you so much pointing me to the issues log, that helped in resolving the login issue.

Thanks
Venkat

Sent from my iPhone

On Jan 19, 2015, at 6:24 PM, Sebastiaan van Stijn [email protected] wrote:

@paturuv have you looked in current open issues? A quick search for "x509" in the open issues showed some other people having problems getting it to work; https://github.com/docker/docker/issues?q=is%3Aissue+is%3Aopen+x509 perhaps those contain useful information to get it working.


Reply to this email directly or view it on GitHub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setup API authentication