Skip to content

Support for multiple registries with pull and search commands#2

Merged
rhatdan merged 1 commit intorhatdan:registryfrom
miminar:registry
Jan 26, 2015
Merged

Support for multiple registries with pull and search commands#2
rhatdan merged 1 commit intorhatdan:registryfrom
miminar:registry

Conversation

@miminar
Copy link
Copy Markdown

@miminar miminar commented Jan 26, 2015

This PR fixes support for --registry-append and --registry-replace options
for pull command and adds their support for search command.

Pull and Search commands iterate over a list of registries given with
options --registry-append or --registry-replace. In case of pull command
the iteration stops when a matching image is found. In case of search
command the iteration never stops - all the results are accumulated.

--registry-append is kind of confusing - it does exact oposite. It
prepends registries separated with commas in reversed order to official
index (docker.io). So the last registry specified is queried as the
first one.

TODO:

  • rename --registry-append to something like --registry-prepend.
  • add a repository origin to the results table for search command if
    multiple registries were queried

Pull and Search commands iterate over a list of registries given with
options --registry-append or --registry-replace. In case of pull command
the iteration stops when a matching image is found. In case of search
command the iteration never stops - all the results are accumulated.

--registry-append is kind of confusing - it does exact oposite. It
prepends registries separated with commas in reversed order to official
index (docker.io). So the last registry specified is queried as the
first one.

TODO:
  * rename --registry-append to something like --registry-prepend.
  * add a repository origin to the results table for search command if
    multiple registries were queried

Signed-off-by: Michal Minar <[email protected]>
rhatdan added a commit that referenced this pull request Jan 26, 2015
Support for multiple registries with pull and search commands
@rhatdan rhatdan merged commit 38372bf into rhatdan:registry Jan 26, 2015
@rhatdan
Copy link
Copy Markdown
Owner

rhatdan commented Jan 26, 2015

I have taken care of the append->prepend switch.

rhatdan pushed a commit that referenced this pull request Mar 23, 2015
rhatdan pushed a commit that referenced this pull request Aug 2, 2015
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.

Quoting MkdirAll documentation:

> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.

This means two things:

1. If a directory to be created already exists, no error is returned.

2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.

The above is a theory, based on quoted documentation and my UNIX
knowledge.

3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.

Because of #1, IsExist check after MkdirAll is not needed.

Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.

Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.

[v2: a separate aufs commit is merged into this one]

[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <[email protected]>
rhatdan pushed a commit that referenced this pull request Feb 26, 2016
Fix exec start api with detach and AttachStdin at same time. fixes #2
rhatdan pushed a commit that referenced this pull request Jan 19, 2018
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there"

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.

Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.

NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).

For more details, a quote from my runc commit 6f82d4b (July 2015):

    TL;DR: check for IsExist(err) after a failed MkdirAll() is both
    redundant and wrong -- so two reasons to remove it.

    Quoting MkdirAll documentation:

    > MkdirAll creates a directory named path, along with any necessary
    > parents, and returns nil, or else returns an error. If path
    > is already a directory, MkdirAll does nothing and returns nil.

    This means two things:

    1. If a directory to be created already exists, no error is
    returned.

    2. If the error returned is IsExist (EEXIST), it means there exists
    a non-directory with the same name as MkdirAll need to use for
    directory. Example: we want to MkdirAll("a/b"), but file "a"
    (or "a/b") already exists, so MkdirAll fails.

    The above is a theory, based on quoted documentation and my UNIX
    knowledge.

    3. In practice, though, current MkdirAll implementation [1] returns
    ENOTDIR in most of cases described in #2, with the exception when
    there is a race between MkdirAll and someone else creating the
    last component of MkdirAll argument as a file. In this very case
    MkdirAll() will indeed return EEXIST.

    Because of #1, IsExist check after MkdirAll is not needed.

    Because of #2 and #3, ignoring IsExist error is just plain wrong,
    as directory we require is not created. It's cleaner to report
    the error now.

    Note this error is all over the tree, I guess due to copy-paste,
    or trying to follow the same usage pattern as for Mkdir(),
    or some not quite correct examples on the Internet.

    [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <[email protected]>
rhatdan pushed a commit that referenced this pull request Jul 15, 2018
When go-1.11beta1 is used for building, the following error is
reported:

> 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads
> arg #2, but call has 1 arg

While fixing this, let's also fix a few other things in this
very function (startServiceVMIfNotRunning):

1. Do not use fmt.Printf when not required.
2. Use `title` whenever possible.
3. Don't add `id` to messages as `title` already has it.
4. Remove duplicated colons.
5. Try to unify style of messages.
6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/
...

In general, logging/debugging here is a mess and requires much more
love than I can give it at the moment. Areas for improvement:

1. Add a global var logger = logrus.WithField("storage-driver", "lcow")
and use it everywhere else in the code.
2. Use logger.WithField("id", id) whenever possible (same for "context"
and other similar fields).
3. Revise all the errors returned to be uniform.
4. Make use of errors.Wrap[f] whenever possible.

Signed-off-by: Kir Kolyshkin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants