Should be added to compose with the x-bake extension too:

Silly (?) idea; would it work if, instead of adding an additional flag, we would allow --no-cache to take these options?

--no-cache (no args) same as today (disable all caches)
--no-cache (what's implemented in this PR

@thaJeztah No, I don't think this is possible. Even with some crazy hack (that is not worth the maintenance cost) to flag parsing, it is ambiguous to positional args in some cases.

Theoretically, if we wanted we could deprecate the old flag and add something that signifies "all stages" to new flag. But I think the current flag is already too common.

@thaJeztah sgty?

Even with some crazy hack (that is not worth the maintenance cost) to flag parsing, it is ambiguous to positional args in some cases.

Hm.. yeah, it may be tricky to implement it (with the positional args). I was curious if we wouldn't already be in this situation with --no-cache (because boolean flags also can accept true / false as argument);

mkdir true
echo "FROM busybox" > true/Dockerfile

docker build -t foo true
[+] Building 0.1s (5/5) FINISHED
...


docker build -t foo --no-cache true
[+] Building 0.1s (5/5) FINISHED
...

But it looks like (unlike for other type of flags?) boolean flags require values to be passed with = ? (TIL)

docker build -t foo --no-cache true true
"docker build" requires exactly 1 argument.
See 'docker build --help'.

docker build -t foo --no-cache=true true
[+] Building 0.1s (5/5) FINISHED
...

So the "short answer" is that I don't really like --no-cache-filter. It's quite verbose for a flag name, it overlaps with --no-cache, and (imo) it would still be nicer if it would be possible with the existing flag, but yes, I see how it may be a bit of a challenge to get it right.

As to deprecating --no-cache; I'm not sure if we'd be able to pull that off; it's been around for a long time, and I doubt we'd be able to remove it without it being a large breaking change.

Irregardless of what we choose in that respect, I think we should consider making them mutually exclusive, and print a conflict error if both is used. Looking at (e.g.) this example from moby/buildkit#1213:

--no-cache --no-cache-filter !target1,!target2 invalidates cache for all targets but target1 and target2 (less common usecase)

Looking at that example, I don't think the --no-cache flag adds any value there, and can be deduced from the --no-cache-filter flag alone (--no-cache-filter implicitly says; "use cache, (except) for ...")

One more thing I observed in the examples from that ticket;

--no-cache-filter mount:id1 invalidates cache for the cache mount with id id1 (defined with RUN --mount=type=cache,id=id1,...)

Less of a strong opinion on that, but I recall at some point, we "deprecated" some syntaxes that used a colon (:) as separator (e.g. --security-opt seccomp:unconfined) in favor of using = (--security-opt seccomp=unconfined). I'm wondering if we should do the same here, so that it would become:

--no-cache-filter mount=id1

Also wondering; if the above is to "use all cache mounts, except for mount id1", would there also be a reverse? ("only use cache mount id1"), and what that would look like?

--no-cache-filter !mount:id1

If we'd go for =, that could become

--no-cache-filter mount!=id1

(which feels the 'natural' thing to do), but if we want parsing to be handled using the "csv" notation used elsewhere, I guess the "correct" way would be;

--no-cache-filter mount=!id1

Which, hm, would be a bit awkward as well

(overall, the double negative is a bit hard to grasp; --no-cache combined with exclusions is easy to get "wrong")

@thaJeztah What changes are you asking? I can't implement ! or mount-id atm as there is no support for either in the current buildkit. While stage based no-cache has always been supported in buildkit but never exposed to docker.

I don't think --no-cache can be deduced for all cases with exclusions. What does foo,!bar mean?

ping @thaJeztah

I have a use case for that one 🙂

As discussed;

• perhaps good to (for now) produce an error if both flags are set
• if possible; add docs to describe the behavior (including the "typo" case? 😉)

@thaJeztah PTAL

@tonistiigi Hello, not sure where the right place to ask this is, but I am unclear on what to provide as an argument to the --no-cache-filter flag. For example, if I want to skip one specific step (e.g. 12/14), what would the argument need to be? I tried several things, but none of them seemed to work (at least, the output from docker build --progress plain always showed the step as #CACHED).

@taranlu-houzz you need to move your step to a separate stage

FROM alpine AS base
RUN first
RUN second

to skip second:

FROM alpine AS base
RUN first
FROM base AS mid
RUN second

--no-cache-filter mid

Hmm, I see. Okay, I will try that out. Is this documented anywhere? I am new-ish to docker and I couldn't really find any examples online of how to use --no-cache-filter.