fix(cache): handle missing cache hits when chaining two run steps #30762
No reviewers
Labels
No labels
abi/f32
abi/ilp32
abi/sf
accepted
arch/21k
arch/6502
arch/aarch64
arch/alpha
arch/amdgcn
arch/arc
arch/arc32
arch/arc64
arch/arm
arch/avr
arch/bfin
arch/bpf
arch/colossus
arch/cris
arch/csky
arch/dlx
arch/epiphany
arch/fr30
arch/frv
arch/hexagon
arch/hppa
arch/hppa64
arch/ia64
arch/kalimba
arch/kvx
arch/lanai
arch/lm32
arch/loongarch32
arch/loongarch64
arch/m32r
arch/m68k
arch/m88k
arch/mcore
arch/microblaze
arch/mips
arch/mips64
arch/mmix
arch/moxie
arch/mrisc32
arch/msp430
arch/nds32
arch/ns32k
arch/nvptx
arch/or1k
arch/powerpc
arch/powerpc64
arch/propeller
arch/riscv32
arch/riscv64
arch/rl78
arch/rx
arch/s390x
arch/sh
arch/sparc
arch/sparc64
arch/spirv
arch/spu
arch/tricore
arch/v850
arch/vax
arch/vc4
arch/ve
arch/wasm
arch/x86
arch/x86_64
arch/xcore
arch/xtensa
autodoc
backend/c
backend/llvm
backend/self-hosted
binutils
breaking
build system
debug info
docs
error message
frontend
fuzzing
incremental
lib/c
lib/compiler-rt
lib/cxx
lib/std
lib/tsan
lib/ubsan-rt
lib/unwind
linking
miscompilation
os/android
os/contiki
os/dragonfly
os/driverkit
os/emscripten
os/freebsd
os/fuchsia
os/haiku
os/hermit
os/hurd
os/illumos
os/ios
os/linux
os/maccatalyst
os/macos
os/managarm
os/netbsd
os/ohos
os/openbsd
os/plan9
os/redox
os/rtems
os/serenity
os/tvos
os/uefi
os/visionos
os/wasi
os/watchos
os/windows
proposal
release notes
testing
zig cc
zig fmt
zig reduce
bounty
bug
contributor-friendly
downstream
enhancement
infra
optimization
question
regression
upstream
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
ziglang/zig!30762
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "bfredl/zig:cache4"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
fixes https://github.com/ziglang/zig/issues/19817
This is the same as https://github.com/ziglang/zig/pull/25788 but rebased due to unrelated
std.Ioname changes (I don't think I can close it over there)This improves the efficiency of the cache when chaining muliple commands
like
assume that step2 takes much long time than step1
if we make a change to "src/input.c" which produces an identical
"output1.h" as a previous input, one would expect step2 not to
rerun as the cached output2.h only depends on the content of output1.h
However, this does not work yet as the hash of src/input.c leaks into
the file name of the cached output1.h, which the second run step
interprets as a different cache key. Not using the ".zig-build/o/{HASH}"
part of the file name in the hash key fixes this.
https://github.com/bfredl/zig-run4run is updated for zig 0.16 as a demonstration. e.g. in src/foo.c changing implementation_of_foo() to new_implementation_of_foo() should only rerun the first step, not the second.
e965df5c7fcb7d0dc557Sorry I still need more time to get around to helping get this landed.
cb7d0dc557c223d50428rebased on top of #35428 (took a guesswork at how serialization should be done, seems to still work but lmk if it is wrong)
@ -10,11 +10,13 @@ root_dir: Cache.Directory,/// The path, relative to the root dir, that this `Path` represents./// Empty string means the root_dir is the path.sub_path: []const u8 = "",content_hash_name: ?[]const u8 = null,I definitely want to solve the issue being tackled here, but this approach introduces this "content hash" into an abstraction which is only meant to represent an open root directory handle along with the information needed to convert it to a string representation.
The problem needs to be solved at a higher layer than here.
What would be the right abstraction to solve this? The reason this struct was touched is that it currently represents the info associated with a GeneratedFileIndex which is passed from the producer to the consumer.
Thus I can see two different paths ahead:
Step.Runregardless if it is used as a chained input or not.Does any of these seem right to you?
Regardless, I realized that it is unnecessary to serialize any extra info into Configuration, so I just removed that for now (without addressing this concern just yet)
c223d50428cce3110c03cce3110c0344c2b186c6View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.