linuxkit bake#2648
Conversation
|
Please sign your commits following these rules: $ git clone -b "pkg_collect" [email protected]:w9n/linuxkit.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -fAmending updates the existing PR. You DO NOT need to open a new one. |
|
I don't understand the explanation for this PR (maybe I'm missing some context). Could you first describe which problem you try to solve? |
|
added some context to the description. |
I have a hard time understanding how that would work with immutable images (which is what LinuxKit is all about). I think this might be easier to understand if you gave sample |
|
i basically thought of a not buildable config when used with I think the main practical reason is to use it in CI where it is necessary to know which pkg has changed to not rebuild everything on a merge. It also gives gives rollback capabilities outside the git history. But I reevaluated priorities and think its a nice to have but linuxkit is working fine now and i dont think its worth the moby dep. I might create a little external tool for practicing, but low priority. There is a different problem that a container |
|
So, is this basically a way of saying, "for some packages, I want the latest available hash at build time"? |
|
yep, but still create the immutable ymls so its reproducable what has been build and compare what has been changed. |
|
So rather than the usual moby |
|
exactly. It would enable other possibilities as well as described above but there might be other ideas for CI and real hashes. |
|
OK, I get it now; want to think it over somewhat. @rn had commented earlier; let's see his thoughts. I am going to rename the issue. |
|
This would be better done in the |
|
Ah yes, the @w9n does that address the question? |
|
yes, it pretty much goes is in the right direction. I will read and think about the related Issues and PRs in the next days. |
|
I'm still catching up on my post PTO backlog(s) so I haven't fully grokked this one yet, but one quick though I had was that what is called |
|
i didnt choose |
3ebd47f to
557bb5c
Compare
|
Since moby became a dependency this works now depending on moby/tool#194 test,ymlresult hash of the yml files bytes (5f39869eef39b525e90661e2327ed645c1a9df8a.yml)_ |
|
I don't understand what the output of |
|
It's a new file which you can pass to |
|
Does it output to stdout or what? This needs to be documented properly. |
|
Okay, so I could live with this somewhere: Its basically the same information as if tags -> hashes would be in build.yml(what i proposed) just at a different location (undecided if this is the better position or just a cache). The advantage is that you can e.g. hotfix v1.0 or what else for one pkg without influencing everything else. @justincormack yes its writing to stdout and logging to stderr but i could also give it a |
|
@w9n please read my comment #2648 (comment). Could we initially focus on the templating stuff itself (ie replacing |
added EDIT: I would need to make a git method public (or hack together something) to get the hash of a pkgroot for a comment in the bake or cli context. |
|
i dont really trust the test time, something must be wrong... |
This can actually be done with EDIT: Additional tags in build.yml could still be helpful (future, not in this pr). There is https://github.com/a8m/envsubst I could implement this from a file, global conf, cli and/or env after merging the moby confs and before substituting the latest tag, for 1:1 substitution on the whole config file. This could allow something like It had a few "bugs" for nested stuff i tried but should work for simple envsubst otherwise there could be go templates but it gets pretty ugly from my experience or |
|
updated the last comment, otherwise i guess i should revert setting global config in its own namespace and pass the repos as strings into |
Signed-off-by: Robin Winkelewski <[email protected]>
Signed-off-by: Robin Winkelewski <[email protected]>
65a8b51 to
48fdb00
Compare
|
Sorry for all these changes
here is a result yml: DetailsIt should be easier to review and extend now. I didnt implement 1:1 substitution because its not needed yet and pipes can be used. EDIT: needed to add a deep copy function of |
8560131 to
bd4e342
Compare
Signed-off-by: Robin Winkelewski <[email protected]>
Signed-off-by: Robin Winkelewski <[email protected]>
Signed-off-by: Robin Winkelewski <[email protected]>
fixed test. Should cover everything but global config parsing. |
Signed-off-by: Robin Winkelewski <[email protected]>
|
I think we probably still want to have some improvement in this area, but this PR is pretty outdated so closing it for now |
When writing a moby yml I do not care about the container version and hashes (only if I specifically want to stay with a version) and would like to simplify the process of writing them. I also do not want to keep track of all pkg version I use downstream and update them manually when something changes upstream.
The idea is to optionally split the moby config into 1 mutable and n immutable versions. The mutable version describes the desired state which uses image names like
linuxkit/containerd:mutable(or the usual hash tags for pinning down on a version), while the immutable configs stay like currently and get created on demand or mutation from a specifc pkg git repo. They can be versioned and stored in the end.Therefore
linuxkit pkg collect --pkgroot /path/to/repo/linuxkit mutable.ymlfirst collects which container in the mutable yml are tagged mutable and then walks recursivly through pkgroot and looks forbuild.ymlof a package with the same image name. When found it calculates and stores the package hash (Not sure about error case). At this Point we could compare with the last hash of the image and implement build and push on change. At the end a yml with the hashes gets written versioned into a specific dir near the mutable yml and persist as an Artifact for a possible build.This could also be used for CI that triggers
linuxkit pkg collect --pushon every pull for every mutable image yml. Would need to save the immutable artifacts somewhere.The big disadvantage is that you probably dont want moby as dependency in linuxkit which is needed to parse the moby config (On the other side the linuxkit git pkg for the hashes is private). Let me know what you think about it!
(Im just getting used to go, so its a bit messy and uncomplete yet)