Thanks for the PR

Isnt't the source of kind "json" enough?
You can have different kind of usage by looking at https://github.com/updatecli/updatecli/blob/main/e2e/updatecli.d/success.d/json.yaml

Have you considered chaining multiple sources with one of the kind "json" instead of using it in the transformers?
I like the transformer because it's more concise but I am concerned about adding a new way to achieve a similar behavior

@olblak -- I'm not exactly sure what you mean by chaining sources together -- so

1. curious about that, and
2. Yes, had considered the JSON source.

The actual data is coming from an HTTP source -- and while I know you can use a URL with the JSON source, the HTTP call itself is a POST with a body (hence my other PR!) and additional auth headers, which can't be done with the simple URL support within the JSON source.

I had considered looking at supporting this type of HTTP support within the JSON source (and other file-based sources) but that looked like a much bigger change!

I had also considered (and even PoC'ed) a HTTP source -> File Target -> JSON source -> File Target... BUT that is a heck of a lot of steps vs just HTTP source (w/transformer) -> File Target...

I also felt that while that PoC was a viable solution (if clunky), it spreads the logic of getting the source data (and cleaning it) into multiple places... unlike simply using transforms within the originating source to output the data exactly as needed by the next steps.

To your concern though, I would suggest being able to process JSON data as both a source and as a transformer is actually powerful. Originally I thought it could deprecate the JSON source -- ie use a File source and a JSON transformer BUT I think the way you think about this is that the transformer is for more simple cases, or where (like my case) the source data isn't coming from a file whereas the JSON source is what you use if you need a more powerful solution, ie could support more powerful JSON handling in the future without overly complicating the transformers

Thanks for the detailed explanation.

I had considered looking at supporting this type of HTTP support within the JSON source (and other file-based sources) but that looked like a much bigger change!

I agree with your observation, we decided to add a HTTP source because we didn't want to bloat the current file sources used for json, yaml,etc.

@olblak -- I'm not exactly sure what you mean by chaining sources together -- so

When we need to pass information from one source to another one, we use something like:

sources:
    debian:
        name: get latest image tag for "debian"
        kind: dockerimage
        spec:
            image: debian
            tagfilter: ^\d*$
            versionfilter:
                kind: semver
                pattern: '>=8'
    debian-digest:
        dependson:
            - debian
        name: get latest image "debian" digest
        kind: dockerdigest
        spec:
            image: debian
            tag: '{{ source "debian" }}'

where dependson ensure that source "debian-digest" is executed after "debian"

That's how I thought about having a source of kind "HTTP" followed by a source of kind "JSON"

But I agree that your proposal about using a json transformer would simplify the manifest