Feature Request

What challenge are you facing?

Very happy that we have webhook support at all, as it allows us to build pipelines that are more reactive than ones polling with check_every. However, not forwarding the event payload stops us from having pipelines that are as tightly integrated with some services (thinking of Github especially), which hurts us in a couple of areas:

• For continuous integration, I'm jealous at the responsiveness of e.g. Travis compared to a Concourse pipeline. They both receive the same payload, but since Concourse throws away the payload itself, time (and API calls) are wasted by triggering the check to discover a new version (which was just sent over the webhook), and then wait for Concourse to schedule the job based on the newly discovered version.
• Integrating against Github currently requires manually adding deploy keys and/or access keys and then making them available to Concourse (or making a solution which works completely on the side). A better solution would be to have a Github App for e.g. a team, which could handle the creation of access keys/deploy keys whenever they installed the Github App for a new repository. This is something that could quite possibly be done by a Concourse resource, but Github Apps are expected to listen for webhook events in order to respond to new installations or uninstalls.

There are probably more examples of services that just work better with webhooks, but Github is top of my mind. It also seems like we could be making better resources if we had access to webhook payloads.

A Modest Proposal

Obviously, Concourse would have to not only receive the payload and forward it, but resources would have to know what to behave very differently if they received a webhook payload rather than just doing a regular check. As such, I think a solution would have to be a part of (or follow) the resources v2 implementation.

I suggest we add support for another artifact in resources v2 to handle webhook payloads:

{
  "version": "2.0",
  "interfaces": {
    "artifacts": {
      "get": "/opt/resource/get",
      ...
      "webhook": "/opt/resource/webhook"
    }
  }

For e.g. a github-pr-resource, we could invoke /opt/resource/webhook and pass it the payload on stdin, and it could just reshape the JSON payload and spit out a new version without having to do any API calls. This would make it more responsive without hammering the API, and also allow us to make resources for things like Github Apps.

(If the resource does not implement the webhook interface, a normal check is just performed instead).