We've been building wpt.live (previously web-platform-tests.live) as a replacement for w3c-test.org. The latter provides two distinct services:

1. publishing WPT at the latest version of the repository's master branch
2. publishing a "mirror" of pull requests filed against WPT's GitHub.com project

The first is relatively straightforward, and we completed it without any integration with the WPT repository. The second is more complex, and this issue is intended to facilitate discussion on its implementation.

Our initial implementation is available under the Bocoup GitHub organization and outlined in this presentation. In brief: it's designed to poll the WPT git repository for special refs and create git working trees based on that information. This allows it to function without secrets, trusted access to the git repository, or any configuration in WPT's GitHub project.

We originally planned to create the required git refs from the GitHub "Actions" that ran in response to each pull request. Unfortunately, the feature was subsequently changed in an incompatible way (and we subsequently removed the infrastructure from WPT). We're interested in re-implementing it, and here are some alternatives we're considering:

1. create the required refs from a dedicated server in response to GitHub "webhook" events
2. create the required refs from a "scheduled" GitHub Action which identifies relevant pull requests using the GitHub API

The first approach involves granting write permission to an external service, explicit configuration with the WPT GitHub project, and maintaining a secret in an open source project.

The second approach creates noise in the GitHub Actions UI (through many no-op executions), is susceptible to API rate limiting, and will likely result in a less responsive user experience.

Suggestions for other approaches are welcome, although we probably should favor solutions that operate via git refs (since we've already built and deployed a system that works with them).

/cc @foolip @gsnedders @Hexcles @jgraham @sideshowbarker