@stgraber, quick notes and questions.

1. You mentioned that we should forbid using the name parameter when performing direct backups; there are several places in the code where that could happen, should I put the check in the REST API functions in cmd/incusd?
2. I noticed that image backup lifecycle events are done in cmd/incusd/backup.go, but that is not the case for the others (storage volumes and buckets); any particular reason or just legacy?
3. I didn’t care about making the operations cancellable; do you insist on that?

You mentioned that we should forbid using the name parameter when performing direct backups; there are several places in the code where that could happen, should I put the check in the REST API functions in cmd/incusd?

Yeah, request validation should happen as early as possible, so directly in the Post handler would be good.

I noticed that image backup lifecycle events are done in cmd/incusd/backup.go, but that is not the case for the others (storage volumes and buckets); any particular reason or just legacy?

I don't think there's any particular reason for that. The package setup for backups is a bit unusual so that may have had something to do with it. In practice, so long as it's done in a spot that's reliable, we're fine :)

I didn’t care about making the operations cancellable; do you insist on that?

Nah, very few operations are cancellable, it's basically limited to operations that download stuff from the Internet and which can therefore be easily cancelled.

In the case of streaming backups out, the background operation wouldn't even really be needed as the API call is synchronous in this case and a cancellation would come in the way of a closed connection from the user having closed their side of the connection. That will make all write operations fail which will in turn cancel everything.

I don't think there's any particular reason for that. The package setup for backups is a bit unusual so that may have had something to do with it. In practice, so long as it's done in a spot that's reliable, we're fine :)

Alright, I’m wondering if streamed backups should trigger a lifecycle event. I don’t know if these events implicitly convey some kind of object creation on the server, in which case they should be disabled.

Nah, very few operations are cancellable, it's basically limited to operations that download stuff from the Internet and which can therefore be easily cancelled.

Well, closing a pipe is also pretty easy, it’s just that I felt lazy.

In the case of streaming backups out, the background operation wouldn't even really be needed as the API call is synchronous in this case and a cancellation would come in the way of a closed connection from the user having closed their side of the connection. That will make all write operations fail which will in turn cancel everything.

Agreed, but I think it’s pretty nice to have a reporting that this action is currently running. In a way, console access isn’t completely asynchronous either, from a high level (there’s bidirectional data that can only flow if the consumer stays there).
Now, imagine that I’m the cluster admin and I want to free up some space, really quickly. A user is backing up a 1TiB instance and I need that TiB now, so I’m deleting the instance. My understanding is that the data will only be cleared when the user has finished their backup, which can impact my usage of the cluster.

Maybe I’m whatifing too much :)

Alright, I’m wondering if streamed backups should trigger a lifecycle event. I don’t know if these events implicitly convey some kind of object creation on the server, in which case they should be disabled.

The event should be emitted. Lifecycle events are meant for auditing activity on an Incus server.

Eh, there are quite a few more edge cases to fix. Pipes are fun. I also need to prevent creating DB entries.

Alright, that should be good now.

This actually ended up looking a lot cleaner than I expected ;)

Ok I’ll do those quick patches later today.

I got caught up in other matters, I’m looking into this now.
(Rebasing in a separate force-push to simplify auditing the diffs)