### Motivation

Now Swift 5.9 is released and 5.10 nightly images are available, we
should update our CI to use the official release for 5.10 and setup a
new CI for 5.10.
 
### Modifications

- Update 5.9 CI to use `swift:5.9-jammy`
- Add new CI for `swiftlang/swift:nightly-5.10-jammy`

### Result

- 5.9 CI is using official released image.
- 5.10 CI can be brought online.

### Test Plan

Running both of these commands locally succeed:

```console
% docker-compose -f docker/docker-compose.yaml -f docker/docker-compose.2204.59.yaml run test
```
```console
% docker-compose -f docker/docker-compose.yaml -f docker/docker-compose.2204.510.yaml run test
```

### Motivation

- Relates to apple/swift-openapi-generator#165

### Modifications

- Enable new rules in swift-format and address changes required

### Result

- New swift format rules will be implemented

### Test Plan

- Run the tests

Bump swift-format to 5.9

### Motivation

As per apple/swift-openapi-generator#175.

### Modifications

Bumped swift-format for CI, no other changes needed.

### Result

Using swift-format 5.9.

### Test Plan

Soundness passed.


Reviewed by: dnadoba

Builds:
     ✔︎ pull request validation (5.10) - Build finished. 
     ✔︎ pull request validation (5.8) - Build finished. 
     ✔︎ pull request validation (5.9) - Build finished. 
     ✔︎ pull request validation (nightly) - Build finished. 
     ✔︎ pull request validation (soundness) - Build finished. 

#17

Add a soundness --fix flag

### Motivation

When running `./scripts/soundness.sh` produces swift-format warnings, we ask adopters to manually copy/paste a call to swift format to fix the warnings up. This is tedious and unnecessary.

### Modifications

Add a `--fix` option on the `soundness.sh` script to actually apply the fixes as well, avoiding the need to copy/paste long commands.

### Result

Easier fixing up of formatting warnings.

### Test Plan

Manually tested the workflow locally.


Reviewed by: glbrntt

Builds:
     ✔︎ pull request validation (5.10) - Build finished. 
     ✔︎ pull request validation (5.8) - Build finished. 
     ✔︎ pull request validation (5.9) - Build finished. 
     ✔︎ pull request validation (nightly) - Build finished. 
     ✔︎ pull request validation (soundness) - Build finished. 

#19

Same as in the other repos: disable warnings as errors in nightlies.

…tent styling (#20)

### Motivation

- Relates to
[#230](apple/swift-openapi-generator#230)

### Modifications

- Disable respectsExistingLineBreaks .swift-format rule and address
changes requested

### Result

- One of the .swift-format rules will be disabled

### Test Plan

- Run Tests

Co-authored-by: Honza Dvorsky <[email protected]>

### Motivation

Surface the Swift version and platform support status from Swift Package
Index.

### Modifications

Added badges, plus a quick link to the docc docs, to the top of the
README.

### Result

Easier to quickly see our support matrix, plus the quick link to docs.

### Test Plan

Previewed locally.

### Motivation

Recently, the `ClientTransport` protocol was updated to be in terms of
`HTTPBody`, which is an `AsyncSequence<ArraySlice<UInt8>>`. Until now,
the URLSession transport has buffered the request and response bodies in
memory. This PR seeks to bring streaming support to recent Darwin
platforms.

### Modifications

On new enough Darwin platforms[^1], use URLSession delegate to perform
bidirectional streaming with backpressure. Request body backpressure is
provided by the use of `StreamDelegate` with a bound stream pair.
Response body backpressure is provided by the use of
`AsyncBackpressuredStream` which allows for a high and low watermark to
suspend and resume the URLSession task.

[^1]: macOS 12, iOS 15, tvOS 15, watchOS 8

In more detail:

- Vendor internal `AsyncBackpressuredStream` implementation from
SE-0406.
- Add `HTTPBodyOutputStreamBridge` which provides the
`OutputStreamDelegate` required to bridge the `AsyncSequence`-based
`HTTPBody` to an `OutputStream`.
- Add `BidirectionalStreamingURLSessionDelegate` which makes use of
`HTTPBodyOutputStreamBridge` for streaming request bodies and also uses
the delegate callbacks to bridge the response body to `HTTPBody`.
- Add `URLSession.bidirectionalStreamingRequest(...) async throws`,
which provides a high-level URLSession-like API for setting up the
appropriate URLSession task and delegate.
- Add `URLSession.bufferedRequest(...) async throws` to provide a
similar interfaces on platforms that don't support the streaming APIs on
which we depend.
- Add internal `enum URLSessionTransport.Configuration.Implementation`
to control whether to use buffered or streaming implementation.
- Detect appropriate implementation depending on the platform.

### Result

- On new enough Darwin platforms[^1], streaming will be used.

### Test Plan

- Add a set of tests that run with both buffered and streaming
implementation on platforms that support streaming.
- Add a set of tests that only run on platforms that support streaming,
to test request flows only possible with streaming.

However, it's worth noting that our CI only runs on Linux, so we won't
be testing the majority of this new feature in CI.

### Motivation

The tests fail to compile on anything older than macOS 13, because they
use a number of APIs only available in macOS 13+:

- `Duration`
- `Task.sleep(for:)`
- `DispatchQueue.asyncAndWait` (closure variant)

### Modifications

Use the following APIs instead:

- `NIO.TimeInterval` (we have NIO dependency in tests already).
- `Task.sleep(nanoseconds:)`
- `DispatchQueue.asyncAndWait(execute:)`

### Result

Tests can be built on older platforms.

### Motivation

Some of our bidirectional streaming tests were failing intermittently.
When they failed, the symptom was that more bytes were received by the
end user (in the test) than the server sent.

For example, in the test
`testStreamingDownload_1kChunk_10kChunks_100BDownloadWatermark`, we
expect:

- ✅ The server sends 10,000 chunks of 1024 bytes:
  ```console
  ❯ cat repro.txt | grep -i "server sent body chunk" | head -3
  Server sent body chunk 1/10000 of 1024
  Server sent body chunk 2/10000 of 1024
  Server sent body chunk 3/10000 of 1024

  ❯ cat repro.txt | grep -i "server sent body chunk" | wc -l
  10000
  ```

- ✅ URLSession `didReceive data` callback called a non-deterministic
number of times because it may re-chunk the bytes internally, but the
total number of bytes through the delegate calls is 10,240,000:
  ```console
  ❯ cat repro.txt | grep "didReceive data" | head -3
  Task delegate: didReceive data (numBytes: 1024)
  Task delegate: didReceive data (numBytes: 2048)
  Task delegate: didReceive data (numBytes: 1024)

❯ cat repro.txt | grep "didReceive data" | awk '{ print $6 }' | tr -d \)
| paste -sd+ | bc -l
  10240000
  ```

- ❌ The response body chunks emitted by the `AsyncBackpressuredStream`
to the user (in the test) match 1:1 those received by the delegate
callback:
  ```console
❯ cat repro.txt | grep "Client received some response body bytes" | head
-3
  Client received some response body bytes (numBytes: 1024)
  Client received some response body bytes (numBytes: 2048)
  Client received some response body bytes (numBytes: 1024)

❯ cat repro.txt | grep "didReceive data" | awk '{ print $6 }' | tr -d \)
| wc -l
  333

❯ cat repro.txt | grep "Client received some response body bytes" | wc
-l
  334
  ```

- ❌ The total number of bytes emitted by the `AsyncBackpressuredStream`
to the user (in the test) match is 10,240,000 and it can then
reconstruct 10,000 chunks of 1024 to match what the server sent:
  ```console
❯ cat repro.txt | grep "Client received some response body bytes" | awk
'{ print $8 }' | tr -d \) | paste -sd+ | bc -l
  10280960

  ❯ cat repro.txt | grep "Client reconstructing" | tail -3
  Client reconstructing and verifying chunk 10038/10000
  Client reconstructing and verifying chunk 10039/10000
  Client reconstructing and verifying chunk 10040/10000
  ```

So we see that there was one more element emitted from the
`AsyncBackpressuredStream` than the delegate callback wrote, which meant
the test saw an additional 40960 bytes than it expected to and
consequently reconstructed an additional 40 chunks of size 1024 over
what the server sent.

We can see that the `AsyncBackpressuredStream` duplicates an element
along the way, of 40960 bytes,

```diff
❯ diff -u --label=received-in-delegate-callback <(cat repro.txt | grep "didReceive data" | awk '{ print $6 }' | tr -d \)) --label=received-through-async-sequence <(cat repro.txt | grep "Client received some response body bytes" | awk '{ print $8 }' | tr -d \))
--- received-in-delegate-callback
+++ received-through-async-sequence
@@ -305,6 +305,7 @@
 2048
 1024
 40960
+40960
 24576
 2048
 34841
 ```

After some investigation, it turned out there was a missing transition in the state machine that underlies the `AsyncBackpressuredStream`. When calling `suspendNext` when there are buffered elements, but we are above the watermark, we popped the first item from the buffer and returned it _without_ updating the state (with the new buffer, without the popped element).

Consequently, this meant that the _next_ call to `next()` would return the same element again. 

### Modifications

The following modifications have been made in separate commits to aid review:
- Add debug logging to the state machine functions, logging the old state, event, new state, and resulting action.
- Add two tests which reproduce this error.
- Add the missing state transition (which causes the newly added tests to reliably pass).

### Result

Duplicate elements are no longer emitted from the response body.

### Test Plan

- Unit tests were added that fail without the fix, that now pass reliably.

### Additional notes

The implementation we are using for `AsyncBackpressuredStream` was taken from an early draft of SE-0406. We should probably move to using something closer matching that of the current PR to the Swift tree, or that used by swift-grpc, which has also adopted this code and cleaned it up to remove the dependencies on the standard library internals. Additionally, that implementation does not have this missing state transition and also adds an intermediate state to the state machine to avoid unintended copy-on-write.

…abled (#26)

### Motivation

The transport code has some debug logging, which is compile-time omitted
in release builds, and only enabled in tests.

A number of tests produce an enormous amount of output (e.g. the
streaming tests that pull 10GB through 10MB buffers), which are unwieldy
in CI and locally.

Additionally, we found that the logging produced by these tests had
interleaved output, both because of the mixed use of `print` and `debug`
in tests and the lack of locking between multiple threads

Finally, even when running in debug mode with debug logging _disabled_,
the function arguments to the `debug` function were still evaluated
unnecessarily.

### Modifications

- Update the `debug` function to log to standard error (cf. standard
out).
- Pass an autoclosure as argument to `debug` so that it is only
evaluated when logging is enabled.
- Add a lock around the use of `FileHandle.standardError`.
- Replace all stray uses of `print` in tests with `debug`.

### Result

- No debug logging should appear in tests by default (you can enable
them locally when reproducing failures).
- When enabled, log lines should not be all jumbled up.
- When disabled, no string interpolation is performed.

### Test Plan

- Added tests that check that the log message is not interpolated when
disabled.

#28)

### Motivation

If the request completes with error after the response has already been
received we can potentially resume a continuation twice. This was shown
by running some of the tests in long loop.

### Modifications

- Add a shorter test that more reliably reproduced the issue. 
- Add the fix: to set the continuation to `nil` after resuming it, while
still holding the lock.

### Result

- Removed a source of a potential runtime crash.

### Test Plan

- The newly added test, which reliably failed, now reliably passes.

### Motivation

As a follow up to #27, we noted it would be good to align on the latest
draft implementation of SE-0406 (AsyncStream with backpressure) to both
pickup the latest improvements in performance and correctness, and to
minimise the churn if/when this lands in the standard library or
standalone package.

### Modifications

In order to simplify reviewing the following modifications have been
made in independent commits:

* `Add updated SE-0406 implementation as BufferedStream, incl. tests`:
Skim over this—it's vendored in wholesale.
* `Port the custom watermark support to BufferedStream`: Skim over
this—it's a 1:1 port of the logic that was added to
`AsyncBackpressuredStream`.
* `Switch from AsyncBackpressuredStream to BufferedStream in delegate`:
Review this—it's a minimal change.
* `Remove AsyncBackpressuredStream and its vendored locks`: Skim over
this—it's removing the old implementation.

### Result

No functional change, but the internal async sequence we're using should
be more robust, performant, and more likely to match a future standard
library type.

### Test Plan

- The new vendored `BufferedStream` actually comes with a much greater
number of vendored tests than the previous revision.
- Also ported the tests from this repo for the custom watermark logic.
- All our URLSessionTransport-specific tests continue to pass.

---------

Signed-off-by: Si Beaumont <[email protected]>