…#8228)

## Summary of changes

Adds a simple `IArrayPool<char>` for use by Newtonsoft.JSON, and uses it
everywhere we can

## Reason for change

Newtonsoft.JSON fundamentally works with .NET's `char` type (UTF-16),
(as opposed to System.Text.Json which works with UTF-8 where it can). To
do so, it needs to create a bunch of `char[]` instances to use as
buffers.

The `JsonTextReader` and `JsonTextWriter` abstractions allow plugging in
an `IArrayPool<char>` implementation, and luckily this matches (pretty
much exactly) the API exposed by the `ArrayPool` in corelib (+
vendored), so it's easy to implement.

This should help alleviate some GC pressure, as we currently do a fair
amount of serializing and deserializing.

## Implementation details

Pretty simple:
- Implement `IArrayPool<char>` using `ArrayPool<char>.Shared`
- Get 🤖 to find everywhere that we could use it (`JsonTextReader` and
`JsonTextWriter`) and initialize
- Fix some cases where these weren't being disposed.

> [!WARNING]
> It's important that we _do_ dispose these, so that the arrays are
correctly returned to the pool, so that we don't leak memory

There are actually other places we can update too, as this PR doesn't
cover the common `JsonConvert.Serialize()` etc, but I'll follow up with
those in a separate PR.

## Test coverage

All the existing tests should pass. I worked on this as part of general
perf work on remote config, so the results are a bit fuzzy (as I can't
remember if it includes the savings from #8226 as well), but the results
are pretty conclusive, especially for big payloads 😅


| Method | size | Mean | Error | Allocated |
| ---------------------------- | ----- | ----------: | ------------: |
---------: |
| DeserializeResponse_Original | Small | 22.71 ns | 2,076.771 ns | 23.83
KB |
| DeserializeResponse_Updated | Small | 13.70 us | 0.186 us | 17.23 KB |
| | | | | |
| DeserializeResponse_Original | Big | 1,953.04 us | 58,665.219 ns
|2,343.26 KB |
| DeserializeResponse_Updated | Big | 614.46 us | 11.988 us | 252.37 KB
|


## Other details


https://datadoghq.atlassian.net/browse/LANGPLAT-940

Discovered this while exploring remote config optimizations

## Summary of changes

Stop sending the fixed `RootVersion = 1` with every remote config
request

## Reason for change

Currently we're sending a fixed value of `RootVersion = 1` for all our
remote config requests, but doing so causes the agent to repeatedly send
us all the root certificates, significantly increasing the payload size,
because it thinks we haven't seen them. Sending the "final" version,
acknowledges it, and stops all the extra data, saving ~35kB per call.

## Implementation details

- Deserialize the roots in `GetRcmResponse`, leaving them as base64
encoded `string`s (which is how they are sent in the payload)
- When processing the response, de-encode the _last_ root object, and
deserialize it
- Using the `Base64DecodingStream` introduced in #8226 to avoid extra
allocations
  - Using the `IArrayPool` from #8228
- Introduces a `MinimalTufRoot` (in contrast to `TufRoot`) so that we
only materialize what we need (the `roots.signed.version` key)

This implementation avoids ~35kB per call for subsequent remote config
requests.

## Test coverage

Added unit test, and did manual test with the real agent, to confirm the
expected behaviour (reduction in data sent)

## Other details

https://datadoghq.atlassian.net/browse/LANGPLAT-940

All part of the Remote Config perf stack

## Summary of changes

Adds `Base64DecodingStream` helper class

## Reason for change

In a few places (particularly Remote Config and Newtonsoft.JSON) we have
code like this:

```csharp
string someValue = // 
var contentDecode = Convert.FromBase64String(someValue);
using var stream = new MemoryStream(contentDecode);
```

This takes an existing `string`, decodes it from base64 to a `byte[]`,
then feeds that `byte[]` into a `MemoryStream`. For big strings, that's
a potentially large extra array allocation we can avoid. Instead,
`Base64DecodingStream` acts effectively as a `MemoryStream` over the
`string`, doing the decode either on the fly (.NET Core), or by using a
pooled buffer to decode the input string in chunks.

## Implementation details

Basically asked 🤖 Claude to do this, then reviewed and tidied up and
iterated. The general idea is:

- Keep track of where in the string we are
- **.NET Framework/.NET Standard only**
- Decode the next section of the string into an array-pool rented
buffer, using the vendored `Base64.DecodeFromUtf8InPlace`
  - Copy the buffer into the destination `byte[]`
- **.NET Core only**
- Decode the next section of the string directly into the destination
`Span<T>`

It didn't seem worth trying to unify these two paths, given the lack of
`Convert.TryFromBase64Chars` in .NET Framework. We _could_ use
`Base64.DecodeFromUtf8` to write directly to the destination span
instead, but we would still need to do the narrowing, and doing that in
the destination span is a little risky, as it would mean we write a
bunch of bytes which are then leftover junk. It's _probably_ still fine,
but I don't know that it's worth the complexity/risk.

> [!WARNING]
> Rather than handle the case where you're passed a destination buffer
that's <3 bytes, this implementation currently just throws. Otherwise we
have to decode into a stackallocated buffer, and hang onto the
"overflow" bytes to avoid returning 0 when we're not EOF, which is a bit
of a pain. For the places we currently use it, I don't think this will
be a problem, but if others disagree, we can handle the edge case too.

## Test coverage

Added a variety of unit tests for the implementation

## Other details

https://datadoghq.atlassian.net/browse/LANGPLAT-940

This will be part of a Remote config stack, but for now kept it agnostic

…#8228)

## Summary of changes

Adds a simple `IArrayPool<char>` for use by Newtonsoft.JSON, and uses it
everywhere we can

## Reason for change

Newtonsoft.JSON fundamentally works with .NET's `char` type (UTF-16),
(as opposed to System.Text.Json which works with UTF-8 where it can). To
do so, it needs to create a bunch of `char[]` instances to use as
buffers.

The `JsonTextReader` and `JsonTextWriter` abstractions allow plugging in
an `IArrayPool<char>` implementation, and luckily this matches (pretty
much exactly) the API exposed by the `ArrayPool` in corelib (+
vendored), so it's easy to implement.

This should help alleviate some GC pressure, as we currently do a fair
amount of serializing and deserializing.

## Implementation details

Pretty simple:
- Implement `IArrayPool<char>` using `ArrayPool<char>.Shared`
- Get 🤖 to find everywhere that we could use it (`JsonTextReader` and
`JsonTextWriter`) and initialize
- Fix some cases where these weren't being disposed.

> [!WARNING]
> It's important that we _do_ dispose these, so that the arrays are
correctly returned to the pool, so that we don't leak memory

There are actually other places we can update too, as this PR doesn't
cover the common `JsonConvert.Serialize()` etc, but I'll follow up with
those in a separate PR.

## Test coverage

All the existing tests should pass. I worked on this as part of general
perf work on remote config, so the results are a bit fuzzy (as I can't
remember if it includes the savings from #8226 as well), but the results
are pretty conclusive, especially for big payloads 😅


| Method | size | Mean | Error | Allocated |
| ---------------------------- | ----- | ----------: | ------------: |
---------: |
| DeserializeResponse_Original | Small | 22.71 ns | 2,076.771 ns | 23.83
KB |
| DeserializeResponse_Updated | Small | 13.70 us | 0.186 us | 17.23 KB |
| | | | | |
| DeserializeResponse_Original | Big | 1,953.04 us | 58,665.219 ns
|2,343.26 KB |
| DeserializeResponse_Updated | Big | 614.46 us | 11.988 us | 252.37 KB
|


## Other details


https://datadoghq.atlassian.net/browse/LANGPLAT-940

Discovered this while exploring remote config optimizations

## Summary of changes

A variety of minor performance improvements to remote config

## Reason for change

I did some initial benchmarking of remote config, as well as running a
test app with ASM/Debugger enabled (which use RCM), and the results
weren't great. Given we make RCM requests every 5s, smallish changes
here should add up, though the real-world effect will be tricky to
gauge.

## Implementation details

Most of the individual changes are small. In summary:
- Making work lazy where possible (delaying creating collections)
  - Don't re-create dictionaries inside loops if not required
- Avoid creating empty dictionaries and collections for no-op RCM
responses
- Avoid creating empty collections in JSON objects when they're not in
the JSON
- Cache things that change rarely
- The `ExtraServicesProvider` will rarely see new service names, so
cache the array (as the collection is append-only)
- Don't create a new request each time, just update any values that may
have changed
  - Cache the capabilities array which will rarely change
- There's actually a potential threading bug around the use of
`BigInteger`. We probably could/should use `ulong` instead but that I'll
look at that in a separate PR
- Use abstractions introduced earlier in the stack (e.g. #8226, #8228)

Additionally, I did a little bit of cleanup:
- Remove unused members from the `IRcmSubscriptionManager` interface
- Add `#nullable enable` to the RCM types (and fix nullability where
required)

## Test coverage

Mostly covered by existing unit tests, also did some manual testing.
Finally, ran a few benchmarks, but it's a bit tricky to check reliably.
This benchmark is benchmarking `_manager.SendRequest(_rcmTracer, _ =>
_steadyStateResponseTask)` and passing the same response every time. In
practice, the response changes every time, so this isn't strictly
representative, but with the changes to request caching, this should
actually mean our improvements are _better_ than the original would be.
The regression in the .NET duration is curious, but I'm not massively
concerned, and the allocations are obviously down a lot at least


| Method | Runtime | Mean | Error | Allocated |
| ------------------------ | ------------------ | ----------: |
-----------: | --------: |
| PollSteadyState_Original | .NET 10.0 | 14,235.8 ns | 275.09 ns | 12.04
KB |
| PollSteadyState_Updated | .NET 10.0 | 20,472.0 ns | 1,352.567 ns |
3.90 KB |
| | | | | |
| PollSteadyState_Original | .NET 6.0 | 33,915.7 ns | 3,339.42 ns |
12.27 KB |
| PollSteadyState_Updated | .NET 6.0 | 14,979.7 ns | 447.784 ns | 3.99
KB |
| | | | | |
| PollSteadyState_Original | .NET Core 2.1 | 21,488.2 ns | 365.39 ns |
12.85 KB |
| PollSteadyState_Updated | .NET Core 2.1 | 17,847.2 ns | 693.510 ns |
4.04 KB |
| | | | | |
| PollSteadyState_Original | .NET Core 3.1 | 18,925.2 ns | 304.16 ns |
12.23 KB |
| PollSteadyState_Updated | .NET Core 3.1 | 15,407.7 ns | 313.26 ns |
4.01 KB |
| | | | | |
| PollSteadyState_Original | .NET Framework 4.8 | 20,946.8 ns | 320.06
ns | 14.21 KB |
| PollSteadyState_Updated | .NET Framework 4.8 | 16,635.2 ns | 266.526
ns | 4.40 KB |


## Other details
https://datadoghq.atlassian.net/browse/LANGPLAT-940

All part of the Remote Config perf stack. I think this is probably about
the end of it for now