Description
Found it when writing tests for getting the container logs:

• integration: Add logs tests #43642

The logs are corrupted when capturing both stdout and stderr at the same time.
This isn't an issue with reading the logs, because the corruption is already visible in the json log file.

# cat /var/lib/docker/containers/babb35c51972046a575a42c414cf2ad7d58d270b9594a251fb6d59234ccc021a/babb35c51972046a575a42c414cf2ad7d58d270b9594a251fb6d59234ccc021a-json.log
{"log":"accidents happ"st,"stre"std"std,rr","time":"2022-05-26T07:43:19.448576002Z"}
{"log":"accidents happ"st,"stre"std"std,rr","time":"2022-05-26T07:43:19.44857621Z"}

First commit with the regression is ae5f664 (#43294)

The issue is that marshalFunc used in LogFile use a shared buffer. See:

• https://github.com/moby/moby/blob/master/daemon/logger/jsonfilelog/jsonfilelog.go#L98
• https://github.com/moby/moby/blob/master/daemon/logger/local/local.go#L96

With the mutex lock around WriteLogEntry got removed in the linked commit, the marshaller can be called for both stdout and stderr at the same time, which makes them write to the same buffer.

Allocating a buffer inside the marshaller call fixes the problem. But this is probably not a good idea because it would cause too many memory allocation? Maybe we should restore the critical section for the WriteLogEntry?

Steps to reproduce the issue:
docker logs -f $(docker run -d busybox sh -c 'echo stdout; echo stderr >&2')

Describe the results you received:
Randomly one of:

1. Correct output
2. both stdout printed
3. both stderr printed
4. Error getting logs, for example: error from daemon in stream: Error grabbing logs: json: cannot unmarshal string into Go value of type jsonlog.JSONLog

Describe the results you expected:
Command should print

stdout
stderr