Skip to content

Honor tail selection when container log capture starts asynchronously #305

Open

Description

Problem

The first container log request can return more history than requested by tail=N because runtime capture starts asynchronously. This is a pre-existing V1 limitation also present in the V2 runtime-capture implementation under development in #295. Track it separately rather than expanding that PR.

Code context

The runtime capture is started with follow enabled:

CaptureContainerLogs(..., containers.StreamContainerLogsOptions{
    Follow: true,
    Timestamps: true,
})

Docker/Podman return after launching the capture process, before historical output is necessarily written. The response path immediately opens the capture file and applies:

if opts.Tail != nil {
    source = usvc_io.NewTailReader(source, int(*opts.Tail))
}

TailReader selects the last N lines at its first EOF and then delegates future reads directly to the underlying file. EOF on a still-growing capture file does not establish that runtime history has finished arriving. Relevant code: internal/logs/containerlogs/container_logstreamer.go, internal/docker/cli_orchestrator.go, internal/podman/cli_orchestrator.go, and pkg/io/tail_reader.go. V2's analogous path is internal/logs/v2containerlogs/container_logstreamer.go in #295.

Reproduction and evidence

A focused probe opened an empty capture with TailReader(..., 1), read its initial EOF, then appended the delayed historical batch:

first
second
third

Subsequent reads returned all three lines, not only third. That reproduces the reader behavior combined with the existing asynchronous capture contract; it is not an in-the-wild incident report. Prewarmed captures or synchronous simulated capture can hide the problem.

Expected behavior

The first request must honor the requested historical tail even when capture history arrives asynchronously. In follow mode, subsequent live lines should continue after that history, without loss or duplication. Non-follow requests should return the requested available history rather than depending on arbitrary scheduling delays.

Design constraints and validation

Establish a reliable history/live boundary before implementing the fix. Possibilities to evaluate include runtime-side tail selection and a finite historical capture followed by a well-defined live handoff; neither is chosen by this issue. A sleep or waiting for the first byte is not evidence that the entire history has arrived. Preserve shared-capture behavior and existing V1/V2 patterns.

Cover a first request against delayed, multi-batch history; follow and non-follow behavior; preexisting captures; and independent logical/physical consumers. Keep this work separate from watcher ordering, container lifecycle, and image pull policy changes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions