Skip to content

Set stderr to empty string when using terminal on Windows#10412

Merged
samuelkarp merged 1 commit intocontainerd:mainfrom
TinaMor:tinamor/dev
Jul 15, 2024
Merged

Set stderr to empty string when using terminal on Windows#10412
samuelkarp merged 1 commit intocontainerd:mainfrom
TinaMor:tinamor/dev

Conversation

@TinaMor
Copy link

@TinaMor TinaMor commented Jul 2, 2024

PR Description

Windows requires that stderr is an empty string when running a container with TTY. In Containerd, pkg/cio/io.go#L298:L302, the Stderr is set to a value, e.g.

res = "binary:///C:/_xxx_/nerdctl.exe?_NERDCTL_INTERNAL_LOGGING=C%3A%5CProgramData%5Cnerdctl%5C052055e3"

which causes microsoft/hcsshim service_internal.go#L127 to fail to create a new task.

To address this issue, we refactor TerminalLogURI() and TerminalBinaryIO functions by adding an optional parameter that specifies whether stderr should be set to an empty string when the terminal is used. This change will help avoid the "failed precondition" error when creating a new task on Windows.

Bug reference:
#10415 : Refactor TerminalLogURI() to Support Optional Stderr Configuration for Terminal Compatibility on Windows

Related issues:
nerdctl bug #2966

@k8s-ci-robot
Copy link

Hi @TinaMor. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@TinaMor TinaMor marked this pull request as ready for review July 2, 2024 14:34
@MikeZappa87
Copy link
Member

My main concern with this is if we are making it harder to troubleshoot any issues between containerd->shim->runhcs

pkg/cio/io.go Outdated
// TerminalBinaryIO forwards container STDOUT|STDERR directly to a logging binary
// It also sets the terminal option to true
func TerminalBinaryIO(binary string, args map[string]string) Creator {
func TerminalBinaryIO(binary string, args map[string]string, setStderr ...bool) Creator {
Copy link
Member

@dcantah dcantah Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably just have a model similar to NewFIFOSetInDir where this is already handled. The unix version (and this is implying that we should split this into OS specific versions) doesn't do anything fancy, and the windows version has a special case for the terminal bool we pass in.

Unix:

// NewFIFOSetInDir returns a new FIFOSet with paths in a temporary directory under root
func NewFIFOSetInDir(root, id string, terminal bool) (*FIFOSet, error) {
	if root != "" {
		if err := os.MkdirAll(root, 0700); err != nil {
			return nil, err
		}
	}
	dir, err := os.MkdirTemp(root, "")
	if err != nil {
		return nil, err
	}
	closer := func() error {
		return os.RemoveAll(dir)
	}
	return NewFIFOSet(Config{
		Stdin:    filepath.Join(dir, id+"-stdin"),
		Stdout:   filepath.Join(dir, id+"-stdout"),
		Stderr:   filepath.Join(dir, id+"-stderr"),
		Terminal: terminal,
	}, closer), nil
}

Windows:

// NewFIFOSetInDir returns a new set of fifos for the task
func NewFIFOSetInDir(_, id string, terminal bool) (*FIFOSet, error) {
	stderrPipe := ""
	if !terminal {
		stderrPipe = fmt.Sprintf(`%s\ctr-%s-stderr`, pipeRoot, id)
	}
	return NewFIFOSet(Config{
		Terminal: terminal,
		Stdin:    fmt.Sprintf(`%s\ctr-%s-stdin`, pipeRoot, id),
		Stdout:   fmt.Sprintf(`%s\ctr-%s-stdout`, pipeRoot, id),
		Stderr:   stderrPipe,
	}, nil), nil
}

Copy link
Author

@TinaMor TinaMor Jul 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For backward compatibility, we want to avoid introducing a new required argument that could potentially disrupt other applications using the method.

cc. @samuelkarp

Copy link
Member

@dcantah dcantah Jul 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the function always sets terminal to true, and on windows we require that stderr is not filled in, why do we need a new parameter at all? The Windows version of the method can assign stderr to empty string, and unix can get away with whatever it does today I'd think.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored.

@dcantah
Copy link
Member

dcantah commented Jul 2, 2024

Can you preface in the title this is mainly for windows?

@fuweid fuweid added the platform/windows Windows label Jul 2, 2024
@TinaMor
Copy link
Author

TinaMor commented Jul 3, 2024

My main concern with this is if we are making it harder to troubleshoot any issues between containerd->shim->runhcs

@MikeZappa87

The current implementation redirects all output to a single stream, unifying stderr and stdout.

containerd/pkg/cio/io.go

Lines 261 to 262 in 1fb1882

Stdout: uri.String(),
Stderr: uri.String(),

This already matches Windows' default behavior, which combines both streams. However, Windows HCSShim requires that stderr is an empty string and is designed to check this specific precondition. If it detects that stderr is not an empty string, it will fail the operation.

@TinaMor TinaMor changed the title Set stderr to empty string when using terminal Set stderr to empty string when using terminal on Windows Jul 3, 2024
@k8s-ci-robot k8s-ci-robot added size/L and removed size/S labels Jul 3, 2024
@TinaMor TinaMor force-pushed the tinamor/dev branch 4 times, most recently from 83d5757 to 5e91c7a Compare July 8, 2024 06:20
@samuelkarp samuelkarp changed the title Set stderr to empty string when using terminal on Windows [Windows] Set stderr to empty string when using terminal on Windows Jul 9, 2024
@samuelkarp
Copy link
Member

/ok-to-test

@samuelkarp
Copy link
Member

/retest

@samuelkarp samuelkarp added this pull request to the merge queue Jul 9, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jul 9, 2024
[Windows] Set stderr to empty string when using terminal on Windows
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Jul 9, 2024
Copy link

@CharityKathure CharityKathure left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MikeZappa87
Copy link
Member

@mikebrow when you are free :-)

Copy link
Member

@mikebrow mikebrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mikebrow mikebrow added this pull request to the merge queue Jul 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 15, 2024
@samuelkarp samuelkarp added this pull request to the merge queue Jul 15, 2024
Merged via the queue into containerd:main with commit 0262714 Jul 15, 2024
@samuelkarp samuelkarp changed the title [Windows] Set stderr to empty string when using terminal on Windows Set stderr to empty string when using terminal on Windows Aug 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants