Skip to content

Write stack dump to os.TempDir() as well #3190

Merged
estesp merged 1 commit intocontainerd:masterfrom
lowenna:jjh/stack2file
Apr 18, 2019
Merged

Write stack dump to os.TempDir() as well #3190
estesp merged 1 commit intocontainerd:masterfrom
lowenna:jjh/stack2file

Conversation

@lowenna
Copy link
Copy Markdown

@lowenna lowenna commented Apr 5, 2019

Signed-off-by: John Howard [email protected]

As it says in the title. @jterry75 PTAL.

Comment thread cmd/containerd/command/main.go Outdated
@lowenna lowenna changed the title Windows:Write stack dump to %TEMP% file as well Write stack dump to os.TempDir() as well Apr 6, 2019
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #3190 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #3190   +/-   ##
======================================
  Coverage    45.2%   45.2%           
======================================
  Files         111     111           
  Lines       11982   11982           
======================================
  Hits         5416    5416           
  Misses       5732    5732           
  Partials      834     834
Flag Coverage Δ
#linux 49.23% <ø> (ø) ⬆️
#windows 40.58% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d77a5bf...c0f8b46. Read the comment docs.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Apr 6, 2019

Codecov Report

Merging #3190 into master will decrease coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3190      +/-   ##
==========================================
- Coverage   44.63%   44.62%   -0.01%     
==========================================
  Files         113      113              
  Lines       12161    12161              
==========================================
- Hits         5428     5427       -1     
- Misses       5898     5899       +1     
  Partials      835      835
Flag Coverage Δ
#linux 48.65% <ø> (ø) ⬆️
#windows 39.85% <ø> (-0.01%) ⬇️
Impacted Files Coverage Δ
runtime/v2/shim/shim_windows.go 37.93% <0%> (-0.58%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b819d05...7718d06. Read the comment docs.

Copy link
Copy Markdown
Contributor

@jterry75 jterry75 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread cmd/containerd/command/main.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we log the location where the dump is written to?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also wondering if we should add something random (or a time stamp) to the name, as Pid might be reused

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to just use https://golang.org/pkg/io/ioutil/#TempFile . That adds the random component and does the create.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah @dmcgowan Randomising it is actually the complete opposite of what I was deliberately trying to achieve....

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should probably explain the rationale here - getting stacks out is really a pretty advanced thing to do. You really only care about the stacks at this point in time, usually to investigate a deadlock. For that reason, in many ways even using the PID here is overkill.

Comment thread cmd/containerd/command/main.go Outdated
Copy link
Copy Markdown
Member

@estesp estesp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This makes sense to me as-is; it is writing files in the temp dir of the OS, so will not cause long-term storage issues. My expectation is this is a debug "helper" given the only way to get stackdump now is to have info level logging (or higher) enabled before containerd start. This allows you to quickly get a stack dump into a separate file without a restart of containerd if your loglevel isn't high enough. I don't see the need for any more "randomness" in the filename as it is a manual step to force a stackdump (usually because of a hang), and so you know the PID, and the filename having the PID makes it easy to find.

@thaJeztah
Copy link
Copy Markdown
Member

thaJeztah commented Apr 10, 2019

Would #3190 (comment) be useful? Something along the lines of

logrus.Infof("goroutine stack dump written to %s", path)

@lowenna
Copy link
Copy Markdown
Author

lowenna commented Apr 10, 2019

Would #3190 (comment) be useful? Something along the lines of
logrus.Infof("goroutine stack dump written to %s", path)

@thaJeztah Updated

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (not a maintainer)

thanks!

@estesp
Copy link
Copy Markdown
Member

estesp commented Apr 16, 2019

@jhowardmsft your last push lost the initial / for the opening copyright comment, causing all kinds of confusion in CI :)

@lowenna
Copy link
Copy Markdown
Author

lowenna commented Apr 16, 2019

your last push lost the initial / for the opening copyright comment, causing all kinds of confusion in CI :)

@estesp Oooops. Fixed.

@estesp
Copy link
Copy Markdown
Member

estesp commented Apr 18, 2019

Please rebase on master to pick up the fix for AppVeyor CI

@lowenna
Copy link
Copy Markdown
Author

lowenna commented Apr 18, 2019

Please rebase on master to pick up the fix for AppVeyor CI

@estesp Done

Copy link
Copy Markdown
Member

@estesp estesp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@estesp estesp merged commit ffe0b69 into containerd:master Apr 18, 2019
@lowenna lowenna deleted the jjh/stack2file branch April 18, 2019 19:46
@jterry75
Copy link
Copy Markdown
Contributor

@kevpar - Can we verify that we dont write this file on a etw capture state?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants