Skip to content

Modify klog to be verbose when controller log-level is set to debug#2221

Merged
dadjeibaah merged 2 commits intomasterfrom
dad/klogs
Feb 7, 2019
Merged

Modify klog to be verbose when controller log-level is set to debug#2221
dadjeibaah merged 2 commits intomasterfrom
dad/klogs

Conversation

@dadjeibaah
Copy link
Contributor

The controller logs innocuous messages when control plane proxies aren't ready to route requests during startup from each control plane component. i.e. tap, public-api and proxy-api. Setting the log level in the control plane to INFO would not hide these log messages and would still show up on control plane startup.

This PR modifies klogs initial flag set to route innocuous logs to /dev/null if the controller log level is set to INFO. If set to debug, we output all loglines to stderr.

Fixes #2171 #2168
Signed-off-by: Dennis Adjei-Baah [email protected]

klog.InitFlags(nil)
flag.Set("logtostderr", "true")
// -stderrthreshold=FATAL forces klog to only log FATAL errors to stderr.
// -logtostderr to false to not log to stderr by default.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll want to set -v to 0 here as well.

}
log.SetLevel(level)

klog.SetOutput(ioutil.Discard)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason to do this. But you will want to set the log file to /dev/null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While working on this I found that any flags that were set in this method would not have any effect on the logs because we would be modifying flags after flags.Parse() was called. So setting the log_file to /dev/null would not work here. Setting the output to ioutil.DIscardseemed to work.

Copy link
Contributor

@grampelberg grampelberg Feb 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it is because you created the second flag set. This works:

func ConfigureAndParse() {
	// -stderrthreshold=FATAL forces klog to only log FATAL errors to stderr.
	// -logtostderr to false to not log to stderr by default.

	klog.InitFlags(nil)
	flag.Set("stderrthreshold", "FATAL")
	flag.Set("logtostderr", "false")
	flag.Set("log_file", "/dev/null")
	flag.Set("v", "0")
	logLevel := flag.String("log-level", log.InfoLevel.String(),
		"log level, must be one of: panic, fatal, error, warn, info, debug")
	printVersion := flag.Bool("version", false, "print version and exit")

	flag.Parse()

	setLogLevel(*logLevel)
	maybePrintVersionAndExit(*printVersion)
}

func setLogLevel(logLevel string) {
	level, err := log.ParseLevel(logLevel)
	if err != nil {
		log.Fatalf("invalid log-level: %s", logLevel)
	}
	log.SetLevel(level)

	// Anything lower than the INFO level according to log is sent to /dev/null
	if level == log.DebugLevel {
		flag.Set("stderrthreshold", "INFO")
        	flag.Set("logtostderr", "true")
		flag.Set("v", "10")
	}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! Thanks! 👍

// Anything lower than the INFO level according to log is sent to /dev/null
if level == log.DebugLevel {
// Set stderr to INFO severity see https://github.com/kubernetes/klog/issues/23
klog.SetOutputBySeverity("INFO", os.Stderr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget a -v of 8 or 10.

// Anything lower than the INFO level according to log is sent to /dev/null
if level == log.DebugLevel {
// Set stderr to INFO severity see https://github.com/kubernetes/klog/issues/23
klog.SetOutputBySeverity("INFO", os.Stderr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just -stderrthreshold to info and that's good enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same deal here as my previous comment pointed out.

Copy link
Member

@siggy siggy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fixes! Confirmed works for me locally.

Copy link
Contributor

@grampelberg grampelberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me!

Copy link
Contributor

@klingerf klingerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⭐️ 🚀 Great, this will be a huge improvement for our controller pods.

if level == log.DebugLevel {
flag.Set("stderrthreshold", "INFO")
flag.Set("logtostderr", "true")
flag.Set("v", "10")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL you can call flag.Set after flag.Parse and it will still work. I wasn't expecting that, but have verified that this implementation does what you want it to do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, it frightens me a little bit. There be dragons there =)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants