Skip to content

Update go files to add timestamps to gcluster logs#5198

Merged
agrawalkhushi18 merged 5 commits intoGoogleCloudPlatform:developfrom
agrawalkhushi18:enhanced-logging
Feb 12, 2026
Merged

Update go files to add timestamps to gcluster logs#5198
agrawalkhushi18 merged 5 commits intoGoogleCloudPlatform:developfrom
agrawalkhushi18:enhanced-logging

Conversation

@agrawalkhushi18
Copy link
Copy Markdown
Contributor

@agrawalkhushi18 agrawalkhushi18 commented Feb 5, 2026 •
edited
Loading

This PR includes changes to file significantly improve the observability, trace, and debuggability of the gcluster tool by implementing consistent RFC3339 UTC timestamps across all log outputs. This includes both the application's internal logs and the real-time output from external tools orchestrated by gcluster, such as terraform and packer.

Key technical changes:

  • pkg/logging/logging.go: Added a new public variable TsColor exported as color.New(color.FgMagenta) from github.com/fatih/color to ensure a distinctly colored magenta terminal appearance for all injected timestamps, making lines easier to read.
  • pkg/shell/writer.go: Added newTimestampWriter, an io.Writer decorator that buffers incoming byte slices and scans for newline characters (\n). It safely parses partial line writes (such as those generated by progress bars) and prepends an evaluated magenta-colored timestamp at the start of every distinct line. Included mutual exclusion (sync.Mutex) to prevent concurrency issues across output streams.
  • pkg/shell/terraform.go: Wrapped the external Terraform command's Stdout and Stderr bindings with newTimestampWriter when executing commands in standard logging situations to get unified timestamping across standard output formats.
  • pkg/shell/packer.go: Similarly wrapped internal Packer stream bindings to inject prefix timestamps during execution.
  • pkg/shell/writer_test.go: Added unit tests leveraging gopkg.in/check.v1 suite tests to comprehensively cover buffer chunking, line breaks, and validating correct RFC3339 timestamp prepending behavior ensuring full (100%) test coverage.

Impact:
All relevant shell output generated by the toolkit will now be systematically timestamped, providing a significant boost to debugging and profiling long-running infrastructure deployments.

Submission Checklist

NOTE: Community submissions can take up to 2 weeks to be reviewed.

Please take the following actions before submitting this pull request.

  • Fork your PR branch from the Toolkit "develop" branch (not main)
  • Test all changes with pre-commit in a local branch #
  • Confirm that "make tests" passes all tests
  • Add or modify unit tests to cover code changes
  • Ensure that unit test coverage remains above 80%
  • Update all applicable documentation
  • Follow Cluster Toolkit Contribution guidelines #

@agrawalkhushi18 agrawalkhushi18 added enhancement New feature or request go Pull requests that update Go code release-improvements Added to release notes under the "Improvements" heading. labels Feb 5, 2026
@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist bot commented Feb 5, 2026

Summary of Changes

Hello @agrawalkhushi18, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the observability and debuggability of the gcluster tool by standardizing log outputs across the application. It ensures that both internal application logs and the real-time outputs from orchestrated external tools, such as Terraform and Packer, are consistently prepended with RFC3339 UTC timestamps. This change provides a unified and easily traceable timeline for all events, making it much simpler to diagnose issues and understand the flow of operations.

Highlights

  • Consistent Timestamps for Logs: All core logging functions (Info, Error, Fatal) in pkg/logging/logging.go now automatically prepend RFC3339 UTC timestamps to their output, enhancing log clarity and debuggability.
  • New Timestamp Writer for External Tool Output: A new timestampWriter has been introduced in pkg/shell/terraform.go to consistently add RFC3339 UTC timestamps to multi-line and partial outputs from external tools, ensuring proper formatting even for progress bars or fragmented writes.
  • Integration with Packer and Terraform: The outputs from external Packer and Terraform commands are now piped through the new timestampWriter, ensuring that all standard output and error streams from these tools are consistently timestamped.
Changelog
  • pkg/logging/logging.go
    • Added time and github.com/fatih/color imports.
    • Introduced tsColor for magenta-colored timestamps.
    • Implemented formatTs() function to generate colored RFC3339 UTC timestamps.
    • Modified Info, Error, and Fatal functions to automatically prepend timestamps to their messages.
  • pkg/shell/packer.go
    • Changed outBuf and errBuf variable types from io.ReadWriter to io.Writer.
    • Integrated newTimestampWriter for os.Stdout and os.Stderr when printToScreen is enabled, ensuring Packer outputs are timestamped.
    • Updated error handling for io.Copy when printToScreen is false, adding type assertions for bytes.Buffer to ensure compatibility with the io.Writer interface.
  • pkg/shell/terraform.go
    • Added bytes, io, sync, time, and github.com/fatih/color imports.
    • Implemented a timestampWriter struct and its Write method to handle line-by-line timestamping, including partial writes, with mutex protection for concurrent safety.
    • Applied newTimestampWriter to tf.SetStdout and tf.SetStderr in applyPlanJsonOutput and applyPlanConsoleOutput to timestamp Terraform outputs.
Activity
  • The author, agrawalkhushi18, created this pull request with a detailed description outlining the changes and their benefits.
  • A submission checklist was included in the PR description, indicating the author's adherence to contribution guidelines.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 5, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly enhances the observability and debuggability of the gcluster tool by introducing consistent RFC3339 UTC timestamps to all log outputs. The implementation includes updates to core logging functions and a new timestampWriter to handle external tool outputs, such as Terraform and Packer. The changes are well-structured and address a clear need for improved logging. However, there are a few areas related to efficiency and adherence to Go's io.Writer interface contract that could be improved. Specifically, the repeated creation of color objects can be optimized, and the Write method's return value should accurately reflect bytes written in case of an error.

@agrawalkhushi18 agrawalkhushi18 marked this pull request as ready for review February 10, 2026 06:57
@agrawalkhushi18 agrawalkhushi18 requested review from a team and samskillman as code owners February 10, 2026 06:57
kadupoornima
kadupoornima reviewed Feb 10, 2026
View reviewed changes
kadupoornima
kadupoornima reviewed Feb 10, 2026
View reviewed changes
kadupoornima
kadupoornima previously approved these changes Feb 10, 2026
View reviewed changes
@agrawalkhushi18
Copy link
Copy Markdown
Contributor Author

agrawalkhushi18 commented Feb 12, 2026 •
edited
Loading

Tested the changes by triggering few gke and slurm tests:
gke-g4 - Build_id
gke-a3-ultragpu-onspot- Build_id
ml-h4d-onspot-slurm - Build_id
ml-g4-onspot-slurm - Build_id

@agrawalkhushi18 agrawalkhushi18 merged commit 45d7b66 into GoogleCloudPlatform:develop Feb 12, 2026
23 of 87 checks passed
kadupoornima pushed a commit to kadupoornima/cluster-toolkit that referenced this pull request Feb 17, 2026
@agrawalkhushi18 @kadupoornima
Update go files to add timestamps to gcluster logs (GoogleCloudPlatfo…
42fe9f9 
…rm#5198)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vikramvs-gg vikramvs-gg vikramvs-gg approved these changes

@kadupoornima kadupoornima kadupoornima approved these changes

@samskillman samskillman Awaiting requested review from samskillman samskillman is a code owner

@shubpal07 shubpal07 Awaiting requested review from shubpal07

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

enhancement New feature or request go Pull requests that update Go code release-improvements Added to release notes under the "Improvements" heading.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@agrawalkhushi18 @vikramvs-gg @kadupoornima