Merge epic/file-schema-v6 into main by kyasbal · Pull Request #787 · GoogleCloudPlatform/khi

kyasbal · 2026-06-18T15:44:56Z

Merge the long running epic branch epic/file-schema-v6 into the main branch to prepare releasing 0.56.0.
This epic branch has dozens of change including some breaking changes. It originally started as file schema change then it changed the domain model used in frontend. The data structure change made necessity to restructure how it should be visualized or filtered and also allowed us to improve frontend side quite easier.

Here are the list of notable changes included in this merge PR:

Breaking change

.khi file format structure change

Previously .khi file format used JSON for storing its metadata and gzipped raw array buffer for long text part. This limits the maximum size of the logs due to the max string length accepted on V8.
This version changed .khi to use protocol buffer based file schema. It allows performant serialization/deserialization while we maintain file schema interface across backend and frontend.
Protocol buffer has limitation on the max size per single message, thus KHI concatenate multiple protos on simple KHI's container file format.

Comparison by file format update

scenario	log volume	old format size	new format size	reduction ratio
basic scenario	223464 logs	128.2MB	19.4MB	x 6.6
larger scenario	1386843 logs	885.3MB	144.4MB	x6.1

This is not only for the file size issue. KHI needs decompress string data from the file at the beginning of the load to perform filtering and consume a lot of memory(The larger scenario data with old format couldn't be loaded on my environment.) This allows frontend to open larger log data.

Note that even if we support larger log volume, still it's recommended to set appropriate filter to query. Larger logs takes longer time to filter and the inspection experience would be worse.

UI changes

More types of timelines

KHI was initially designed only for visualizing Kubernetes logs. Thus timeline layer count was fixed and it always has the structure APIVersion -> Kind -> Namespace -> Resource -> SubResource.
However, now we have a lot of types of visualization are placed with hierarchical relationships such as .status.conditions or .metadata.ownerReference. We recently added support for CSM TD resources or Managed airflow. These were needed to be placed on timeline with mapping their concept to kubernetes concept forcibly.
New change allows defining new timeline type for each parsers and the count of layers of timeline hierarchy is not limited.

With this change, Kubernetes resource hierarchy was changed to cluster name -> api version -> kind -> namespace -> resource -> subresource with aiming to support multi clusters.

New timeline filters

By the change of timeline types, KHI no longer guarantee given data contains namespace or kind timeline. It needed more widely applicable filtering mechanism.

CEL based filtering

The new filtering system is based on CEL based highly flexible filtering system. Now advanced user can input the filter directly. One of the example of this usage is to filter timelines by its manifest property.
The screenshot captures filtering timelines which has status.qosClass = Burstable. This allows us to find resources with more complex conditions.

Severity filtering

KHI hasn't supported severity based filtering. This was not from the architectural limit, but more like my own ideological policy. But now I supported it as there were many requests on it.
But note that severities in distributed system sometimes become misleading because each component decides its severity from its own perspective. Severity is not a universal criteria of logs.

Log body search and diff body search

Our log or diff components hasn't provided searching capability. It lead user to use in-browser search but it sometimes hit other elements not on the logs or the manifest.

Graph view integration on the dockable window

hevc.mov

Now KHI's graph view is integrated on the dockable window. It allows us to see the diagram along with the timeline.

… migration (GoogleCloudPlatform#618) * feat(proto): add proto definitions and build setup for file schema v7 migration - Add Protobuf definitions for KHI file schema v7 (split into shared.proto, style.proto, timeline.proto, etc.) to migrate from the previous JSON-based file format. - Set up Buf configuration (buf.yaml, buf.gen.yaml) to generate Go and TypeScript code. - Add `proto-gen` target to `scripts/make/codegen.mk` using `npx @bufbuild/buf` to support the new build process. - Generate Go code in `pkg/generated/khifile/` and TypeScript code in `web/src/app/generated/`. * fixed typos on proto fields * Specified proto rule and build target * Rebuild the generated files from the updated proto * Added linguist-generated for ignoreing generated files by proto * fix reviewed points and lint error * fix version number * Skip proto breaking change check until it's merged * Fixed shared type names * fix typos on comments

…form#623)

…m#621) * feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator

* feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator * Adding generated files I forgot to push

* feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator * Adding generated files I forgot to push * Fix the lint thing on the generated codes

…oudPlatform#628)

…orm#622) * feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator * Adding generated files I forgot to push * Fix the lint thing on the generated codes * Adding interning utility used in khi file format v6

…CloudPlatform#624)

…m#631)

…oogleCloudPlatform#633) khifile

…oogleCloudPlatform#632)

…m#630) * feat(web): add v6 parser binary reader and streamer * address review comments

…ogleCloudPlatform#634)

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…dPlatform#656)

…loudPlatform#650)

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix bugs to register the CSM TD parser tasks (GoogleCloudPlatform#647) * Fix the same issue of GoogleCloudPlatform#638 (GoogleCloudPlatform#659) * Adding make clean command (GoogleCloudPlatform#651) * Addming make clean command * fixed commented points * Removing unused old headers on frontend (GoogleCloudPlatform#660) * removed prefix 'v2' in shader files (GoogleCloudPlatform#661) * rename commonlogk8sauditv2 to commonlogk8saudit (GoogleCloudPlatform#662) --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix bugs to register the CSM TD parser tasks (GoogleCloudPlatform#647) * Fix the same issue of GoogleCloudPlatform#638 (GoogleCloudPlatform#659) * Adding make clean command (GoogleCloudPlatform#651) * Addming make clean command * fixed commented points * Removing unused old headers on frontend (GoogleCloudPlatform#660) * removed prefix 'v2' in shader files (GoogleCloudPlatform#661) * rename commonlogk8sauditv2 to commonlogk8saudit (GoogleCloudPlatform#662) * fixed the flaky backend test due to race condition (GoogleCloudPlatform#665) * improved sticky header to show resource layer (GoogleCloudPlatform#663) --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

This type is duplication of the pb.InternedStruct. It's internal type of the domain store but its a shared proto type which is expected not to change by file version update.

…yles (GoogleCloudPlatform#672)

…loudPlatform#673) registration

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix bugs to register the CSM TD parser tasks (GoogleCloudPlatform#647) * Fix the same issue of GoogleCloudPlatform#638 (GoogleCloudPlatform#659) * Adding make clean command (GoogleCloudPlatform#651) * Addming make clean command * fixed commented points * Removing unused old headers on frontend (GoogleCloudPlatform#660) * removed prefix 'v2' in shader files (GoogleCloudPlatform#661) * rename commonlogk8sauditv2 to commonlogk8saudit (GoogleCloudPlatform#662) * fixed the flaky backend test due to race condition (GoogleCloudPlatform#665) * improved sticky header to show resource layer (GoogleCloudPlatform#663) * Removed redundant v from the version string (GoogleCloudPlatform#671) * bugfix: automatic log scroll is not matching to the actual element (GoogleCloudPlatform#675) * bugfix: ignore changing selected timeline when user clicked a log from log list (GoogleCloudPlatform#674) * bugfix: automatic scroll to the beginning time of log query when user selected a timeline without point to a log (GoogleCloudPlatform#676) * bugfix: scaling around right edge of query duration may be clipped to jump to other timing (GoogleCloudPlatform#677) --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

… are associated with the same log (GoogleCloudPlatform#772) * bug: fix unintentionally jump to another timeline when 2 or more logs are associated with the same log * Addressed review comments for PR 772

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix bugs to register the CSM TD parser tasks (GoogleCloudPlatform#647) * Fix the same issue of GoogleCloudPlatform#638 (GoogleCloudPlatform#659) * Adding make clean command (GoogleCloudPlatform#651) * Addming make clean command * fixed commented points * Removing unused old headers on frontend (GoogleCloudPlatform#660) * removed prefix 'v2' in shader files (GoogleCloudPlatform#661) * rename commonlogk8sauditv2 to commonlogk8saudit (GoogleCloudPlatform#662) * fixed the flaky backend test due to race condition (GoogleCloudPlatform#665) * improved sticky header to show resource layer (GoogleCloudPlatform#663) * Removed redundant v from the version string (GoogleCloudPlatform#671) * bugfix: automatic log scroll is not matching to the actual element (GoogleCloudPlatform#675) * bugfix: ignore changing selected timeline when user clicked a log from log list (GoogleCloudPlatform#674) * bugfix: automatic scroll to the beginning time of log query when user selected a timeline without point to a log (GoogleCloudPlatform#676) * bugfix: scaling around right edge of query duration may be clipped to jump to other timing (GoogleCloudPlatform#677) * Updated CODEOWNER file (GoogleCloudPlatform#678) * Updated the release.yml for GitHub (GoogleCloudPlatform#679) * fix: confirm refresh when backend disconnects (GoogleCloudPlatform#692) * fix: confirm refresh when backend disconnects Signed-off-by: Haihan Jiang <[email protected]> * chore: use absolute imports in root tests Signed-off-by: Haihan Jiang <[email protected]> * Align unload confirmation with disconnect review --------- Signed-off-by: Haihan Jiang <[email protected]> * Handle slog attributes in KHI log formatter (GoogleCloudPlatform#708) * Handle slog attributes in KHI log formatter * Optimize KHI log attribute formatting * Handle KHI log formatter review feedback * Change default listen host to 127.0.0.1 from localhost (GoogleCloudPlatform#759) * Improve retry logic to consider temporal failure due to ECP issue (GoogleCloudPlatform#760) * Improve retry logic to consider temporal failure due to ECP issue * fixed GCA's reviewed points --------- Signed-off-by: kyasbal <[email protected]> Signed-off-by: Haihan Jiang <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Haihan Jiang <[email protected]>

…#777) * Optimize log lookup and resolve sticky header overlay placement - Resolve target timeline fallback order (selectedTimeline -> descendants -> all timelines) - Compensate hover placement offset when hovering directly over the sticky header - Optimize preceding and succeeding log extraction using bisectLeft - Standardize timestamps to bigint (nanoseconds) across hover resolution * Visualize current cursor time with a dedicated separator line - Propagate cursor time as bigint (nanoseconds) to the hover overlay - Update lastClickedTime dynamically on mouse movement across empty timeline periods - Conditionally insert a dedicated cursor separator row (isCursor: true) only when hovering over empty timeline periods - Render cursor row with flex container and CSS dashed lines flanking centered text - Enforce dashed lines expansion with flex: 1 1 20px and height: 1px - Update Storybook stories to include mock cursorTime property * addressed comments on PR777

GoogleCloudPlatform#776) * feat: show RevisionState description popup on hover in timeline legend * addressed comments on PR776

* feat: embed graph view as a GoldenLayout tab * Remove diagram button from the toolbar * addressed PR comments on GoogleCloudPlatform#780

* make storybook not freezed by serialization * addressed PR781 review comments

* Make graph draw feature async * addressed PR782 review comments

* Removing unused CORS code (GoogleCloudPlatform#619) * Removing unused CORS code * Update pkg/core/init/default/defaultextension.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Replaced unsafe innerHTML with textContent (GoogleCloudPlatform#620) * feat(startup): implement smart component and integrate data loading (GoogleCloudPlatform#599) * feat(startup): add inspection list components * feat(startup): implement smart component and integrate data loading GoogleCloudPlatform#558 * Fix name of the function pointed out by the review * chore: fix make setup error in jj and update dev guide (GoogleCloudPlatform#629) * chore: fix make setup error in jj and update dev guide * fix commented issues on review * fixed minor issues on Makefile (GoogleCloudPlatform#644) * Merge epic/csm-parser into main (GoogleCloudPlatform#645) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name (GoogleCloudPlatform#635) * Add fleet PJ form task and query task to gather CSM TD audit logs (GoogleCloudPlatform#636) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for TD resource logs (GoogleCloudPlatform#637) * Add Inventory/Discovery tasks for parsing the map from NEG name to BS name * Adding form task to receive the fleet project and query task to gather logs from Cloud Logging * Add timeline mapper for traffic director resource logs * 🐛 fix set-input suggestion being wrapped for every lines (GoogleCloudPlatform#638) * fix set-input suggestion being wrapped for every lines * Update web/src/app/timeline-toolbar/components/set-input-popup.component.scss Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: kyasbal <[email protected]> --------- Signed-off-by: kyasbal <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix bugs to register the CSM TD parser tasks (GoogleCloudPlatform#647) * Fix the same issue of GoogleCloudPlatform#638 (GoogleCloudPlatform#659) * Adding make clean command (GoogleCloudPlatform#651) * Addming make clean command * fixed commented points * Removing unused old headers on frontend (GoogleCloudPlatform#660) * removed prefix 'v2' in shader files (GoogleCloudPlatform#661) * rename commonlogk8sauditv2 to commonlogk8saudit (GoogleCloudPlatform#662) * fixed the flaky backend test due to race condition (GoogleCloudPlatform#665) * improved sticky header to show resource layer (GoogleCloudPlatform#663) * Removed redundant v from the version string (GoogleCloudPlatform#671) * bugfix: automatic log scroll is not matching to the actual element (GoogleCloudPlatform#675) * bugfix: ignore changing selected timeline when user clicked a log from log list (GoogleCloudPlatform#674) * bugfix: automatic scroll to the beginning time of log query when user selected a timeline without point to a log (GoogleCloudPlatform#676) * bugfix: scaling around right edge of query duration may be clipped to jump to other timing (GoogleCloudPlatform#677) * Updated CODEOWNER file (GoogleCloudPlatform#678) * Updated the release.yml for GitHub (GoogleCloudPlatform#679) * fix: confirm refresh when backend disconnects (GoogleCloudPlatform#692) * fix: confirm refresh when backend disconnects Signed-off-by: Haihan Jiang <[email protected]> * chore: use absolute imports in root tests Signed-off-by: Haihan Jiang <[email protected]> * Align unload confirmation with disconnect review --------- Signed-off-by: Haihan Jiang <[email protected]> * Handle slog attributes in KHI log formatter (GoogleCloudPlatform#708) * Handle slog attributes in KHI log formatter * Optimize KHI log attribute formatting * Handle KHI log formatter review feedback * Change default listen host to 127.0.0.1 from localhost (GoogleCloudPlatform#759) * Improve retry logic to consider temporal failure due to ECP issue (GoogleCloudPlatform#760) * Improve retry logic to consider temporal failure due to ECP issue * fixed GCA's reviewed points * bug: fix CSM's access log query was wrong for GDC clusters (GoogleCloudPlatform#779) * bug: fix CSM's access log query was wrong for GDC clusters * addressed comment in GoogleCloudPlatform#779 --------- Signed-off-by: kyasbal <[email protected]> Signed-off-by: Haihan Jiang <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Haihan Jiang <[email protected]>

gemini-code-assist

Code Review

This pull request introduces the KHI v6 file format and serializer architecture, replacing legacy parser and mapper tasks with new v6 implementations (GroupedLogIngesterV2, LogIngesterV2, and LogToTimelineMapperV2) along with extensive documentation, style guides, and Protobuf configurations. The review feedback highlights a critical issue in container.go where io.LimitReader silently truncates oversized payloads, an optimization opportunity in field_filter.go to avoid wrapping nodes when no keys are filtered, and a discrepancy in metadata_accumulator.go where unknown metadata types fail with an error instead of falling back to JSON serialization as documented.

kyasbal and others added 30 commits April 20, 2026 09:13

feat: update InternedValue proto to have int64 value (GoogleCloudPlat…

3e7835e

…form#623)

Adding KHI file schema v6 namespaced ID generator (GoogleCloudPlatfor…

860dc50

…m#621) * feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator

Adding generated files I forgot to push (GoogleCloudPlatform#626)

375e1aa

* feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator * Adding generated files I forgot to push

Rolling back GoogleCloudPlatform#626 (GoogleCloudPlatform#627)

353dfc7

* feat: update InternedValue proto to have int64 value * Adding KHI file schema v6 namespaced ID generator * Adding generated files I forgot to push * Fix the lint thing on the generated codes

feat(web): add v6 parser foundation (errors and interfaces) (GoogleCl…

d9958d1

…oudPlatform#628)

feat(khifile): implement v6 chunk container writer and reader (Google…

37c1220

…CloudPlatform#624)

Adding serializer/deserializer for InternedStruct (GoogleCloudPlatfor…

73a1bc9

…m#631)

Add LogAccumulator storing all logs before serializing them into the (G…

60b0710

…oogleCloudPlatform#633) khifile

Add TimelinePath utilities and IDNamespace used for timeline types (G…

e6d82b2

…oogleCloudPlatform#632)

feat(web): add v6 parser binary reader and parser (GoogleCloudPlatfor…

8047026

…m#630) * feat(web): add v6 parser binary reader and streamer * address review comments

Added serializer for timeline registry to flush contents as proto (Go…

137f98b

…ogleCloudPlatform#634)

Added interned pool types and struct decoder (GoogleCloudPlatform#640)

c6a0fcc

feat: Adding TimelineStyleRegistry (GoogleCloudPlatform#639)

9c6b3db

Added style related domain stores (GoogleCloudPlatform#658)

bc5a6be

Adding metadata accumulator (GoogleCloudPlatform#646)

00302f9

Change input of style registration not to depend on proto (GoogleClou…

7bfa0de

…dPlatform#656)

Adding KHI file format spec and serializer/deserializer spec (GoogleC…

73f6816

…loudPlatform#650)

improve ts coding rule (GoogleCloudPlatform#667)

717152c

Removed InternedStuctDTO (GoogleCloudPlatform#664)

4fc3332

This type is duplication of the pb.InternedStruct. It's internal type of the domain store but its a shared proto type which is expected not to change by file version update.

add v6 khi file builder (GoogleCloudPlatform#652)

8caa934

Add listing methods in style-store (GoogleCloudPlatform#668)

431a633

Added log related domain stores (GoogleCloudPlatform#641)

409b25b

Updated the frontend-codegen to generate icon list from registered st…

9ba9b47

…yles (GoogleCloudPlatform#672)

Adding color helper functions and better validation on style (GoogleC…

b295be3

…loudPlatform#673) registration

kyasbal and others added 12 commits June 16, 2026 10:54

bug: fix unintentionally jump to another timeline when 2 or more logs…

da2872e

… are associated with the same log (GoogleCloudPlatform#772) * bug: fix unintentionally jump to another timeline when 2 or more logs are associated with the same log * Addressed review comments for PR 772

update descriptive messages used on KHI (GoogleCloudPlatform#774)

b23b5c0

feat: show RevisionState description popup on hover in timeline legend (

a0665e8

GoogleCloudPlatform#776) * feat: show RevisionState description popup on hover in timeline legend * addressed comments on PR776

Added diagram view to the golden layout tab (GoogleCloudPlatform#780)

160aa8a

* feat: embed graph view as a GoldenLayout tab * Remove diagram button from the toolbar * addressed PR comments on GoogleCloudPlatform#780

make storybook not freezed by serialization (GoogleCloudPlatform#781)

90c3828

* make storybook not freezed by serialization * addressed PR781 review comments

Make graph draw feature async (GoogleCloudPlatform#782)

2d4d194

* Make graph draw feature async * addressed PR782 review comments

removed unused frontend codes (GoogleCloudPlatform#783)

aa00e6a

Improved layout menu buttons (GoogleCloudPlatform#784)

b924c1b

Merge main into epic/file-schema-v6

22a1435

gemini-code-assist Bot reviewed Jun 18, 2026

View reviewed changes

Comment thread pkg/model/khifile/v6/container.go

Comment thread pkg/common/structured/field_filter.go

Comment thread pkg/model/khifile/v6/metadata_accumulator.go

kyasbal added the enhancement New feature or request label Jun 19, 2026

kyasbal linked an issue Jun 19, 2026 that may be closed by this pull request

Migrate KHI File Format to Protocol Buffers #425

Closed

7 tasks

kyasbal added this to the KHI file format v6 milestone Jun 19, 2026

This was linked to issues Jun 19, 2026

Improve Frontend Codebase Readability and Structure #419

Closed

Refactor pkg/model/enum to Support Decentralized Constant Registration #424

Closed

kyasbal marked this pull request as ready for review June 19, 2026 01:12

kyasbal requested review from K53, RyuSA, jyane, kkuchima and renamoo as code owners June 19, 2026 01:12

kyasbal self-assigned this Jun 19, 2026

renamoo approved these changes Jun 19, 2026

View reviewed changes

kyasbal added this pull request to the merge queue Jun 19, 2026

Merged via the queue into GoogleCloudPlatform:main with commit dff3574 Jun 19, 2026
15 checks passed

kyasbal deleted the epic/file-schema-v6 branch June 19, 2026 02:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merge epic/file-schema-v6 into main#787

Merge epic/file-schema-v6 into main#787
kyasbal merged 114 commits into
GoogleCloudPlatform:mainfrom
kyasbal:epic/file-schema-v6

kyasbal commented Jun 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kyasbal commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Breaking change

.khi file format structure change

Comparison by file format update

UI changes

More types of timelines

New timeline filters

CEL based filtering

Severity filtering

Log body search and diff body search

Graph view integration on the dockable window

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kyasbal commented Jun 18, 2026 •

edited

Loading