Skip to content

Commit 4e6ce46

Browse files
authored
feat(dashboard): add logs and metrics views to the console (#5218)
## What this PR does Adds logs and resource-consumption views to the console, backed by the observability proxy from the design proposals [Tenant application logs](cozystack/community#95) and [Tenant resource-consumption metrics](cozystack/community#94). This PR is useful only together with #5214, which adds the console nginx route to the proxy. The application tabs need #5219 and the per-application roots from its follow-up PR. Until those are installed, each view shows a quiet "not available on this cluster" state instead of an error, so the order of merging does not break the console. What is added: - A "Logs" page at the namespace level, under a new "Observability" sidebar group, reading the namespace logs root. It lives at `/console/-/observability/logs`: every one- or two-segment console path is an application list or detail page, and `-` cannot start an application plural or name, so the page never shadows an application's own pages. - "Logs" and "Metrics" tabs on application pages (every application kind except VMDisk, which runs no containers), reading the per-application roots. - Logs view: - time range, a LogsQL filter, and pod and container pickers that turn into a stream filter; - "Follow" opens a live tail. The status shows "Live" as soon as the stream is open, even before the first line; on a dropped stream or a load refusal (`429`) it reconnects from the newest line seen, with backoff; it stops on `400`/`401`/`403`/`404`; - the reader can scroll up while lines stream; the newest 1000 lines are kept; - while a new range or filter loads, the previous lines stay on screen with a note saying so, or the view says it is loading; it never reports "no lines" for a query it has not read yet. - Metrics view: per-pod CPU and memory charts and application-wide network charts, refreshed every minute, with ranges from 1 hour to 7 days. The `rate` window is at least 5 minutes and four steps, so queries routed to the long-term store (coarser deduplication) still return data. No query names a namespace or kind; the proxy adds the scope. - Error handling by proxy status: scope-wide answers (`401`/`403`/`404`) replace the whole view with one notice; limit answers keep the proxy's own advice. A Kubernetes `Status` body on a proxy path means the request reached kube-apiserver instead of the proxy and is shown as "not available". - Shared UI pieces: an accessible `Select` and a text `Input` matching the console's controls, and an `EmptyState`. - `K8sClient.raw()` for untyped requests over the same credential path as `/apis`; the client now keeps caller headers given in any form. - Charts use uPlot, loaded in a separate lazy chunk, so the main bundle does not grow. #### How it was tested - `pnpm typecheck`, `pnpm test`, `pnpm lint` and `pnpm build` pass. - Screenshots of the views were taken against a mock API and on a test cluster. - Verified on a test cluster through the full browser chain (OIDC login, gatekeeper, gateway, console nginx, proxy), as a user with the `view` role in one tenant: the Logs page works; Follow keeps a single tail open for more than two minutes without reconnecting; Follow on a quiet filter shows "Live"; the Metrics tab of a Redis instance draws its charts; requests from the same session to another tenant's roots answer `403`. - Not verified: a screen reader (roles and names are asserted in tests only) and viewports narrower than 1440px. #### Known limitation The first lines a newly started pod writes can be missing from an already open tail, although a history query shows them. This comes from the log store's live-tail window relative to log collection delay, not from the console. ### Screenshots <img width="1440" height="900" alt="cluster-follow-live" src="https://github.com/user-attachments/assets/6efd9bfb-f05b-4498-bd20-0af4e5cda01f" /> <img width="1440" height="900" alt="cluster-logs-empty" src="https://github.com/user-attachments/assets/673d8295-cd3d-409d-b871-290705d0a6a2" /> <img width="1440" height="900" alt="cluster-logs" src="https://github.com/user-attachments/assets/621c2312-8427-4353-b199-44bdd12b78f7" /> <img width="1440" height="900" alt="cluster-pod-select" src="https://github.com/user-attachments/assets/92ea9c72-4a3c-4562-9c3a-2170730ab085" /> <img width="1400" height="1000" alt="cluster-redis-metrics" src="https://github.com/user-attachments/assets/3533d069-01b9-41de-bba9-70b940d05c6e" /> <img width="1440" height="900" alt="follow-waiting" src="https://github.com/user-attachments/assets/eb318d13-5188-4632-842a-43c0bbbcff5a" /> <img width="1440" height="900" alt="logs-empty" src="https://github.com/user-attachments/assets/c3d89e81-7249-444d-a3fa-9ab44c1ca86e" /> <img width="1440" height="900" alt="logs" src="https://github.com/user-attachments/assets/ec6caa9c-5eda-4361-99ea-359e5d002f04" /> <img width="1440" height="900" alt="metrics-empty" src="https://github.com/user-attachments/assets/db16da15-a618-4dbe-a531-593bc9b916dc" /> ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Downstream: no trigger matched. Candidate: none affected, because the whole diff is inside the vendored console (`packages/system/dashboard/images/console`), which per the trigger map belongs to this repository and has no downstream copy. A user guide page for the new console views on the website would be useful but is not required by the trigger map. ### Release note ```release-note feat(dashboard): the console gains a namespace Logs page and Logs and Metrics tabs on application pages, with live log following, per-pod CPU and memory charts, and application-wide network charts. The views need the observability proxy; without it they show "not available on this cluster". ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Added Metrics and Logs tabs to supported application details, plus a tenant-wide Logs page in the sidebar. * View CPU, memory, and network charts over selectable time ranges, with optional sample tables and automatic refresh. * Search logs using pod and container filters, choose a history range, and follow live updates with connection status. * Added distinct loading, empty, and error messages for observability views, including guidance for unavailable services and query limits. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 17bbca2 + e123cef commit 4e6ce46

45 files changed

Lines changed: 3184 additions & 7 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/system/dashboard/images/console/CLAUDE.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Implications:
3232
- **RJSF** (`@rjsf/core` + `validator-ajv8`) for application config forms.
3333
- **Vitest + jsdom + @testing-library/react** for tests.
3434
- **Monaco** for YAML view, **noVNC** for the VM console tab.
35+
- **uPlot** for the Metrics tab charts, lazy-loaded with that tab.
36+
- Metrics and logs come from the observability proxy, not kube-apiserver: `lib/observability/proxy.ts` reads its native Prometheus JSON and VictoriaLogs NDJSON through `K8sClient.raw`, under the paths nginx routes to it ahead of `/apis`.
3537

3638
## Dev loop
3739

@@ -79,6 +81,7 @@ the apiserver checks that user's RBAC.
7981
to a chunked-encoding WATCH against the same `resourceVersion`. Don't
8082
poll. Don't add `refetchInterval`. If you need a one-shot, pass
8183
`{ watch: false }`.
84+
The one exception is the metrics panels in `components/observability`: the observability proxy serves the Prometheus query API, which has no watch, so they refresh on a fixed interval instead. Log lines stream through the tail endpoint and are not polled.
8285
4. **Tenant scoping.** Most resources live in `tenant-<name>` namespaces.
8386
Pull the active tenant from `useTenantContext()` — never read the
8487
namespace from a URL param or guess it.

‎packages/system/dashboard/images/console/apps/console/package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@
3131
"react": "^19.2.4",
3232
"react-dom": "^19.2.4",
3333
"react-router": "^7.6.1",
34-
"simple-icons": "^16.22.0"
34+
"simple-icons": "^16.22.0",
35+
"uplot": "^1.6.32"
3536
},
3637
"devDependencies": {
3738
"@tailwindcss/vite": "^4.2.2",
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
import { describe, it, expect, vi, afterEach } from "vitest"
2+
import { K8sClient } from "@cozystack/k8s-client"
3+
4+
afterEach(() => {
5+
vi.unstubAllGlobals()
6+
})
7+
8+
describe("K8sClient request headers", () => {
9+
it("lets a method's own content type replace the JSON default", async () => {
10+
const fetchMock = vi.fn(async () => new Response("{}"))
11+
vi.stubGlobal("fetch", fetchMock)
12+
const client = new K8sClient({ getToken: async () => "t0k" })
13+
14+
await client.patch("apps", "v1", "deployments", "web", { spec: {} }, "default", "merge")
15+
16+
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
17+
const headers = new Headers(init.headers)
18+
expect(headers.get("Content-Type")).toBe("application/merge-patch+json")
19+
expect(headers.get("Authorization")).toBe("Bearer t0k")
20+
})
21+
22+
it("sends JSON by default", async () => {
23+
const fetchMock = vi.fn(async () => new Response("{}"))
24+
vi.stubGlobal("fetch", fetchMock)
25+
26+
await new K8sClient().getApiGroups()
27+
28+
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
29+
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json")
30+
})
31+
})
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
import { describe, it, expect, vi, afterEach } from "vitest"
2+
import { K8sClient } from "@cozystack/k8s-client"
3+
4+
afterEach(() => {
5+
vi.unstubAllGlobals()
6+
vi.restoreAllMocks()
7+
})
8+
9+
describe("K8sClient.raw", () => {
10+
it("returns the response untouched, including a non-2xx one", async () => {
11+
const res = new Response("not found", { status: 404 })
12+
vi.stubGlobal("fetch", vi.fn(async () => res))
13+
const client = new K8sClient()
14+
15+
await expect(client.raw("/x")).resolves.toBe(res)
16+
})
17+
18+
it("prefixes the base URL and attaches the bearer token", async () => {
19+
const fetchMock = vi.fn(async () => new Response("{}"))
20+
vi.stubGlobal("fetch", fetchMock)
21+
const client = new K8sClient({ baseUrl: "/base", getToken: async () => "t0k" })
22+
23+
await client.raw("/apis/x", { headers: { Accept: "application/json" } })
24+
25+
const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
26+
expect(url).toBe("/base/apis/x")
27+
const headers = new Headers(init.headers)
28+
expect(headers.get("Accept")).toBe("application/json")
29+
expect(headers.get("Authorization")).toBe("Bearer t0k")
30+
})
31+
32+
it.each([
33+
["a Headers instance", new Headers({ Accept: "application/x-ndjson" })],
34+
["a tuple list", [["Accept", "application/x-ndjson"]] as [string, string][]],
35+
])("keeps caller headers passed as %s", async (_, given) => {
36+
const fetchMock = vi.fn(async () => new Response("{}"))
37+
vi.stubGlobal("fetch", fetchMock)
38+
const client = new K8sClient({ getToken: async () => "t0k" })
39+
40+
await client.raw("/x", { headers: given })
41+
42+
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
43+
const headers = new Headers(init.headers)
44+
expect(headers.get("Accept")).toBe("application/x-ndjson")
45+
expect(headers.get("Authorization")).toBe("Bearer t0k")
46+
})
47+
48+
it("routes a 401 through the client's unauthorized handler", async () => {
49+
const onUnauthorized = vi.fn()
50+
vi.stubGlobal("fetch", vi.fn(async () => new Response("", { status: 401 })))
51+
const client = new K8sClient({ onUnauthorized })
52+
53+
const res = await client.raw("/x")
54+
55+
expect(res.status).toBe(401)
56+
expect(onUnauthorized).toHaveBeenCalledTimes(1)
57+
})
58+
59+
it("does not swallow a network failure", async () => {
60+
vi.stubGlobal("fetch", vi.fn(async () => { throw new TypeError("offline") }))
61+
const client = new K8sClient()
62+
63+
await expect(client.raw("/x")).rejects.toThrow("offline")
64+
})
65+
})
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
import { describe, it, expect } from "vitest"
2+
import { render, screen } from "@testing-library/react"
3+
import { EmptyState } from "./EmptyState.tsx"
4+
5+
describe("EmptyState", () => {
6+
it("announces its message as a status", () => {
7+
render(<EmptyState title="No data for this range." hint="Pick a longer range." />)
8+
9+
expect(screen.getByRole("status")).toHaveTextContent("No data for this range.Pick a longer range.")
10+
})
11+
12+
it("marks itself busy while something is loading", () => {
13+
render(<EmptyState busy title="Loading…" />)
14+
15+
expect(screen.getByRole("status")).toHaveAttribute("aria-busy", "true")
16+
})
17+
})
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import type { ReactNode } from "react"
2+
import { cn, Spinner } from "@cozystack/ui"
3+
4+
interface EmptyStateProps {
5+
title: ReactNode
6+
hint?: ReactNode
7+
action?: ReactNode
8+
busy?: boolean
9+
tone?: "light" | "dark"
10+
className?: string
11+
}
12+
13+
export function EmptyState({ title, hint, action, busy, tone = "light", className }: EmptyStateProps) {
14+
const dark = tone === "dark"
15+
return (
16+
<div
17+
role="status"
18+
aria-busy={busy || undefined}
19+
className={cn("flex h-full flex-col items-center justify-center gap-1.5 px-4 py-8 text-center", className)}
20+
>
21+
{busy && <Spinner className={cn("mb-1 size-5", dark && "text-slate-500")} />}
22+
<p className={cn("text-sm", dark ? "text-slate-300" : "text-slate-600")}>{title}</p>
23+
{hint && <p className={cn("text-xs", dark ? "text-slate-500" : "text-slate-400")}>{hint}</p>}
24+
{action && <div className="mt-2">{action}</div>}
25+
</div>
26+
)
27+
}
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import { describe, it, expect } from "vitest"
2+
import { fireEvent, render, screen } from "@testing-library/react"
3+
import { LogLines } from "./LogLines.tsx"
4+
5+
const entry = (n: number) => ({ _time: `2026-11-04T10:00:0${n}Z`, _msg: `line ${n}`, _stream_id: "s" })
6+
7+
function sized(list: HTMLElement, scrollHeight: number) {
8+
Object.defineProperty(list, "scrollHeight", { configurable: true, value: scrollHeight })
9+
Object.defineProperty(list, "clientHeight", { configurable: true, value: 100 })
10+
}
11+
12+
describe("LogLines", () => {
13+
it("keeps the console frame and shows the placeholder inside it when there are no lines", () => {
14+
render(<LogLines entries={[]} placeholder="Waiting for new log lines…" />)
15+
16+
const frame = screen.getByRole("region", { name: "Log lines" })
17+
expect(frame).toHaveTextContent("Waiting for new log lines…")
18+
expect(screen.queryByRole("list")).toBeNull()
19+
})
20+
21+
it("puts the lines in the same frame once they arrive", () => {
22+
const { rerender } = render(<LogLines entries={[]} placeholder="Waiting for new log lines…" />)
23+
const frame = screen.getByRole("region", { name: "Log lines" })
24+
25+
rerender(<LogLines entries={[entry(1)]} placeholder="Waiting for new log lines…" />)
26+
27+
expect(screen.getByRole("region", { name: "Log lines" })).toBe(frame)
28+
expect(frame).toContainElement(screen.getByRole("list"))
29+
expect(frame).not.toHaveTextContent("Waiting")
30+
})
31+
32+
it("shows a line without _time with no timestamp and no empty source", () => {
33+
render(<LogLines placeholder="" entries={[{ _msg: "visible message" }]} />)
34+
35+
expect(screen.getByRole("listitem")).toHaveTextContent(/^visible message$/)
36+
})
37+
38+
it("shows the remaining fields of a row without _msg", () => {
39+
render(<LogLines placeholder="" entries={[{ kubernetes_pod_name: "db-0", rows: "42" }]} />)
40+
41+
expect(screen.getByRole("listitem")).toHaveTextContent(/^db-0\/ rows=42$/)
42+
})
43+
44+
it("keeps following the newest line while the reader is at the bottom", () => {
45+
const { rerender } = render(<LogLines placeholder="" entries={[entry(1)]} />)
46+
const list = screen.getByRole("list")
47+
sized(list, 500)
48+
49+
rerender(<LogLines placeholder="" entries={[entry(1), entry(2)]} />)
50+
51+
expect(list.scrollTop).toBe(500)
52+
})
53+
54+
it("leaves the position alone once the reader scrolls up", () => {
55+
const { rerender } = render(<LogLines placeholder="" entries={[entry(1)]} />)
56+
const list = screen.getByRole("list")
57+
sized(list, 500)
58+
list.scrollTop = 120
59+
fireEvent.scroll(list)
60+
61+
rerender(<LogLines placeholder="" entries={[entry(1), entry(2)]} />)
62+
63+
expect(list.scrollTop).toBe(120)
64+
})
65+
66+
it("follows again once the list is cleared for a new run", () => {
67+
const { rerender } = render(<LogLines placeholder="" entries={[entry(1)]} />)
68+
const first = screen.getByRole("list")
69+
sized(first, 500)
70+
first.scrollTop = 120
71+
fireEvent.scroll(first)
72+
73+
rerender(<LogLines placeholder="" entries={[]} />)
74+
rerender(<LogLines placeholder="" entries={[entry(2)]} />)
75+
const second = screen.getByRole("list")
76+
sized(second, 300)
77+
rerender(<LogLines placeholder="" entries={[entry(2), entry(3)]} />)
78+
79+
expect(second.scrollTop).toBe(300)
80+
})
81+
})
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
import { useEffect, useRef, type ReactNode } from "react"
2+
import { CONTAINER_FIELD, POD_FIELD } from "../../lib/observability/logs.ts"
3+
import type { LogEntry } from "../../lib/observability/proxy.ts"
4+
5+
const BOTTOM_SLACK_PX = 24
6+
const SHOWN_APART = new Set(["_time", "_msg", "_stream_id", POD_FIELD, CONTAINER_FIELD])
7+
8+
function source(e: LogEntry): string | null {
9+
const pod = e[POD_FIELD]
10+
const container = e[CONTAINER_FIELD]
11+
return pod || container ? `${pod ?? ""}/${container ?? ""}` : null
12+
}
13+
14+
// A `fields` or `stats` pipe can leave a row without _msg; its other fields are then the content.
15+
function message(e: LogEntry): string {
16+
if (e._msg !== undefined) return e._msg
17+
return Object.entries(e)
18+
.filter(([k, v]) => v !== undefined && !SHOWN_APART.has(k))
19+
.map(([k, v]) => `${k}=${v}`)
20+
.join(" ")
21+
}
22+
23+
export function LogLines({ entries, placeholder }: { entries: LogEntry[]; placeholder: ReactNode }) {
24+
const ref = useRef<HTMLOListElement>(null)
25+
const pinned = useRef(true)
26+
27+
useEffect(() => {
28+
if (entries.length === 0) pinned.current = true
29+
else if (ref.current && pinned.current) ref.current.scrollTop = ref.current.scrollHeight
30+
}, [entries])
31+
32+
return (
33+
<section
34+
aria-label="Log lines"
35+
className="h-[60vh] min-h-72 overflow-hidden rounded-lg bg-slate-950 font-mono text-xs leading-5 text-slate-100 shadow-xs"
36+
>
37+
{entries.length === 0 ? (
38+
placeholder
39+
) : (
40+
<ol
41+
ref={ref}
42+
onScroll={(e) => {
43+
const el = e.currentTarget
44+
pinned.current = el.scrollHeight - el.scrollTop - el.clientHeight <= BOTTOM_SLACK_PX
45+
}}
46+
className="h-full overflow-auto p-3"
47+
>
48+
{entries.map((e, i) => {
49+
const from = source(e)
50+
return (
51+
<li key={`${e._stream_id}-${e._time}-${i}`} className="whitespace-pre-wrap break-all">
52+
{e._time !== undefined && <><span className="text-slate-400">{new Date(e._time).toLocaleTimeString()}</span>{" "}</>}
53+
{from !== null && <><span className="text-sky-300">{from}</span>{" "}</>}
54+
{message(e)}
55+
</li>
56+
)
57+
})}
58+
</ol>
59+
)}
60+
</section>
61+
)
62+
}

0 commit comments

Comments
 (0)