Skip to content

Commit 90caf84

Browse files
aacotroneoclaude
andauthored
cli/cloud: fix pulumi insights account scan log pagination (#23256)
## Summary `scan log --all` doesn't paginate to the end; and step mode comes back empty." The Insights scan-logs endpoint and the Pulumi Deployments logs endpoint are served by the same server handler. The insights apitype had drifted away from that shape, which is the whole reason the bugs existed. **What this PR copies from [`apitype.DeploymentLogs`](sdk/go/common/apitype/deployments.go#L351-L354) / [`deployment_log.go`](pkg/cmd/pulumi/deployment/deployment_log.go):** - The response field is `NextToken` with `json:"nextToken,omitempty"` — matches what the server emits. - Step mode returns a `Lines` array (not a raw `Output` string). - The `--all` loop terminates only when the server emits an empty cursor. ### Bug 1 — continuation mode (`--all`) The server emits `nextToken` on the wire but [`apitype.InsightsScanLogs`](sdk/go/common/apitype/insights.go) had the field tagged `continuationToken`. The struct field decoded to `""` on every response, so the page loop terminated after the first page. Renamed `ContinuationToken` → `NextToken` with the matching JSON tag — same shape as [`apitype.DeploymentLogs`](sdk/go/common/apitype/deployments.go), which is the same server handler. ### Bug 2 — step mode (`--job N --step M`) The server returns `{lines: [{timestamp, line}, …]}` in step mode too — same envelope as continuation mode, just scoped to one step, with `nextOffset` (an int line index) instead of `nextToken`. The previous step renderer wrote `r.Output` (a never-populated string field), so the command printed nothing. Dropped the dead `Output` field; both modes now share one line-oriented renderer. `--all` in step mode follows the `nextOffset` chain to accumulate every page of lines. **What this PR deliberately does NOT copy from Deployments:** - The `--count` / `--all` semantics on `insights account scan log` are kept as-is (matching the rest of the Cloud-Ready Insights CLI surface — `account list`, `account scan list`). - The `--org` + positional `<account>` scoping stays — Insights scopes by account, deployments scope by stack. ## Follow-up worth considering (not in this PR) `apitype.DeploymentLogs` and `apitype.InsightsScanLogs` are now literally the same shape (`Lines []…`, `NextToken string`, plus step-mode `NextOffset int64`). They could be unified into a single `apitype.WorkflowLogs` struct shared between the two endpoints. Similarly, the `--all` loops in [`deployment_log.go`](pkg/cmd/pulumi/deployment/deployment_log.go) and [`insights_account_scan_log.go`](pkg/cmd/pulumi/insights/insights_account_scan_log.go) are nearly identical and could share a helper. Both would be refactors that should touch deployments and insights together, not be smuggled into a bug fix. ## Sample output (before / after) ### `--all` continuation mode ``` # Before: stops after one page (~100 lines) $ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --all | wc -l 100 # After: follows the cursor to the end $ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --all | wc -l 8099 ``` ### `--job N --step M --all` step mode ``` # Before: empty output $ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 0 --all (empty) # After: prints the step's log lines and follows nextOffset $ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 0 --all 2026-01-21T15:59:15Z Preparing environment 2026-01-21T15:59:17Z Using runner default image with Pulumi version 3.216.0 … 2026-01-21T15:59:17Z Starting container 2026-01-21T15:59:26Z Container created … 2026-01-21T15:59:26Z Container started … … ``` For a step whose log spans multiple `nextOffset` pages (step 3 in the example above), `--all` now fetches the full content: ``` $ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 3 --all | wc -l 8081 ``` ## Test plan - [x] Updated unit tests for both modes to use the real server response shape (`lines` array in step mode, `nextToken` in continuation mode). - [x] Updated [`client_insights_test.go`](pkg/backend/httpstate/client/client_insights_test.go) to round-trip the same shape. - [x] `mise exec -- make format` - [x] `mise exec -- make lint` (0 issues across 7 packages) - [x] Live smoke against the `pulumi` org: continuation `--all` returns 8099 lines (was ~100), step-mode `--job 0 --step 0 --all` returns the step's log content (was empty), step 3 `--all` fetches 8081 lines across multiple `nextOffset` pages. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 89c75c0 commit 90caf84

5 files changed

Lines changed: 79 additions & 63 deletions

File tree

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
changes:
2+
- type: fix
3+
scope: cli/cloud
4+
description: Fix `pulumi insights account scan log --all` to follow the server's pagination cursor through the end of the log, and render `--job/--step` mode as structured lines instead of an empty raw-string blob

‎pkg/backend/httpstate/client/client_insights_test.go‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -568,7 +568,7 @@ func TestGetInsightsScanLogs(t *testing.T) {
568568
Line: "finished scan",
569569
},
570570
},
571-
ContinuationToken: "next-page",
571+
NextToken: "next-page",
572572
}
573573

574574
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
@@ -588,8 +588,13 @@ func TestGetInsightsScanLogs(t *testing.T) {
588588
t.Parallel()
589589

590590
want := apitype.InsightsScanLogs{
591-
Type: "step",
592-
Output: "scan output text\n",
591+
Type: "DeploymentLogsStep",
592+
Lines: []apitype.InsightsScanLogLine{
593+
{
594+
Timestamp: time.Date(2026, 5, 1, 14, 30, 0, 0, time.UTC),
595+
Line: "scan output text\n",
596+
},
597+
},
593598
NextOffset: 1024,
594599
}
595600

‎pkg/cmd/pulumi/insights/insights_account_scan_log.go‎

Lines changed: 21 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -203,16 +203,16 @@ func (c *insightsAccountScanLogCmd) fetchContinuationMode(
203203
acc.Type = resp.Type
204204
}
205205
acc.Lines = append(acc.Lines, resp.Lines...)
206-
acc.ContinuationToken = resp.ContinuationToken
206+
acc.NextToken = resp.NextToken
207207

208208
if want > 0 && len(acc.Lines) >= want {
209209
acc.Lines = acc.Lines[:want]
210210
break
211211
}
212-
if resp.ContinuationToken == "" || len(resp.Lines) == 0 || want == 0 {
212+
if resp.NextToken == "" || len(resp.Lines) == 0 || want == 0 {
213213
break
214214
}
215-
token = resp.ContinuationToken
215+
token = resp.NextToken
216216
}
217217
return acc, nil
218218
}
@@ -222,18 +222,22 @@ func (c *insightsAccountScanLogCmd) fetchStepMode(
222222
org, account, scanID string,
223223
) (apitype.InsightsScanLogs, error) {
224224
job, step := c.job, c.step
225-
params := apitype.InsightsScanLogsParams{Job: &job, Step: &step}
226-
resp, err := client.GetInsightsScanLogs(ctx, org, account, scanID, params)
225+
resp, err := client.GetInsightsScanLogs(ctx, org, account, scanID,
226+
apitype.InsightsScanLogsParams{Job: &job, Step: &step})
227227
if err != nil {
228228
return apitype.InsightsScanLogs{}, err
229229
}
230230

231-
// --count doesn't apply in step mode (the API paginates by bytes, not
232-
// by entries), so default and --count behave identically.
231+
// --count doesn't apply in step mode (the API paginates by line
232+
// offset, not by entry count), so default and --count behave
233+
// identically here.
233234
if !c.all {
234235
return resp, nil
235236
}
236237

238+
// In step mode the server emits `lines` plus a `nextOffset` pointing
239+
// at the next line index to start from. Follow the offset chain until
240+
// the server stops returning one, accumulating Lines.
237241
acc := resp
238242
for acc.NextOffset > 0 {
239243
offset := acc.NextOffset
@@ -242,40 +246,17 @@ func (c *insightsAccountScanLogCmd) fetchStepMode(
242246
if err != nil {
243247
return apitype.InsightsScanLogs{}, err
244248
}
245-
acc.Output += more.Output
249+
acc.Lines = append(acc.Lines, more.Lines...)
246250
acc.NextOffset = more.NextOffset
247-
if more.Output == "" && more.NextOffset == 0 {
248-
break
249-
}
250251
}
251252
return acc, nil
252253
}
253254

254-
// renderText dispatches by mode: step mode emits a raw text blob, continuation
255-
// mode emits structured records.
255+
// renderText prints the log entries one per line. The shape is the same for
256+
// both modes — the server returns `lines` in either case — only the
257+
// "more available" footer differs (cursor token in continuation mode,
258+
// line-offset in step mode).
256259
func (c *insightsAccountScanLogCmd) renderText(r apitype.InsightsScanLogs) error {
257-
if c.jobSet {
258-
return c.renderStepText(r)
259-
}
260-
return c.renderContinuationText(r)
261-
}
262-
263-
func (c *insightsAccountScanLogCmd) renderStepText(r apitype.InsightsScanLogs) error {
264-
if _, err := io.WriteString(c.w, r.Output); err != nil {
265-
return err
266-
}
267-
// Force a trailing newline so the hint below sits on its own line.
268-
if len(r.Output) > 0 && r.Output[len(r.Output)-1] != '\n' {
269-
fmt.Fprintln(c.w)
270-
}
271-
if r.NextOffset > 0 {
272-
fmt.Fprintf(c.w,
273-
"More output available. Re-run with --all to fetch the rest.\n")
274-
}
275-
return nil
276-
}
277-
278-
func (c *insightsAccountScanLogCmd) renderContinuationText(r apitype.InsightsScanLogs) error {
279260
if len(r.Lines) == 0 {
280261
fmt.Fprintln(c.w, "No log entries.")
281262
return nil
@@ -300,7 +281,11 @@ func (c *insightsAccountScanLogCmd) renderContinuationText(r apitype.InsightsSca
300281
fmt.Fprintln(c.w, text)
301282
}
302283
}
303-
if r.ContinuationToken != "" {
284+
switch {
285+
case c.jobSet && r.NextOffset > 0:
286+
fmt.Fprintln(c.w,
287+
"\nMore output available. Re-run with --all to fetch the rest.")
288+
case !c.jobSet && r.NextToken != "":
304289
fmt.Fprintln(c.w,
305290
"\nMore entries available. Re-run with --count <N> or --all to fetch more.")
306291
}

‎pkg/cmd/pulumi/insights/insights_account_scan_log_test.go‎

Lines changed: 33 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -100,9 +100,26 @@ func continuationPage(lines int, token string) apitype.InsightsScanLogs {
100100
})
101101
}
102102
return apitype.InsightsScanLogs{
103-
Type: "continuation",
104-
Lines: entries,
105-
ContinuationToken: token,
103+
Type: "continuation",
104+
Lines: entries,
105+
NextToken: token,
106+
}
107+
}
108+
109+
// stepPage builds a step-mode response: structured lines plus an optional
110+
// nextOffset cursor mirroring what the live API returns.
111+
func stepPage(lines int, prefix string, nextOffset int64) apitype.InsightsScanLogs {
112+
entries := make([]apitype.InsightsScanLogLine, 0, lines)
113+
for i := 0; i < lines; i++ {
114+
entries = append(entries, apitype.InsightsScanLogLine{
115+
Timestamp: time.Date(2026, 5, 1, 14, 30, i, 0, time.UTC),
116+
Line: prefix + "-" + string(rune('0'+i)) + "\n",
117+
})
118+
}
119+
return apitype.InsightsScanLogs{
120+
Type: "DeploymentLogsStep",
121+
Lines: entries,
122+
NextOffset: nextOffset,
106123
}
107124
}
108125

@@ -266,11 +283,7 @@ func TestScanLog_StepDefault(t *testing.T) {
266283
t.Parallel()
267284

268285
client := &mockScanLogClient{
269-
responses: []apitype.InsightsScanLogs{{
270-
Type: "step",
271-
Output: "step output chunk 1\n",
272-
NextOffset: 1024,
273-
}},
286+
responses: []apitype.InsightsScanLogs{stepPage(2, "setup", 1024)},
274287
}
275288
cmd, buf := newTestScanLogCmd()
276289
cmd.jobSet = true
@@ -289,7 +302,8 @@ func TestScanLog_StepDefault(t *testing.T) {
289302
assert.Nil(t, client.calls[0].params.Offset, "default mode never sets offset")
290303

291304
out := buf.String()
292-
assert.Contains(t, out, "step output chunk 1")
305+
assert.Contains(t, out, "setup-0")
306+
assert.Contains(t, out, "setup-1")
293307
assert.Contains(t, out, "More output available. Re-run with --all")
294308
}
295309

@@ -298,9 +312,9 @@ func TestScanLog_StepAll(t *testing.T) {
298312

299313
client := &mockScanLogClient{
300314
responses: []apitype.InsightsScanLogs{
301-
{Type: "step", Output: "chunk-1\n", NextOffset: 1024},
302-
{Type: "step", Output: "chunk-2\n", NextOffset: 2048},
303-
{Type: "step", Output: "chunk-3\n", NextOffset: 0},
315+
stepPage(2, "page0", 1024),
316+
stepPage(2, "page1", 2048),
317+
stepPage(1, "page2", 0),
304318
},
305319
}
306320
cmd, buf := newTestScanLogCmd()
@@ -326,9 +340,12 @@ func TestScanLog_StepAll(t *testing.T) {
326340
}
327341

328342
out := buf.String()
329-
assert.Contains(t, out, "chunk-1")
330-
assert.Contains(t, out, "chunk-2")
331-
assert.Contains(t, out, "chunk-3")
343+
// Every line from every page is accumulated and rendered.
344+
assert.Contains(t, out, "page0-0")
345+
assert.Contains(t, out, "page0-1")
346+
assert.Contains(t, out, "page1-0")
347+
assert.Contains(t, out, "page1-1")
348+
assert.Contains(t, out, "page2-0")
332349
assert.NotContains(t, out, "More output available")
333350
}
334351

@@ -469,9 +486,7 @@ func TestNewInsightsAccountScanLogCmd_FlagBinding_StepMode(t *testing.T) {
469486
t.Parallel()
470487

471488
client := &mockScanLogClient{
472-
responses: []apitype.InsightsScanLogs{{
473-
Type: "step", Output: "data\n",
474-
}},
489+
responses: []apitype.InsightsScanLogs{stepPage(1, "data", 0)},
475490
}
476491
cmd := newInsightsAccountScanLogCmd(stubScanLogFactory(client, "acme"))
477492

‎sdk/go/common/apitype/insights.go‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -380,13 +380,20 @@ type InsightsScanLogsParams struct {
380380

381381
// InsightsScanLogs is the response from GetScanLogs. Type is the
382382
// DeploymentLogsBase discriminator; only the fields relevant to the active
383-
// mode (continuation-token vs step) are populated.
383+
// mode (continuation vs step) are populated:
384+
//
385+
// - continuation mode (no Job/Step in the request): the server emits a
386+
// `lines` array spanning every step, paginated with `nextToken`.
387+
// - step mode (Job and Step set): the server emits a `lines` array scoped
388+
// to one step, paginated with `nextOffset` (a line index).
389+
//
390+
// The wire field name (`nextToken`) matches `apitype.DeploymentLogs`, since
391+
// both endpoints share a server-side handler.
384392
type InsightsScanLogs struct {
385-
Type string `json:"__type,omitempty"`
386-
Lines []InsightsScanLogLine `json:"lines,omitempty"`
387-
ContinuationToken string `json:"continuationToken,omitempty"`
388-
Output string `json:"output,omitempty"`
389-
NextOffset int64 `json:"nextOffset,omitempty"`
393+
Type string `json:"__type,omitempty"`
394+
Lines []InsightsScanLogLine `json:"lines,omitempty"`
395+
NextToken string `json:"nextToken,omitempty"`
396+
NextOffset int64 `json:"nextOffset,omitempty"`
390397
}
391398

392399
// InsightsScanLogLine mirrors apitype.DeploymentLogLine.

0 commit comments

Comments
 (0)