Skip to content

Trace#370

Merged
SuperCoolPencil merged 9 commits intomainfrom
trace
Apr 16, 2026
Merged

Trace#370
SuperCoolPencil merged 9 commits intomainfrom
trace

Conversation

@SuperCoolPencil
Copy link
Copy Markdown
Member

@SuperCoolPencil SuperCoolPencil commented Apr 14, 2026

Greptile Summary

This PR adds connection pre-warming via hedged dialing (prewarmConnections), exposes a new DialHedgeCount setting end-to-end through config, engine types, and the TUI settings view, and refactors TUI initialization from init() functions to explicit InitializeTUI() / InitializeStyles() / InitializeStatusCache() calls using sync.Once. A comprehensive test suite accompanies all changes.

  • TestConcurrentDownloader_PrewarmConnections has a P1 defect: MockServer.handleRequest returns immediately when CustomHandler is set, so the custom handler in this test replaces file-serving entirely. Download workers receive HTTP 200 with no body, stall, and the 5-second context deadline expires — the test will fail.

Confidence Score: 4/5

Mostly safe to merge; one P1 test bug should be fixed first to avoid a failing CI gate.

Production code (pre-warming, DialHedgeCount wiring, TUI init refactor) is correct. The P1 finding is confined to a test: TestConcurrentDownloader_PrewarmConnections uses WithHandler in a way that breaks the mock server's file-serving. No runtime or data-integrity risk, but it would block CI and leaves the pre-warming feature without a passing integration test.

internal/engine/concurrent/prewarm_test.go — WithHandler replaces file serving, making the download under test fail.

Important Files Changed

Filename Overview
internal/engine/concurrent/prewarm_test.go New prewarm integration test, but WithHandler completely replaces file-serving logic, causing download workers to receive empty responses and the test to fail.
internal/engine/concurrent/downloader.go Adds prewarmConnections() with hedged dialing; logic is sound after prior WaitGroup fix; ready-before-Close ordering already flagged in previous threads.
internal/engine/types/config_convert_test.go New exhaustive conversion tests are valuable but the comment for TestConvertRuntimeConfig_Exhaustive incorrectly describes the coverage direction (checks output fields, not input consumption).
internal/engine/types/config_convert.go Adds DialHedgeCount mapping; all types.RuntimeConfig fields are correctly populated from config.RuntimeConfig.
internal/config/settings.go Adds DialHedgeCount to NetworkSettings and ToRuntimeConfig mapping; default value of 4 is reasonable.
internal/tui/view_settings.go Adds dial_hedge_count UI support (unit label, reset case); integrates cleanly with existing settings framework.

Sequence Diagram

sequenceDiagram
    participant CLI as cmd/root.go
    participant Settings as config.Settings
    participant RC as config.RuntimeConfig
    participant Conv as types.ConvertRuntimeConfig
    participant TRC as types.RuntimeConfig
    participant DL as ConcurrentDownloader
    participant Prewarm as prewarmConnections

    CLI->>Settings: getSettings()
    Settings->>RC: ToRuntimeConfig() [incl. DialHedgeCount]
    RC->>Conv: ConvertRuntimeConfig(rc)
    Conv->>TRC: types.RuntimeConfig{DialHedgeCount: ...}
    CLI->>DL: NewConcurrentDownloader(..., runtime)
    DL->>DL: Download(ctx, url, mirrors, destPath, fileSize)
    alt hedgeCount > 0
        DL->>Prewarm: prewarmConnections(ctx, client, numRequired, hedgeCount, mirrors)
        Prewarm-->>DL: N connections warmed in HTTP pool
    end
    DL->>DL: workers download chunks using pooled connections
Loading

Comments Outside Diff (1)

  1. internal/engine/types/config_convert.go, line 6-21 (link)

    P1 Pre-warming dead in production — DialHedgeCount never propagated

    ConvertRuntimeConfig (and Settings.ToRuntimeConfig in settings.go, plus the nil-guard default in NewConcurrentDownloader) all omit DialHedgeCount. Because GetDialHedgeCount explicitly treats zero as "disabled" (not as "use the constant default"), every production-built RuntimeConfig yields hedgeCount == 0, and the if hedgeCount > 0 guard in Download means prewarmConnections is never called. The constant DialHedgeCount = 4 is effectively unreachable via any real code path.

    Add the field to all three construction sites:

    // config_convert.go
    return &RuntimeConfig{
        // ...existing fields...
        DialHedgeCount: DialHedgeCount, // forward the constant default
    }
    // NewConcurrentDownloader nil-guard
    runtime = &types.RuntimeConfig{
        MaxConnectionsPerHost: types.PerHostMax,
        MinChunkSize:          types.MinChunk,
        WorkerBufferSize:      types.WorkerBuffer,
        DialHedgeCount:        types.DialHedgeCount,
    }

    Without this, the entire pre-warming subsystem is dead code for every production download.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: internal/engine/types/config_convert.go
    Line: 6-21
    
    Comment:
    **Pre-warming dead in production — `DialHedgeCount` never propagated**
    
    `ConvertRuntimeConfig` (and `Settings.ToRuntimeConfig` in `settings.go`, plus the nil-guard default in `NewConcurrentDownloader`) all omit `DialHedgeCount`. Because `GetDialHedgeCount` explicitly treats zero as "disabled" (not as "use the constant default"), every production-built `RuntimeConfig` yields `hedgeCount == 0`, and the `if hedgeCount > 0` guard in `Download` means `prewarmConnections` is never called. The constant `DialHedgeCount = 4` is effectively unreachable via any real code path.
    
    Add the field to all three construction sites:
    
    ```go
    // config_convert.go
    return &RuntimeConfig{
        // ...existing fields...
        DialHedgeCount: DialHedgeCount, // forward the constant default
    }
    ```
    
    ```go
    // NewConcurrentDownloader nil-guard
    runtime = &types.RuntimeConfig{
        MaxConnectionsPerHost: types.PerHostMax,
        MinChunkSize:          types.MinChunk,
        WorkerBufferSize:      types.WorkerBuffer,
        DialHedgeCount:        types.DialHedgeCount,
    }
    ```
    
    Without this, the entire pre-warming subsystem is dead code for every production download.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: internal/engine/concurrent/prewarm_test.go
Line: 29-43

Comment:
**`WithHandler` replaces file serving, breaking the download under test**

`MockServer.handleRequest` returns immediately when `CustomHandler` is set (`if m.CustomHandler != nil { m.CustomHandler(w, r); return }`), so the custom handler here *completely replaces* normal file serving. The callback only sets boolean flags and writes no response body, so every request the actual download workers make receives an HTTP 200 with an empty body. Workers will read 0 bytes for each chunk, stall (the `Downloaded` counter never reaches `fileSize`), and the 5-second context deadline expires — making `downloader.Download` return an error and the test fail with `t.Fatalf`.

The pattern works in `switch_429_test.go` because that custom handler explicitly writes a 429 status, which is the intended behaviour. Here, the intent is to *observe* requests while still serving the file normally.

A fix is to add an `ObserveHandler` option to `MockServer` that runs the callback before (not instead of) the default file-serving path, or write the full range response inside the custom handler.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: internal/engine/types/config_convert_test.go
Line: 88-123

Comment:
**Misleading test comment overstates the coverage direction**

The doc-comment says the test verifies "EVERY field in `config.RuntimeConfig` is mapped to something in `types.RuntimeConfig`," but the loop iterates over the *output* struct (`types.RuntimeConfig`) and checks no output field is zero — it says nothing about whether every *input* field is consumed. If a new field is added to `config.RuntimeConfig` without a corresponding field in `types.RuntimeConfig`, this test will not detect the omission. Consider updating the comment to accurately describe what is verified.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (3): Last reviewed commit: "test: add exhaustive reflection-based te..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 14, 2026

Binary Size Analysis

⚠️ Size Increased

Version Human Readable Raw Bytes
Main 18.78 MB 19689764
PR 18.78 MB 19693860
Difference 4.00 KB 4096

Comment thread internal/engine/concurrent/downloader.go Outdated
Comment thread internal/engine/concurrent/downloader.go
Comment thread cmd/root.go Outdated
Comment on lines +423 to +436
if traceFile != "" {
f, err := os.Create(traceFile)
if err != nil {
fmt.Fprintf(os.Stderr, "failed to create trace file: %v\n", err)
os.Exit(1)
}
if err := trace.Start(f); err != nil {
fmt.Fprintf(os.Stderr, "failed to start trace: %v\n", err)
os.Exit(1)
}
// Use a finalizer or a global cleanup to stop the trace
// Since Cobra doesn't have a reliable PersistentPostRun for all exit paths (like signals),
// we'll handle it in specific exit points.
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Trace file handle not closed after trace.Stop()

f is opened in PersistentPreRun but is never stored for later cleanup. trace.Stop() in shutdown.go flushes all trace events to the writer but does not close the underlying file. The handle will eventually be closed by the GC finalizer or OS on process exit, but explicit close is the correct pattern and avoids a dangling descriptor in edge cases (e.g., multiple re-invocations in tests, or OOM scenarios where the GC does not run).

Store the file in a package-level variable and close it after trace.Stop():

var traceFileHandle *os.File

// In PersistentPreRun:
traceFileHandle = f

// In defaultGlobalShutdown (shutdown.go), after trace.Stop():
trace.Stop()
if traceFileHandle != nil {
    traceFileHandle.Close()
    traceFileHandle = nil
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: cmd/root.go
Line: 423-436

Comment:
**Trace file handle not closed after `trace.Stop()`**

`f` is opened in `PersistentPreRun` but is never stored for later cleanup. `trace.Stop()` in `shutdown.go` flushes all trace events to the writer but does not close the underlying file. The handle will eventually be closed by the GC finalizer or OS on process exit, but explicit close is the correct pattern and avoids a dangling descriptor in edge cases (e.g., multiple re-invocations in tests, or OOM scenarios where the GC does not run).

Store the file in a package-level variable and close it after `trace.Stop()`:

```go
var traceFileHandle *os.File

// In PersistentPreRun:
traceFileHandle = f

// In defaultGlobalShutdown (shutdown.go), after trace.Stop():
trace.Stop()
if traceFileHandle != nil {
    traceFileHandle.Close()
    traceFileHandle = nil
}
```

How can I resolve this? If you propose a fix, please make it concise.

junaid2005p
junaid2005p previously approved these changes Apr 16, 2026
Comment on lines +29 to +43
testutil.WithFileSize(fileSize),
testutil.WithRangeSupport(true),
testutil.WithHandler(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()

rng := r.Header.Get("Range")
if rng == "bytes=0-0" {
prewarmSeen = true
} else if rng != "" {
// Actual download request usually has a real range
downloadSeen = true
}
}),
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 WithHandler replaces file serving, breaking the download under test

MockServer.handleRequest returns immediately when CustomHandler is set (if m.CustomHandler != nil { m.CustomHandler(w, r); return }), so the custom handler here completely replaces normal file serving. The callback only sets boolean flags and writes no response body, so every request the actual download workers make receives an HTTP 200 with an empty body. Workers will read 0 bytes for each chunk, stall (the Downloaded counter never reaches fileSize), and the 5-second context deadline expires — making downloader.Download return an error and the test fail with t.Fatalf.

The pattern works in switch_429_test.go because that custom handler explicitly writes a 429 status, which is the intended behaviour. Here, the intent is to observe requests while still serving the file normally.

A fix is to add an ObserveHandler option to MockServer that runs the callback before (not instead of) the default file-serving path, or write the full range response inside the custom handler.

Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/engine/concurrent/prewarm_test.go
Line: 29-43

Comment:
**`WithHandler` replaces file serving, breaking the download under test**

`MockServer.handleRequest` returns immediately when `CustomHandler` is set (`if m.CustomHandler != nil { m.CustomHandler(w, r); return }`), so the custom handler here *completely replaces* normal file serving. The callback only sets boolean flags and writes no response body, so every request the actual download workers make receives an HTTP 200 with an empty body. Workers will read 0 bytes for each chunk, stall (the `Downloaded` counter never reaches `fileSize`), and the 5-second context deadline expires — making `downloader.Download` return an error and the test fail with `t.Fatalf`.

The pattern works in `switch_429_test.go` because that custom handler explicitly writes a 429 status, which is the intended behaviour. Here, the intent is to *observe* requests while still serving the file normally.

A fix is to add an `ObserveHandler` option to `MockServer` that runs the callback before (not instead of) the default file-serving path, or write the full range response inside the custom handler.

How can I resolve this? If you propose a fix, please make it concise.

@SuperCoolPencil SuperCoolPencil merged commit a4ae483 into main Apr 16, 2026
16 checks passed
@SuperCoolPencil SuperCoolPencil deleted the trace branch April 16, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants