Skip to content

[Repo Assist] Add URL support to Frame.ReadCsv — closes #38#658

Merged
dsyme merged 2 commits intomasterfrom
repo-assist/fix-issue-38-readcsv-url-support-47ee888362e829ae
Mar 19, 2026
Merged

[Repo Assist] Add URL support to Frame.ReadCsv — closes #38#658
dsyme merged 2 commits intomasterfrom
repo-assist/fix-issue-38-readcsv-url-support-47ee888362e829ae

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

🤖 Repo Assist — implementing the URL-support feature requested in #38.

Problem

The documentation for Frame.ReadCsv described the location parameter as accepting "a file name or web location (URL)", but the implementation only handled local file paths — URLs silently caused an I/O exception. This discrepancy has been open since the issue was filed in 2014.

Solution

Detect whether the location string is an HTTP/HTTPS URL and, if so, fetch its content via HttpClient, then pass a StringReader to the CSV parser — leaving the local file path unchanged for all other inputs.

// Now works:
let df = Frame.ReadCsv("(example.com/redacted)
let df = Frame.ReadCsv("https://raw.githubusercontent.com/user/repo/main/data.csv")

// Still works:
let df = Frame.ReadCsv("local/path/data.csv")
// C# also supported:
var df = Frame.ReadCsv("(example.com/redacted)

Implementation

Added a private HttpReader module in FrameExtensions.fs:

module private HttpReader =
    let private httpClient = lazy (new System.Net.Http.HttpClient())
    let openLocation (location: string) : System.IO.TextReader =
        if location.StartsWith("(redacted) StringComparison.OrdinalIgnoreCase) ||
           location.StartsWith("https://", StringComparison.OrdinalIgnoreCase) then
            let content = httpClient.Value.GetStringAsync(location).GetAwaiter().GetResult()
            new System.IO.StringReader(content) :> System.IO.TextReader
        else
            new System.IO.StreamReader(location) :> System.IO.TextReader
```

- `HttpClient` is shared via a `lazy` static instance to avoid socket exhaustion.
- `GetAwaiter().GetResult()` keeps `ReadCsv` synchronous — no change to callers.
- `StringReader` only needs the already-fetched string, so the `HttpClient` lifetime is irrelevant after the fetch.
- All three `ReadCsv` path/location overloads are updated: the C# extension and both F# extensions.

## Test Status

```
Passed! - Failed: 0, Passed: 669, Skipped: 0, Total: 669

All 669 tests pass on Ubuntu. No new network-dependent tests added (these would be fragile in CI without a local server).

Generated by Repo Assist

Generated by Repo Assist ·

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@30f2254f2a7a944da1224df45d181a3f8faefd0d

Frame.ReadCsv(path, ...) now accepts HTTP/HTTPS URLs in addition to
local file paths. The URL is detected by the 'http://' or 'https://'
prefix and downloaded synchronously via a shared HttpClient instance.

Both the C# (Frame.ReadCsv) and F# (Frame.ReadCsv / Frame.ReadCsv<'R>)
path-based overloads benefit from the change. Stream- and TextReader-
based overloads are unaffected.

The documentation for the `path` parameter already said 'a file name or
a web location of the resource' — this commit makes that promise real.

Co-authored-by: Copilot <[email protected]>
@dsyme dsyme marked this pull request as ready for review March 19, 2026 01:12
@dsyme dsyme merged commit a76a627 into master Mar 19, 2026
2 checks passed
@dsyme dsyme deleted the repo-assist/fix-issue-38-readcsv-url-support-47ee888362e829ae branch March 19, 2026 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant