Repository navigation
fix(fetcher): allow connecting to a proxy set through the environment - #4513
Conversation
The private-network check refuses to dial a proxy configured through HTTP_PROXY or HTTPS_PROXY, even though the transport still routes requests to it. A proxy on a loopback or LAN address is therefore enabled and then blocked, and the fetch fails with: proxyconnect tcp: dial tcp 127.0.0.1:8888: fetcher: refusing to access private network host "127.0.0.1" The trusted-hop exemption covers the feed proxy, the application proxy and the proxy rotator, but the environment variables were not considered, so proxyDialAddress stayed empty and the blocking dialer was used for the connection to the proxy itself. The proxy for the request is now resolved with x/net/http/httpproxy rather than http.ProxyFromEnvironment, because the latter caches the environment on first use and needs a request that does not exist yet at that point. Reading it directly also keeps the proxy used for routing and the proxy exempted from the check in agreement, and leaves NO_PROXY evaluated per request URL, so a host excluded from the proxy is still checked. x/net is already a direct dependency, so nothing new is required.
fguillot
left a comment
fguillot
left a comment
There was a problem hiding this comment.
proxyDialAddress is derived only from the initial URL, but redirects may select a different proxy (e.g. HTTP_PROXY -> HTTPS_PROXY). That proxy is then incorrectly blocked as private.
The trusted-hop address was resolved once from the initial request URL, but the proxy taken from the environment is resolved per request URL. A redirect that changes the scheme selects the other variable, so a chain that starts on HTTP_PROXY and is redirected to an https URL dials the HTTPS_PROXY host with the blocking dialer and is refused: Get "https://secure.invalid/rss.xml": proxyconnect tcp: dial tcp 127.0.0.1: fetcher: refusing to access private network host "127.0.0.1" The addresses are now collected in the transport's Proxy function, which is the single place where the proxy for a hop is decided. Every proxy the transport actually selects is a trusted hop, and nothing else is, so NO_PROXY and any redirect target stay covered without resolving the proxy twice.
|
fixed in b08db3a The address was resolved once from the initial URL, so a chain that starts on The trusted addresses are now collected inside the transport's
|
For tests we are using httpproxy.FromEnvironment().ProxyFunc() from golang.org/x/net/http/httpproxy directly, because http.ProxyFromEnvironment() caches it and changes of env vars don't change cached value. It breaks tests. Inspired by miniflux#4513
|
There was recently CVE-2026-108735 assigned associated with https://hackmd.io/@haind03/miniflux-feed-proxy-url-private-network-bypass . This change should fix it? |
Description: Fixes #4456. With
FETCHER_ALLOW_PRIVATE_NETWORKS=0, a proxy configured throughHTTP_PROXY/HTTPS_PROXYis still used for routing, but the connection to the proxy itself is refused when the proxy sits on a private address:Motivation: #4245 made explicitly configured proxies a trusted hop, but it enumerated the three miniflux-side sources (feed proxy, application proxy, proxy rotator). The environment variables were not included, so
proxyDialAddressstays empty and the blocking dialer is used for the hop to the proxy, whiletransport.Proxystill routes through it. The proxy is enabled and blocked at the same time.This matters because the environment variables are the documented way to proxy everything, and the example in the docs is
http://127.0.0.1:8888, which is exactly the case that stopped working.The trusted hop is recorded inside the transport's
Proxyfunction, which is the single place where the proxy for a hop is decided. The environment proxy is resolved per request URL, so a redirect can select a different proxy than the initial request did (HTTP_PROXY->HTTPS_PROXY), and collecting the addresses as the transport picks them keeps every hop it actually makes covered and nothing else. Resolving it once from the initial URL missed the second proxy, which is what @fguillot pointed out in review.x/net/http/httpproxyis used instead ofhttp.ProxyFromEnvironmentbecause the latter caches the environment on first use.NO_PROXYstays honoured per request URL, so a host excluded from the proxy is still dialled directly and still checked. That is the second test below.x/netis already a direct dependency.go.modandgo.sumare unchanged.Testing: Three tests added to
internal/reader/fetcher/request_builder_test.go:TestRequestBuilder_AllowPrivateEnvironmentProxystands up anhttptestproxy on loopback, pointsHTTP_PROXYat it, and asserts the request arrives at the proxy. On main it fails with the reporter's exact error:TestRequestBuilder_RefusePrivateNetworkWhenExcludedFromEnvironmentProxysetsNO_PROXY=feed.invalidand asserts the proxy is bypassed and the request still fails, so the exemption cannot be used to reach an arbitrary private host.TestRequestBuilder_AllowPrivateEnvironmentProxyOnRedirectpointsHTTP_PROXYandHTTPS_PROXYat two different loopback proxies and redirects from an http URL to an https one, so the hop after the redirect goes through the other proxy. Before b08db3a it failed with the same refusal, for the second proxy:Ran on Go 1.26:
go build ./...,go vet ./internal/reader/fetcher/,gofmtclean, andgo test -count=1 ./internal/...with zero failures.TestRequestBuilder_RefusePrivateNetworkByDefault,TestRequestBuilder_RefusePrivateNetworkOnRedirectand the threeAllowPrivateConfiguredProxycases still pass.Breaking changes: None. Behaviour only changes where the connection was previously refused.