Curl has been supporting this for a while (says here), and it is also possible with requests by overriding the proxy_headers() method of HTTPAdapter.

How hard would it be to implement this in aiohttp?