Repository navigation
Add optional retry mechanism for transient network errors - #2627
Conversation
|
|
|
curl_cffi doesn’t provide the high-level retry logic that yfinance needs. It can retry some low-level transport errors, but it cannot distinguish transient vs. permanent ticker errors or propagate retry settings through threaded/bulk downloads — especially when using Threads. I download around 1500 tickers in different environments once per day, and 2–20 of them consistently fail due to timeouts. I’ve already written my own wrapper, but adding this logic directly inside the library is just a couple of lines of code and avoids having to parse curl_cffi’s textual output, which is very inconvenient. There is example: Such timed outs are always fixed by second retry. If second retry is inside of library no need to write an outer fix. |
|
Fair retort. Can it be moved into I'm working on adding a new config class here #2525 https://github.com/ranaroussi/yfinance/blob/feature/expose-exceptions-v2/yfinance/config.py. So then won't need to add any function parameters, just set once at top of script. But first I need to get it merged in. |
Done Do you need any help? I can write docs. Do all dirty job. |
|
#2525 now merged in so you can incorporate new YfConfig. You probably want to rebase to dev HEAD. |
3a70d67 to
11ff7b1
Compare
|
Done |
|
Did you explore moving the retry logic into YfData? Then all fetches will benefit. |
Retry logic has been moved from multi.py to data.py |
|
I'm happy with code. Do you mind doing a git squash? https://ranaroussi.github.io/yfinance/development/code.html#git-stuff |
bf6d629 to
542878a
Compare
|
Done |
542878a to
944d103
Compare
4e684b7 to
ec5f1c2
Compare
Summary
retriesparameter todownload()function for handling transient network errorsretries=0by default means no change to existing behaviorChanges
retriesparameter todownload()with default value 0_is_transient_error()helper function to distinguish error types_download_one()to support retry loop with exponential backoffretriesparameter through threading and sync code pathsTesting
test_download_retry_on_transient_error- verifies retry on network errorstest_download_no_retry_on_permanent_error- verifies no retry for invalid tickerUsage Examples