-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Memory efficient encoding detection #4112
Copy link
Copy link
Closed as not planned
Labels
HacktoberfestWe think it's good for https://hacktoberfest.digitalocean.com/We think it's good for https://hacktoberfest.digitalocean.com/
Metadata
Metadata
Assignees
Labels
HacktoberfestWe think it's good for https://hacktoberfest.digitalocean.com/We think it's good for https://hacktoberfest.digitalocean.com/
At #2549 @asvetlov proposed useful improvement of detection of content encoding to prevent reading the whole response in memory by allowing to specify max data size for sniffing.
UniversalDetectorcan be used: https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally.