-
Notifications
You must be signed in to change notification settings - Fork 27
ScreenScraper Timeout Issue Report: Super Metroid #215
Description
Issue Summary
I am experiencing consistent request timeouts from ScreenScraper's API specifically when scraping Super Metroid (Japan, USA) (En,Ja).sfc using Skyscraper v3.18.3. The issue persists even though the ROM's checksums (CRC/MD5/SHA1) are correct and match entries in the database. Other ROMs scrape successfully without issue.
Environment
- Software: Skyscraper v3.18.3.aarch64 (using with Scrappy frontend - https://github.com/saitamasahil/scrappy)
- Scraper Module: screenscraper
- Platform: SNES
- ROM: Super Metroid (Japan, USA) (En,Ja).sfc
- ROM Hashes:
- CRC: D63ED5F8
- MD5: 21F3E98DF4780EE1C667B84E57D88675
- SHA1: DA957F0D63D14CB441D215462904C4FA8519C613
Logs & Evidence
Below are the relevant debug logs showing the timeout loop when attempting to scrape this specific file:
[DEBUG] Platform: 'snes'
[DEBUG] Scraping module: 'screenscraper'
[DEBUG] Input folder: '/mnt/union/ROMS/Super Nintendo Entertainment System'
[DEBUG] Setting threads to 1 as allowed for your Screenscraper account.
[DEBUG] Search name #1: 'crc=D63ED5F8&md5=21F3E98DF4780EE1C667B84E57D88675&sha1=DA957F0D63D14CB441D215462904C4FA8519C613&romnom=Super%20Metroid%20(Japan%2C%20USA)%20(En%2CJa).sfc&romtaille=3145728'
[DEBUG] Starting scraping run on 1 files using 1 threads.
[DEBUG] Request timed out after 60 secs, server might be busy / overloaded...
[DEBUG] WARN: QIODevice::read (QNetworkReplyHttpImpl): device not open
[DEBUG] Retrying request...
[DEBUG] Request timed out after 60 secs, server might be busy / overloaded...
...
[DEBUG] Request timed out after 60 secs, server might be busy / overloaded...
[DEBUG] Game 'Super Metroid (Japan, USA) (En,Ja)' not found :(
Observations
- Request Formation: The client correctly sends all checksums (
crc,md5,sha1) and filename. - Behavior: The API does not return "Not Found" instantly; instead, it hangs for 60 seconds until timeout, then retries and fails again.
- Comparison: Other ROMs scrape fine immediately before/after this attempt, suggesting the issue is specific to this query or ROM entry in the database.
- Impact: Causes a 4-5 minute hang in the scraping frontend before finally failing.
Steps to Reproduce
- Use Skyscraper v3.18.3
- Attempt to scrape "Super Metroid (Japan, USA) (En,Ja).sfc" (CRC:
D63ED5F8) using-s screenscraper. - Observe the timeout.
I also found that someone else also faced this issue on ES-DE(which also uses skyscraper as backend) & posted on reddit - https://www.reddit.com/r/RetroArch/comments/1qafr4p/scraper_can_not_find_anything_for_super_metroid/
However, the user deleted that post. Title is still visible & comments can still be read.