Skip to content

The cache in version 3.17.2 does not work as expected. #160

@marioacjr

Description

@marioacjr

Describe the bug
The cache is not working. When running Skyscraper for the second time on the same platform, it downloads and overwrites all media files.

To Reproduce
1-Install the latest version of Skyscraper from scratch;
2-Create a 3DO collection with only 2 games in the folder /home/username/RetroPie/roms/3do;
3-Run Skyscraper with the command Skyscraper -p 3do -s screenscraper;
4-Run the same command again.

Expected behavior
When running the command for the second time, it is expected that the data for games already processed and present in the cache will not be downloaded and overwritten again.

Special circumstances
No special circumstances. Skyscraper is run in the simplest and cleanest way possible.

Terminal output

Running Skyscraper v3.17.2 by Lars Muldjord and contributors

Platform: '3do'
Scraping module: 'screenscraper'
Input folder: '/home/marioalves/RetroPie/roms/3do'
Game list folder: '/home/marioalves/RetroPie/roms/3do'
Covers folder: '/home/marioalves/RetroPie/roms/3do/media/covers'
Screenshots folder: '/home/marioalves/RetroPie/roms/3do/media/screenshots'
Wheels folder: '/home/marioalves/RetroPie/roms/3do/media/wheels'
Marquees folder: '/home/marioalves/RetroPie/roms/3do/media/marquees'
Textures folder: '/home/marioalves/RetroPie/roms/3do/media/textures'
Cache folder: '/home/marioalves/.skyscraper/cache/3do'

DID YOU KNOW: If you use videos and want to save some space, you can
use the "--flags symlink" flag to force Skyscraper to link the videos from the
cache to the destination instead of copying them. Beware that removing the
files from the cache will break the links though.

Forcing 1 thread as this is the anonymous limit in the ScreenScraper scraping module. Sign up for an account at https://www.screenscraper.fr and support them to gain more threads. Then use the credentials with Skyscraper using the '-u user:password' command line option or by setting 'userCreds="user:password"' in '/home/marioalves/.skyscraper/config.ini'.

Looking for optional 'priorities.xml' file in cache folder... Found!
Priorities loaded successfully!

Starting scraping run on 2 files using 1 threads.
Sit back, relax and let me do the work! :)

#1/2 (T1) Pass 1 ---- Game 'Need for Speed, The (1994)(Electronic Arts)(US)[A1115 CC 735507-2 R70]' found! :) ----
Scraper: screenscraper
From cache: NO
Search match: 100 %
Compare title: 'The Need for Speed'
Result title: 'Road & Track Presents : The Need for Speed' ()
Platform: '3DO' ()
Release Date: '1994-01-01' ()
Developer: 'Electronic Arts' ()
Publisher: 'Electronic Arts' ()
Players: '1' ()
Ages: '1+' ()
Tags: 'Racing, Driving' ()
Rating (0-1): '0.85' ()
Cover: YES ()
Screenshot: YES ()
Wheel: YES ()
Marquee: YES ()
Texture: YES ()

Description: ()
'Drive some of your favourite sports cars through wild and fast tracks, including the Dodge Viper, Lamborghini, Ferrari, Porsche, Corvette, Mazda RX7, Toyota Supra Turbo, and Acura NSX. Some of the tracks are open road, meaning you simply fly down a huge highway, with cops following in hot pursuit. One of the options in the game allows you to pick which time of the day to race in.'

'screenscraper' requests remaining: 9190

#1/2, (1/0)
Elapsed time : 00:00:21
Est. time left : 00:00:21

#2/2 (T1) Pass 1 ---- Game 'Road Rash (USA)' found! :) ----
Scraper: screenscraper
From cache: NO
Search match: 100 %
Compare title: 'Road Rash'
Result title: 'Road Rash' ()
Platform: '3DO' ()
Release Date: '1994-01-01' ()
Developer: 'New Level Software' ()
Publisher: 'Electronic Arts' ()
Players: '1' ()
Ages: '' ()
Tags: 'Racing, Driving, Motorcycle race TPV' ()
Rating (0-1): '0.9' ()
Cover: YES ()
Screenshot: YES ()
Wheel: YES ()
Marquee: YES ()
Texture: YES ()

Description: ()
'The game is the third installment in the Road Rash series, and is centered around a series of motorcycle races throughout California that the player must win to advance to higher-difficulty races, while engaging in unarmed and armed combat to hinder the other racers. 5 circuits are available, each with its own set of police, pedestrians and other cars to avoid.
'

'screenscraper' requests remaining: 9180

#2/2, (2/0)
Elapsed time : 00:00:48
Est. time left : 00:00:00

---- Resource gathering run completed! YAY! ----
Writing quick id xml, please wait... Done!
Writing 29 (29 new) resources to cache, please wait... Done!

---- And here are some neat stats :) ----
Total completion time: 00:00:48

Average search match: 100%
Average entry completeness: 96%

Total number of games: 2
Successfully processed games: 2
Skipped games: 0


Running Skyscraper v3.17.2 by Lars Muldjord and contributors

Platform: '3do'
Scraping module: 'screenscraper'
Input folder: '/home/marioalves/RetroPie/roms/3do'
Game list folder: '/home/marioalves/RetroPie/roms/3do'
Covers folder: '/home/marioalves/RetroPie/roms/3do/media/covers'
Screenshots folder: '/home/marioalves/RetroPie/roms/3do/media/screenshots'
Wheels folder: '/home/marioalves/RetroPie/roms/3do/media/wheels'
Marquees folder: '/home/marioalves/RetroPie/roms/3do/media/marquees'
Textures folder: '/home/marioalves/RetroPie/roms/3do/media/textures'
Cache folder: '/home/marioalves/.skyscraper/cache/3do'

DID YOU KNOW: Skyscraper debuted at June, 11th 2017.

Forcing 1 thread as this is the anonymous limit in the ScreenScraper scraping module. Sign up for an account at https://www.screenscraper.fr and support them to gain more threads. Then use the credentials with Skyscraper using the '-u user:password' command line option or by setting 'userCreds="user:password"' in '/home/marioalves/.skyscraper/config.ini'.

Looking for optional 'priorities.xml' file in cache folder... Found!
Priorities loaded successfully!

Starting scraping run on 2 files using 1 threads.
Sit back, relax and let me do the work! :)

#1/2 (T1) Pass 1 ---- Game 'Need for Speed, The (1994)(Electronic Arts)(US)[A1115 CC 735507-2 R70]' found! :) ----
Scraper: screenscraper
From cache: NO
Search match: 100 %
Compare title: 'The Need for Speed'
Result title: 'Road & Track Presents : The Need for Speed' ()
Platform: '3DO' ()
Release Date: '1994-01-01' ()
Developer: 'Electronic Arts' ()
Publisher: 'Electronic Arts' ()
Players: '1' ()
Ages: '1+' ()
Tags: 'Racing, Driving' ()
Rating (0-1): '0.85' ()
Cover: YES ()
Screenshot: YES ()
Wheel: YES ()
Marquee: YES ()
Texture: YES ()

Description: ()
'Drive some of your favourite sports cars through wild and fast tracks, including the Dodge Viper, Lamborghini, Ferrari, Porsche, Corvette, Mazda RX7, Toyota Supra Turbo, and Acura NSX. Some of the tracks are open road, meaning you simply fly down a huge highway, with cops following in hot pursuit. One of the options in the game allows you to pick which time of the day to race in.'

'screenscraper' requests remaining: 9170

#1/2, (1/0)
Elapsed time : 00:00:11
Est. time left : 00:00:11

#2/2 (T1) Pass 1 ---- Game 'Road Rash (USA)' found! :) ----
Scraper: screenscraper
From cache: NO
Search match: 100 %
Compare title: 'Road Rash'
Result title: 'Road Rash' ()
Platform: '3DO' ()
Release Date: '1994-01-01' ()
Developer: 'New Level Software' ()
Publisher: 'Electronic Arts' ()
Players: '1' ()
Ages: '' ()
Tags: 'Racing, Driving, Motorcycle race TPV' ()
Rating (0-1): '0.9' ()
Cover: YES ()
Screenshot: YES ()
Wheel: YES ()
Marquee: YES ()
Texture: YES ()

Description: ()
'The game is the third installment in the Road Rash series, and is centered around a series of motorcycle races throughout California that the player must win to advance to higher-difficulty races, while engaging in unarmed and armed combat to hinder the other racers. 5 circuits are available, each with its own set of police, pedestrians and other cars to avoid.
'

'screenscraper' requests remaining: 9170

#2/2, (2/0)
Elapsed time : 00:00:46
Est. time left : 00:00:00

---- Resource gathering run completed! YAY! ----
Writing quick id xml, please wait... Done!
Writing 29 (29 new) resources to cache, please wait... Done!

---- And here are some neat stats :) ----
Total completion time: 00:00:46

Average search match: 100%
Average entry completeness: 96%

Total number of games: 2
Successfully processed games: 2
Skipped games: 0

Technical information

  • Skyscraper version: 3.17.2
  • Configuration switches applied: None
  • Internet connection: Wfi, Fiber
  • OS and Version: Ubuntu Release 22.04.5 LTS (Jammy Jellyfish) 64-bit

Additional context
Apparently, the issue does not exist in version 3.16.1 and earlier. The problem occurs only in version 3.17.0 and later."

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions