Skip to content

Request for review / play testing - new savegame format#168

Merged
nthykier merged 7 commits intosingularity:masterfrom
nthykier:issue-145-new-savegame-format
Aug 10, 2019
Merged

Request for review / play testing - new savegame format#168
nthykier merged 7 commits intosingularity:masterfrom
nthykier:issue-145-new-savegame-format

Conversation

@nthykier
Copy link
Member

@nthykier nthykier commented Aug 5, 2019

I have taken a serious go at #145 and would like feed back on it before merging it into master.

The branch here provides a fully functional savegame format on a JSON-based file (with a custom header - both in-game time and difficulty is listed in the metadata). In the branch, the game will save both an old-style and a new style savegame file (to enable comparison/testing). The format only stores the essential data and recomputes everything else from scratch. As JSON only works with unicode strings, everything loaded from the game is pure unicode (note, some fields in.

Based on #145, there are the following known omissions:

  • Add Location, Event and Tech list to Player: Was not required to complete the rewrite, so I punted it. It can still be done after the merge of this branch.
  • New format: String + Json with GZIP: At the moment, the game is not GZip'ed. I opted for omitting this during the review as it makes it much easier to debug the savegame without having to fiddle with gzip tooling. I will add it before merge.

@nthykier
Copy link
Member Author

nthykier commented Aug 5, 2019

@sunfall I have not forgotten your idea of using an internal stable "savegame only" ID for everything. However, I would like some initial testing on this approach before investing time on that (to avoid "perfect being the enemy of good").

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

7 similar comments
@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #168 into master will decrease coverage by 0.1%.
The diff coverage is 28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.64%   30.53%   -0.11%     
==========================================
  Files          35       35              
  Lines        4866     4949      +83     
==========================================
+ Hits         1491     1511      +20     
- Misses       3375     3438      +63
Impacted Files Coverage Δ
code/g.py 48.25% <100%> (+0.6%) ⬆️
code/base.py 21.05% <15.38%> (-0.3%) ⬇️
code/location.py 35.08% <16.66%> (-3.46%) ⬇️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/effect.py 18.18% <25%> (-0.43%) ⬇️
code/event.py 35.41% <25%> (-3.05%) ⬇️
code/item.py 41.37% <27.27%> (-1.62%) ⬇️
code/group.py 31.25% <33.33%> (+0.16%) ⬆️
code/tech.py 56.09% <40%> (-3.37%) ⬇️
code/data.py 51.31% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b8febd...4154eb8. Read the comment docs.

@codecov-io
Copy link

codecov-io commented Aug 6, 2019

Codecov Report

Merging #168 into master will decrease coverage by 0.04%.
The diff coverage is 25.64%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
- Coverage   30.69%   30.64%   -0.05%     
==========================================
  Files          35       35              
  Lines        4920     4986      +66     
==========================================
+ Hits         1510     1528      +18     
- Misses       3410     3458      +48
Impacted Files Coverage Δ
code/g.py 48.82% <100%> (+0.6%) ⬆️
code/location.py 34.74% <13.63%> (-4.43%) ⬇️
code/base.py 21.53% <14.28%> (+0.19%) ⬆️
code/buyable.py 51.9% <18.75%> (-4.99%) ⬇️
code/event.py 31.25% <19.04%> (-2.09%) ⬇️
code/item.py 44.44% <25%> (-1.86%) ⬇️
code/effect.py 19.6% <33.33%> (-0.4%) ⬇️
code/group.py 33.8% <33.33%> (+2.72%) ⬆️
code/tech.py 62.85% <40%> (+3.39%) ⬆️
code/data.py 51.42% <62.5%> (+0.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee19c4d...d302fc3. Read the comment docs.

@nthykier
Copy link
Member Author

nthykier commented Aug 6, 2019

Added gzip'ed files. Load transparently handles either - save defaults to gzipped files except in debug mode where it uses the raw json instead.

The JSON can be extracted from a savegame with the following Linux shell pipeline:

tail -n1 $SAVEGAME_FILE_GOES_HERE | base64 -d | gzip -dc 

Example:

tail -n1 ~/.local/share/singularity/saves/save-test-1-gzip.s2 | base64 -d | gzip -dc 

@nthykier nthykier force-pushed the issue-145-new-savegame-format branch from 2cb2ede to 1e665c9 Compare August 6, 2019 07:37
@nthykier nthykier force-pushed the issue-145-new-savegame-format branch 8 times, most recently from 17f5b14 to e143dc4 Compare August 9, 2019 07:54
This enables us to skip oneshot effects and event dialogs when loading
save games.

Signed-off-by: Niels Thykier <[email protected]>
@nthykier nthykier force-pushed the issue-145-new-savegame-format branch from 9e4e86f to e1deb4e Compare August 10, 2019 08:14
Load transparently handles gzip'ed and gunzip'ed files (because it was
trivial to do).  Save by default uses gzip'ed files unless the game
runs in debug mode.

Closes: singularity#145
Signed-off-by: Niels Thykier <[email protected]>
Instead of using the pickled objects, we instead message them into a
format similar to that of the JSON path and use the new style loading.
This enables to restore savegame compatibility with many old savegame
formats (including 0.30c).

Closes: singularity#122
Signed-off-by: Niels Thykier <[email protected]>
@nthykier nthykier force-pushed the issue-145-new-savegame-format branch from e1deb4e to d302fc3 Compare August 10, 2019 08:17
@nthykier nthykier merged commit 70bf266 into singularity:master Aug 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants