Android screenshot tests - #2809
Merged
riccardobl merged 122 commits intoJun 8, 2026
Merged
Conversation
…a issues in screenshots
…ility in the pipelines
…t-tests-2 # Conflicts: # gradle/libs.versions.toml # settings.gradle
|
🖼️ Screenshot tests have failed. The purpose of these tests is to ensure that changes introduced in this PR don't break visual features. They are visual unit tests. 📄 Where to find the report:
✅ If you did mean to change things: ✨ If you are creating entirely new tests: Note; it is very important that the committed reference images are created on the build pipeline, locally created images are not reliable. Similarly tests will fail locally but you can look at the report to check they are "visually similar". See https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-screenshot-tests/README.md for more information Contact @richardTingle (aka richtea) for guidance if required |
…t-tests-2 # Conflicts: # gradle/libs.versions.toml # jme3-screenshot-tests/build.gradle # settings.gradle
richardTingle
marked this pull request as ready for review
June 6, 2026 12:32
Copilot AI
pushed a commit
that referenced
this pull request
Jul 15, 2026
* Make the screenshot tests run in an off frame buffer and correct gamma issues in screenshots * Make the definition of "the same" a bit more flexible to avoid instability in the pipelines * Restore gamma correction * Set the colourspace for the off renderTexture * Set the Srgb for the off renderTexture * Rebake minor visual changes for off frame buffer * Rearrange for submodule screenshot structure * Add in dependencies for android * Bring it closer to the android initialiser project * Temporarily use non local android jme * Simple screenshot test * Wait for emulator * Dont save a snapshot, use mesa * Make the emulator logs less likely to hang * Bit closer to the android-emulator-runner docs * Add logging in test * Try a more modern pixel copy * Capture logcat * Only get my applications logcat * Pull logs inside app step * Prevent snapshots (which are erroring) * Actually grab the logcat * Use the actual location of the file * Save file to public directory * The whole logcat isn't too long * Pull image from quoted location * Whole log cat actually a bit much to put in github logs * Use latch to wait explicitly for write to complete * Clean up the test code * Go back to scoped storage as permissions do not allow downloads folder * Pull file from within test * Use explicit file location * Revert "Use explicit file location" This reverts commit b0ed44d. * Revert "Pull file from within test" This reverts commit b9d850c. * Avoid auto unistall (which means I can't get the file) * Try to get JME to render into the test * Fresh emulator every time * Take the screenshot from the glSurfaceView * Move the screenshot framework to a shared environment * Extract out one of the tests ready for reuse * Start introducing shared android screenshot functionality * Allow for test scenarios to be booted to both android and desktop * Start splitting out android specific elements * Introduce androidx-storage to save the report persistently * Change from BufferedImage to JME Image for android compatibility * Work towards allowing android to write a proto report * Hook in workflow for android screenshots * Hook in workflow for android screenshots * Clean up ExtentReportExtensionJunit4 * Avoid reuse of AssetManager (as we are registering random folders) * Give gradle more heap for build (specifically for the android build) * Remove redundant request to write to external storage * Run the adb pulls even if the gradle step fails * Add the upgrade step for the report * Use more modern android for jackson support * Use x86_64 android * Use x86_64 android * Even more modern android * Try 35 as docs only go up to 35 * Remove the matrix as we only run one * Install test services before running test * Debug error codes * Create report directory before pulling to it * Format the android screenshot tests readme * Error if upgrade of report fails * Correct upgrade code to access the files with absolute paths * Try to trap the error code better * One giant line of doom * One giant line of doom 2 * Just say -1 for a fail * Ensure report has a title * Reinstate deserialisation constructor * Migrate all tests to be android & desktop tests * Ensure that we get a single test report * Run the whole thing in a single shell * Include logcat * Increase ram for emulator * Allow app to have large heap * Even larger heap * Remove some of the more memory intensive tests * Remove some more of the more memory intensive tests * Correct the calculateImageFilePrefix so it is flexible to thread depth which android can mess with) * Remove more memory intensive tests * Try for a wider device * Try to find available profiles * Try to find available profiles * Initialise the reference images * Load references images from resources (to make android happy) * Correct colours are ints, not bytes (as unsigned) * Use consistent pixel comparison * Log pixel errors * Fix inverted "very similar" test * Set android tests to only run on push * Restore android harness as it was * Self review * Single subshell to avoid EOF issues * Avoid richard actually being contacted by the message * Wrap whole shell in a > * Natives moved out of project * More recent java version * Update location of desktop screenshot tests * Fix AndroidTestHarness for change in AndroidHarnessFragment * Ensure file load still works for screenshot tests * Move reference files within normal resources * All desktop screenshot tests with their correct base * Use same layout as in the android example * Simpler layout * Correct javadoc * Correct attached image save location * Gen the generated images back from the android run if they are different sizes * .png suffix on changeed images * Restore accidentally changed ScenarioPostWater values * Rebake images at new size * Rebake images at new size * Add missing copywright notices * Add missing reference images * Don't prematurely fail test, allow all images to be collected * Add missing images * Replace ImagePixelWrapper with the Raster for more cross format pixel manipulation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow a subset of the screenshot tests to be run on Android (a subset because some are too heavy and cause out of memory errors on the emulated android)
Extent reports doesn't work on Android so creates a proto report within android and later upgrades that to a proper report within the pipeline