Adding take a screenshot func for images of screen elements in mobile#4222
Conversation
WalkthroughThe update enhances the Changes
Sequence Diagram(s)sequenceDiagram
participant TestRunner
participant GenericAppiumDriver
participant AppiumDriver
participant UIElement
TestRunner->>GenericAppiumDriver: GetVisibleControls()
GenericAppiumDriver->>AppiumDriver: GetPageSource()
GenericAppiumDriver->>AppiumDriver: GetScreenshot()
loop For each visible element
GenericAppiumDriver->>AppiumDriver: FindElementByXPath()
GenericAppiumDriver->>UIElement: GetLocationAndSize()
GenericAppiumDriver->>GenericAppiumDriver: TakeElementScreenShot(element)
GenericAppiumDriver->>UIElement: Assign cropped screenshot (base64)
end
GenericAppiumDriver-->>TestRunner: Return list of controls with screenshots
Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches
🧪 Generate Unit Tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
4448-4461: 🛠️ Refactor suggestion
remoteFilePathvariable name is misleading & missing directory-existence check
remoteFilePathactually points to a local file (concatenated withLocalFolderPath).
Rename to avoid confusion and ensure the target directory exists beforeWriteAllBytes:- var remoteFilePath = Path.Combine(LocalFolderPath, Path.GetFileName(DeviceFilePath)); + var localFilePath = Path.Combine(LocalFolderPath, Path.GetFileName(DeviceFilePath)); + Directory.CreateDirectory(Path.GetDirectoryName(localFilePath)); ... - File.WriteAllBytes(remoteFilePath, FileContent); + File.WriteAllBytes(localFilePath, FileContent);If the directory is missing the current implementation throws
DirectoryNotFoundException.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(5 hunks)
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (2)
2568-2590:fullScreenshotis still never disposed – unmanaged handle leak re-introducedYou moved the
Screenshotfield to a class-level variable and keep a reference afterGetVisibleControlsreturns.
OpenQA.Selenium.Screenshotwraps an internalBitmap/GDI handle; keeping many instances alive → GDI exhaustion and crashes on long POM scans.The fix that was suggested in the previous review is still relevant:
- Screenshot fullScreenshot = null; + // Capture once inside GetVisibleControls and wrap with using + using Screenshot fullScreenshot = ((ITakesScreenshot)Driver).GetScreenshot();Or cache one
Bitmap(already converted) andDispose()it in afinallyblock.
2660-2693: Bitmap produced for EVERY element → O(n²) memory/CPU
TakeElementScreenShotconverts the sameScreenshotto aBitmapinside ausingblock for every single element.
On pages with hundreds of nodes this multiplies memory & CPU needlessly.Reuse a single
Bitmap:- EI.ScreenShotImage = TakeElementScreenShot(EI, fullScreenshot); + // Convert once + using var fullImage = ScreenshotToImage(fullScreenshot); + ... + EI.ScreenShotImage = TakeElementScreenShot(EI, fullImage);Then change
TakeElementScreenShotto accept aBitmapinstead of aScreenshot.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(5 hunks)
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (2)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (2)
2589-2589: Memory-safety concern:ScreenshotToImagereturns aBitmapbacked by a disposed stream
using var fullImage = ScreenshotToImage(...);disposesfullImagecorrectly, but the helper still instantiates theBitmapfrom aMemoryStreamthat is disposed inside the helper. The resulting bitmap keeps a reference to that stream and will trigger “A generic error occurred in GDI+” on the first access.See past review at lines 2721-2727 – issue not yet fixed.
2721-2727:ScreenshotToImagestill disposes the backing stream too earlyReturning
new Bitmap(ms)inside ausingdisposesms, leaving the bitmap unusable later.-return new Bitmap(ms); +var bmp = new Bitmap(ms); +return (Bitmap)bmp.Clone(); // decouple from streamor load via
Image.FromStream(ms, true, true)then clone.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(5 hunks)
There was a problem hiding this comment.
Actionable comments posted: 2
🔭 Outside diff range comments (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
70-78: Remove unusedSystem.Linq.ExpressionsimportThe namespace isn’t referenced anywhere in this file; keeping it triggers CS8019 and adds needless compile time.
Just delete the directive.
♻️ Duplicate comments (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
2692-2703: Clamp & simplify cropping logic
cropRectcan still point outsidefullImagewhen the element is partially off-screen, throwingGDI+errors.
Before creating the rectangle, clamp X/Y/Width/Height to the bitmap size, then use the cheaper clone API:-Rectangle cropRect = new Rectangle(cropX, cropY, cropWidth, cropHeight); -using (Bitmap elementImage = new Bitmap(cropRect.Width, cropRect.Height)) -{ - using (Graphics g = Graphics.FromImage(elementImage)) - { - g.DrawImage(fullImage, new Rectangle(0,0,cropRect.Width,cropRect.Height), cropRect, GraphicsUnit.Pixel); - } +int x = Math.Max(0, cropX); +int y = Math.Max(0, cropY); +int w = Math.Min(cropWidth, fullImage.Width - x); +int h = Math.Min(cropHeight, fullImage.Height - y); +if (w <= 0 || h <= 0) return null; +Rectangle cropRect = new Rectangle(x, y, w, h); +using (Bitmap elementImage = fullImage.Clone(cropRect, fullImage.PixelFormat)) { @@ }This removes the extra
Graphicsallocation and guards against out-of-range coordinates.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(10 hunks)
🧰 Additional context used
🧠 Learnings (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
undefined
<retrieved_learning>
Learnt from: prashelke
PR: #4232
File: Ginger/GingerCoreNET/ActionsLib/UI/VisualTesting/VRTAnalyzer.cs:219-226
Timestamp: 2025-06-16T10:13:01.319Z
Learning: In VRTAnalyzer.cs for the GenericAppiumDriver class, the mAct.IsFullPageScreenshot flag is not applicable and should not be used to control screenshot behavior. GenericAppiumDriver always uses CaptureFullPageCroppedScreenshot() method regardless of the IsFullPageScreenshot flag setting.
</retrieved_learning>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
2669-2716: Add bounds clamping to prevent exceptions.The cropping logic should validate that the crop rectangle doesn't exceed the full image bounds to prevent
ArgumentExceptionwhen elements are partially off-screen.Rectangle cropRect = new Rectangle(cropX, cropY, cropWidth, cropHeight); + +// Clamp bounds to prevent exceptions +int x = Math.Max(0, cropX); +int y = Math.Max(0, cropY); +int w = Math.Max(0, Math.Min(cropWidth, fullImage.Width - x)); +int h = Math.Max(0, Math.Min(cropHeight, fullImage.Height - y)); + +if (w <= 0 || h <= 0) + return null; + +Rectangle cropRect = new Rectangle(x, y, w, h);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(10 hunks)
🧰 Additional context used
🧠 Learnings (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (1)
undefined
<retrieved_learning>
Learnt from: prashelke
PR: #4232
File: Ginger/GingerCoreNET/ActionsLib/UI/VisualTesting/VRTAnalyzer.cs:219-226
Timestamp: 2025-06-16T10:13:01.319Z
Learning: In VRTAnalyzer.cs for the GenericAppiumDriver class, the mAct.IsFullPageScreenshot flag is not applicable and should not be used to control screenshot behavior. GenericAppiumDriver always uses CaptureFullPageCroppedScreenshot() method regardless of the IsFullPageScreenshot flag setting.
</retrieved_learning>
🔇 Additional comments (4)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (4)
1300-1326: LGTM! Improved biometrics simulation structure.The restructured switch statement with explicit cases for Success, Failure, and Cancel provides better readability and maintainability compared to the previous implementation.
2589-2589: Excellent performance optimization.Taking the screenshot once and converting to Bitmap upfront prevents repeated conversions for each element, significantly improving performance during POM discovery.
2760-2767: LGTM! Proper bitmap cloning implementation.The
ScreenshotToImagemethod correctly clones the bitmap to avoid disposal issues with the underlying MemoryStream. This prevents potential GDI+ errors.
2645-2656: Make XPath check case-insensitive.The XPath checks using
Contains("android")andContains("XCUIElement")are case-sensitive, which may cause elements with differently cased strings to be skipped.-if ((EI.XPath).Contains("android") || (EI.XPath).Contains("XCUIElement")) +if (EI.XPath.Contains("android", StringComparison.OrdinalIgnoreCase) || + EI.XPath.Contains("XCUIElement", StringComparison.OrdinalIgnoreCase))⛔ Skipped due to learnings
Learnt from: prashelke PR: Ginger-Automation/Ginger#4222 File: Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs:2645-2656 Timestamp: 2025-06-19T05:08:45.170Z Learning: In the Ginger mobile automation framework (GenericAppiumDriver.cs), XPath checks for "android" and "XCUIElement" should remain case-sensitive as currently implemented. The team prefers not to make these checks case-insensitive.
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs(9 hunks)
🧰 Additional context used
🧠 Learnings (1)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (2)
undefined
<retrieved_learning>
Learnt from: prashelke
PR: #4232
File: Ginger/GingerCoreNET/ActionsLib/UI/VisualTesting/VRTAnalyzer.cs:219-226
Timestamp: 2025-06-16T10:13:01.319Z
Learning: In VRTAnalyzer.cs for the GenericAppiumDriver class, the mAct.IsFullPageScreenshot flag is not applicable and should not be used to control screenshot behavior. GenericAppiumDriver always uses CaptureFullPageCroppedScreenshot() method regardless of the IsFullPageScreenshot flag setting.
</retrieved_learning>
<retrieved_learning>
Learnt from: prashelke
PR: #4222
File: Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs:2645-2656
Timestamp: 2025-06-19T05:08:45.170Z
Learning: In the Ginger mobile automation framework (GenericAppiumDriver.cs), XPath checks for "android" and "XCUIElement" should remain case-sensitive as currently implemented. The team prefers not to make these checks case-insensitive.
</retrieved_learning>
🔇 Additional comments (3)
Ginger/GingerCoreNET/Drivers/CoreDrivers/Mobile/Appium/GenericAppiumDriver.cs (3)
2588-2589: 👍 Full-page image is captured only once – good optimisationCapturing the screenshot once and re-using the
Bitmapremoves hundreds of device round-trips. Nice catch.
2644-2656: Graceful fallback when element screenshot failsThe new try/catch prevents a single bad node from killing the learn loop – good defensive coding.
2759-2766: Good: bitmap completely detached from the streamCloning after
new Bitmap(ms)avoids the classic GDI+ “generic error” caused by a disposed stream. Nice.
Thank you for your contribution.
Before submitting this PR, please make sure:
Summary by CodeRabbit