-
Notifications
You must be signed in to change notification settings - Fork 8
feat: support scale parameter for mjpeg #110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Warning Rate limit exceeded@gmegidish has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 2 minutes and 35 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
WalkthroughThis change refactors device method signatures to accept configuration structs instead of individual parameters or no-argument calls. Two new types— Changes
Sequence Diagram(s)sequenceDiagram
participant Caller
participant Device
participant DeviceImpl as Device Impl<br/>(Android/iOS/Sim)
Caller->>Device: StartAgent(StartAgentConfig{<br/> OnProgress: callback<br/>})
activate Device
Device->>DeviceImpl: StartAgent(config)
activate DeviceImpl
DeviceImpl->>Device: config.OnProgress("Launching...")
Device-->>Caller: progress event
DeviceImpl->>DeviceImpl: core startup logic
DeviceImpl-->>Device: done/error
deactivate DeviceImpl
deactivate Device
Caller->>Device: StartScreenCapture(ScreenCaptureConfig{<br/> Format: "mjpeg",<br/> OnProgress: callback,<br/> OnData: callback<br/>})
activate Device
Device->>DeviceImpl: StartScreenCapture(config)
activate DeviceImpl
DeviceImpl->>Device: config.OnProgress("Starting stream...")
Device-->>Caller: progress event
loop Frame streaming
DeviceImpl->>Device: config.OnData(frameData)
Device-->>Caller: frame data
end
deactivate DeviceImpl
deactivate Device
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (8)
commands/orientation.go (1)
33-36: Orientation commands correctly updated; consider small helper to reduce repetitionBoth orientation commands now use
device.StartAgent(devices.StartAgentConfig{}), which is correct for the new API. Given this pattern appears in many commands, you might later extract a small helper likeStartAgentOrError(device)to centralize the boilerplate and error message wording, but it’s not required for this PR.Also applies to: 63-66
commands/screenshot.go (1)
11-13: Screenshot command migration to StartAgentConfig is soundImporting
devicesand callingtargetDevice.StartAgent(devices.StartAgentConfig{})aligns with the new interface, and leaves the screenshot behavior intact. If you later add longer-running capture steps here, this would be a natural place to thread anOnProgresscallback through the config.Also applies to: 57-60
commands/input.go (1)
7-9: Input commands consistently migrated to config-based StartAgentAll interaction commands now call
targetDevice.StartAgent(devices.StartAgentConfig{})with unchanged error handling, which is exactly what the new interface requires. Given how many commands share the same “find device → start agent → perform action” pattern, you might later introduce a small shared helper to reduce duplication, but it’s fine to keep as-is for now.Also applies to: 63-66, 89-92, 115-118, 141-144, 167-170, 208-211
cli/device.go (1)
7-8: CLI device info uses new StartAgentConfig correctly; consider avoiding double-startThe
deviceInfoCmdnow callstargetDevice.StartAgent(devices.StartAgentConfig{}), which is the right way to use the new interface. Note thatcommands.InfoCommandalso does its ownFindDeviceOrAutoSelect+StartAgent, so the agent will typically be started twice on this path. That behavior predates this PR but could be tightened up later by letting either the CLI or the command package own agent startup, not both.Also applies to: 50-56
devices/common.go (1)
11-18: Config types and interface changes look solid; add central normalization for ScreenCaptureConfigThe introduction of:
ScreenCaptureConfig(withFormat,Quality,Scale,OnProgress,OnData),StartAgentConfig(withOnProgress),- and the corresponding
ControllableDeviceupdatesis a good fit for the MJPEG use case and future extensibility.
Given you now have
DefaultMJPEGQuality,DefaultMJPEGScale(0.1–1.0 range), andDefaultMJPEGFrameratehere, consider adding a small helper in this package to normalizeScreenCaptureConfig(e.g., apply defaults when fields are zero, and clampQuality/Scaleinto supported ranges). That keeps device-specific implementations from re‑implementing slightly different validation logic and ensures the defaults are applied consistently everywhere.Something along these lines in this package would be enough, to be called by each implementation:
+func NormalizeScreenCaptureConfig(cfg ScreenCaptureConfig) ScreenCaptureConfig { + if cfg.Quality <= 0 || cfg.Quality > 100 { + cfg.Quality = DefaultMJPEGQuality + } + if cfg.Scale <= 0 { + cfg.Scale = DefaultMJPEGScale + } + // optional: clamp maximum scale or handle missing format defaults here + return cfg +}Also applies to: 20-33, 39-69
cli/screenshot.go (1)
88-98: Consider exposing quality as a CLI flag for consistency.The
scaleparameter is now configurable via--scale, butqualityis hardcoded todevices.DefaultMJPEGQuality. For consistency with the screenshot command (which has--quality), consider adding a--qualityflag for screencapture as well.This is optional since the current default (80) is reasonable.
devices/ios.go (1)
280-284: Progress message sent before returning error may be misleading.The progress callback emits "Installing WebDriverAgent" but immediately returns an error stating WebDriverAgent is not installed. This could confuse users who see "Installing..." but then get an error instead of actual installation.
Consider either:
- Removing this progress callback since no installation happens here
- Or changing the message to something like "WebDriverAgent not found"
if webdriverBundleId == "" { - if config.OnProgress != nil { - config.OnProgress("Installing WebDriverAgent") - } return fmt.Errorf("WebDriverAgent is not installed") }server/server.go (1)
740-744: Use consistent logging for write errors.Line 742 uses
fmt.Printlnwhile the rest of the server useslog.Printf(line 195). Consider using consistent logging.OnData: func(data []byte) bool { _, writeErr := w.Write(data) if writeErr != nil { - fmt.Println("Error writing data:", writeErr) + log.Printf("Error writing data: %v", writeErr) return false }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (13)
cli/device.go(2 hunks)cli/screenshot.go(3 hunks)commands/dump.go(1 hunks)commands/info.go(1 hunks)commands/input.go(7 hunks)commands/orientation.go(3 hunks)commands/screenshot.go(2 hunks)commands/url.go(2 hunks)devices/android.go(4 hunks)devices/common.go(3 hunks)devices/ios.go(4 hunks)devices/simulator.go(7 hunks)server/server.go(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (12)
commands/dump.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
commands/screenshot.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
cli/device.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
commands/url.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
commands/orientation.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
devices/ios.go (1)
devices/common.go (3)
StartAgentConfig(30-32)ScreenCaptureConfig(21-27)DefaultMJPEGFramerate(17-17)
commands/info.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
devices/simulator.go (2)
devices/common.go (2)
StartAgentConfig(30-32)ScreenCaptureConfig(21-27)devices/wda/mjpeg/types.go (1)
NewWdaMjpegClient(7-11)
devices/android.go (2)
devices/common.go (2)
StartAgentConfig(30-32)ScreenCaptureConfig(21-27)utils/logger.go (1)
Verbose(19-23)
cli/screenshot.go (2)
devices/common.go (4)
StartAgentConfig(30-32)DefaultMJPEGScale(15-15)ScreenCaptureConfig(21-27)DefaultMJPEGQuality(13-13)commands/commands.go (1)
NewErrorResponse(26-31)
server/server.go (1)
devices/common.go (2)
StartAgentConfig(30-32)ScreenCaptureConfig(21-27)
commands/input.go (1)
devices/common.go (1)
StartAgentConfig(30-32)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: build_on_windows
- GitHub Check: build_on_macos
- GitHub Check: build_on_linux
🔇 Additional comments (11)
commands/dump.go (1)
27-30: StartAgent config wiring looks correctUsing
targetDevice.StartAgent(devices.StartAgentConfig{})cleanly adapts to the new signature without behavior change; error handling is preserved.commands/url.go (1)
26-29: URL command correctly migrated to config-based StartAgent
targetDevice.StartAgent(devices.StartAgentConfig{})matches the new interface and keeps the existing flow and error reporting intact.commands/info.go (1)
20-23: InfoCommand correctly adapted to the new StartAgent signatureSwitching to
targetDevice.StartAgent(devices.StartAgentConfig{})preserves existing semantics and integrates cleanly with the new config type.devices/android.go (2)
387-395: LGTM - StartAgent signature updated correctly.The config parameter is accepted for interface consistency, though Android doesn't require agent startup for online devices. The offline device check is appropriate.
638-694: LGTM - Screen capture implementation with scale parameter.The implementation correctly:
- Validates format is "mjpeg"
- Uses nil-safe progress callbacks
- Passes scale to the MJPEG server command with proper formatting
- Handles the OnData callback return value to stop streaming
One consideration: The function doesn't validate if
config.Scaleis positive. If a caller passes 0 or negative values, they'll be passed directly to the MJPEG server. The CLI handles this with defaults, but direct API callers might not.cli/screenshot.go (1)
77-110: LGTM - Config-based API integration.The CLI properly:
- Uses config structs for both
StartAgentandStartScreenCapture- Provides progress feedback via stderr
- Handles default scale value correctly
- Streams data to stdout with proper error handling
devices/ios.go (1)
242-366: LGTM - StartAgent with progress callbacks.The implementation correctly adds progress notifications at key stages:
- Starting tunnel
- Launching WebDriverAgent
- Waiting for agent to start
All callbacks are nil-safe.
server/server.go (1)
716-752: LGTM - Screen capture streaming with progress notifications.The implementation elegantly:
- Embeds progress messages as JSON parts within the MJPEG multipart stream
- Uses proper MIME boundaries and content-length headers
- Flushes after each write for real-time streaming
- Properly handles the OnData callback to stop on write errors
devices/simulator.go (3)
651-669:config.Qualityandconfig.Scaleare not used for Simulator screen capture.Same as the iOS real device implementation, the Simulator ignores
QualityandScaleparameters from the config. This should be addressed consistently across both iOS implementations.
304-320: LGTM - Progress callbacks for WebDriverAgent installation.The
onProgresscallback properly notifies users during the download and install phases, which can take significant time.
458-557: LGTM - StartAgent with config-driven progress callbacks.The implementation correctly:
- Handles various simulator states (Booted, Shutdown, Booting, ShuttingDown)
- Passes progress callback to InstallWebDriverAgent
- Emits progress for launching and waiting phases
- All callbacks are nil-safe
| func (d IOSDevice) StartScreenCapture(config ScreenCaptureConfig) error { | ||
| // configure mjpeg framerate | ||
| err := d.wdaClient.SetMjpegFramerate(DefaultMJPEGFramerate) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| return d.mjpegClient.StartScreenCapture(format, callback) | ||
| if config.OnProgress != nil { | ||
| config.OnProgress("Starting video stream") | ||
| } | ||
|
|
||
| return d.mjpegClient.StartScreenCapture(config.Format, config.OnData) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
config.Quality and config.Scale are not used for iOS screen capture.
The ScreenCaptureConfig struct includes Quality and Scale fields, but the iOS implementation only uses Format and OnData. This creates an inconsistency where these parameters work on Android but are silently ignored on iOS.
Either:
- Document that Quality/Scale are Android-only
- Implement Quality/Scale support for iOS if WDA supports it
- Return an error if non-default Quality/Scale values are passed on iOS
🌐 Web query:
Does WebDriverAgent MJPEG server support quality and scale parameters?
💡 Result:
Yes. WebDriverAgent's MJPEG server supports both image quality and scaling parameters — exposed as settings/capabilities such as mjpegServerScreenshotQuality (or mjpegServerScreenshotQuality / screenshotQuality) and mjpegScalingFactor (mjpegScalingFactor). Examples in Appium/WDA code and docs show these options. [1] [2] [3]
Sources
- Appium XCUITest driver capabilities docs (mjpeg/screenshot options). [1]
- appium-webdriveragent type definitions (mjpegServerScreenshotQuality, mjpegScalingFactor). [2]
- WebDriverAgent/FBMjpegServer references and config (screenshot quality / scaling). [3]
I need to examine the codebase to understand the full extent of this issue. Now that web search confirms WDA supports these parameters, I should verify how they can be passed through the iOS client implementation.
🏁 Script executed:
#!/bin/bash
# Find the mjpegClient type and StartScreenCapture method signature
rg -nP --type=go '\bStartScreenCapture\s*\(' -A5 -B2 | head -80Length of output: 3916
🏁 Script executed:
#!/bin/bash
# Check Android implementation for comparison
rg -nP --type=go 'func.*Android.*StartScreenCapture' -A15 | head -60Length of output: 889
🏁 Script executed:
#!/bin/bash
# Look for wdaClient methods related to quality/scale/framerate
rg -nP --type=go 'wdaClient\.(Set|Get).*' -C2 | grep -i 'quality\|scale\|framerate' | head -40Length of output: 268
🏁 Script executed:
#!/bin/bash
# Find ScreenCaptureConfig struct definition
ast-grep --pattern 'type ScreenCaptureConfig struct {
$$$
}'Length of output: 373
I need to check the wdaClient interface to see if there are methods to set quality and scale parameters:
🏁 Script executed:
#!/bin/bash
# Find wdaClient interface definition
rg -nP --type=go 'type.*[Cc]lient.*interface' -A30 | head -100Length of output: 0
🏁 Script executed:
#!/bin/bash
# Search for methods on wdaClient that might set quality or scaling
rg -nP --type=go 'SetMjpeg|SetScreenshot|SetQuality|SetScale|SetScaling' -B2 -A5Length of output: 1251
🏁 Script executed:
#!/bin/bash
# Check WdaMjpegClient implementation - see all methods
rg -nP --type=go 'func \(c \*WdaMjpegClient\)' -A3Length of output: 786
🏁 Script executed:
#!/bin/bash
# Find where wdaClient is instantiated/used to understand its type
rg -nP --type=go 'wdaClient.*:=' -B2 -A5 | head -80Length of output: 0
I need to examine SetAppiumSettings and check if quality/scale parameters can be set through it:
🏁 Script executed:
#!/bin/bash
# Find SetAppiumSettings method implementation
rg -nP --type=go 'func.*SetAppiumSettings' -A20Length of output: 1329
🏁 Script executed:
#!/bin/bash
# Search for mjpegServerScreenshotQuality or similar parameter references
rg -nP --type=go 'mjpegServer|mjpegScaling|ScreenshotQuality|ScalingFactor' -B2 -A2Length of output: 340
🏁 Script executed:
#!/bin/bash
# Check what other Set* methods exist on WdaClient
rg -nP --type=go 'func \(c \*WdaClient\) Set' -A3Length of output: 933
🏁 Script executed:
#!/bin/bash
# See Android implementation more completely to understand how Quality/Scale are used
rg -nP --type=go 'func \(d \*AndroidDevice\) StartScreenCapture' -A50 | head -100Length of output: 2817
Implement Quality and Scale parameters for iOS, or validate that defaults are appropriate.
The review comment is valid. Quality and Scale are accepted in ScreenCaptureConfig but silently ignored on iOS (ios.go:656) and Simulator (simulator.go:668), unlike Android which explicitly uses them. WebDriverAgent's MJPEG server supports both image quality and scaling parameters — exposed as settings/capabilities such as mjpegServerScreenshotQuality and mjpegScalingFactor.
The pattern to implement this already exists in the codebase: SetMjpegFramerate uses SetAppiumSettings to configure WDA parameters. Similar methods can be added to set screenshot quality and scaling factor before StartScreenCapture is called.
Either:
- Implement
SetMjpegQualityandSetMjpegScalemethods using theSetAppiumSettingspattern (likeSetMjpegFramerate) - Document and validate that Quality/Scale are Android-only
- Return an error if non-default Quality/Scale are passed
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.