Skip to content

Commit ef57fec

Browse files
klopez4212Terp
andauthored
Fix gallery video selection, thumbnails, and swipe dismissal (#9356)
## Summary Photos currently filters out videos, video messages without poster metadata stay grey until playback, and the native gallery rejects downward dismissal on video pages. - Include photos and videos in the native inline/popover pickers, system picker, and recent-media grid. Mixed selections retain their order and videos use the video upload path. Remove the redundant Video menu row on iOS and Android, whose Photos picker accepts both media types; reduce their menu height to four rows. Native pickers request the current asset representation to avoid compatibility transcoding and export up to three files concurrently, preserving order and cleaning up cancelled/failed batches. - Show local stills for queued videos. For older messages without posters, serialize thumbnail generation for the four rendered stack cards, bound/cancel downloads, remove temporary videos, and save stills in the existing account/community cache. Existing poster metadata is used directly. The native viewer uses the poster while loading instead of a play-button snapshot. - Allow downward dismissal on native video pages while preserving horizontal paging/scrubbing, zoomed-photo panning, and the accessible Close button. Fire selection-click haptics as a swipe begins rather than after the page settles, with no duplicate on completion or cancellation. Page-indicator changes retain immediate feedback. Replace visible video-loading text with the existing white arc-spinner treatment, implemented natively with reduced-motion and lifecycle handling. Completed swipes release the previous video player even after UIKit removes it from the active children. Legacy videos need one initial download (up to the existing 100 MB limit) to generate their cached preview. This does not start playback. ### Related issue Follow-up to merged #9259 and #9314. Searched open and closed gallery/video issues and PRs; no duplicate fix found. ### Testing - Full `just mobile-check` and `just mobile-test` passed at `12e6ed6ae3`: 4,688 Flutter tests (71 skipped), 4 unconfigured-push tests, and 27 crypto-package tests. File-size checks passed. Regressions cover mixed image/video upload routing, queued still previews, the rendered-card poster window, authenticated/cancelled downloads, caching/cleanup, and native poster loading. - Native gallery simulator regression fixture passed: photo/video dismissal guards, real MP4 decoding, paging, selected-only playback loading, and teardown. - Swift type-check of the native picker and its dependencies passed, with existing deprecation/capture warnings. - `just ci` passed its repository-wide lint/type checks. The unrelated Rust test rebuild was stopped after consuming about 13 GB to preserve remaining disk space; only that run's generated caches were removed. Release build at `fb09f9cb8` installed in place and launched on the physical iPhone Air as **Buzz Photo Stack**, with `BUZZ_NATIVE_PHOTO_ZOOM=true`. Existing app updated in place with the same bundle and Keychain identity; no uninstall or reset. Owner tested the physical-device preview and approved readiness for review. No before/after capture was produced. The simulator evidence above is an automated native fixture, not a full Buzz device journey. Native preparation follow-up: `scripts/test-mobile-picker-preparation.sh` passed on an iPhone Air simulator, covering bounded overlap, out-of-order completion, cancellation, and failure cleanup. The updated physical Air preview launched successfully with unchanged preferences metadata. Cloud-only assets may still require preparation. Loader/haptics follow-up native fixture passed: real video readiness hides the loader, error/retry and detach stop/restart it correctly, swipe start triggers one haptic before selection changes, and completion/cancellation/duplicate-start callbacks add none. Updated release installed and launched on the physical Air; owner approved the final preview for review. Review: focused code review and native regression checks completed; physical Air preview tested by the owner, who approved readiness. P2 follow-up: video presentation uses an already decoded poster or the source snapshot immediately, then forwards a pending poster asynchronously. Late poster updates are limited to the selected native page and do not restart playback. Added pending-provider presentation coverage and native late/stale-poster checks. This follow-up has not been installed on the physical preview yet. --------- Signed-off-by: kenny lopez <[email protected]> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz>
1 parent e5dee91 commit ef57fec

30 files changed

Lines changed: 853 additions & 129 deletions

‎mobile/ios/NativeControlTests/NativePhotoZoomTests.swift‎

Lines changed: 76 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,14 @@ struct NativePhotoZoomTests {
2121
viewer.view.layoutIfNeeded()
2222
precondition(!requests.isEmpty && requests.allSatisfy { $0.1 <= 2 }, "Initial requests must stay adjacent")
2323
let pager = viewer.children.compactMap { $0 as? UIPageViewController }.first!
24+
let location = CGPoint(x: 195, y: 400)
25+
precondition(viewer.shouldBeginDismissal(location: location, velocity: CGVector(dx: 0, dy: 300)), "Resting photos dismiss downward")
26+
precondition(!viewer.shouldBeginDismissal(location: location, velocity: CGVector(dx: 300, dy: 10)), "Horizontal swipes must page")
27+
let scroll = descendants(pager.viewControllers!.first!.view).compactMap { $0 as? UIScrollView }.first!
28+
scroll.zoomScale = 2
29+
precondition(!viewer.shouldBeginDismissal(location: location, velocity: CGVector(dx: 0, dy: 300)), "Zoomed photos must pan instead of dismiss")
30+
scroll.zoomScale = 1
31+
2432
let indicator = descendants(viewer.view).compactMap { $0 as? UIPageControl }.first!
2533
let stale = requests
2634
requests.removeAll()
@@ -64,6 +72,7 @@ struct NativePhotoZoomTests {
6472
precondition(closedPhoto.image === closedImage, "Closed viewers must ignore continuing frames")
6573
print("PASS: native photo window, stale responses, eviction and teardown")
6674
videoJourney(data: data)
75+
loaderJourney()
6776

6877
}
6978
}
@@ -73,6 +82,8 @@ struct NativePhotoZoomTests {
7382
let viewer = NativePhotoZoomViewController(image: UIImage(data: data)!,
7483
labels: ["Clip", "Photo", "Second clip"], initialIndex: 0,
7584
canReply: true, canMore: true, canDelete: false, videoIndices: [0, 2])
85+
let feedback = GalleryFeedbackSpy()
86+
viewer.selectionFeedback = feedback
7687
var requested: [Int] = []
7788
var lateVideo: FlutterResult?
7889
viewer.request = { method, index, _, completion in
@@ -87,30 +98,88 @@ struct NativePhotoZoomTests {
8798
viewer.viewDidAppear(false)
8899
let pager = viewer.children.compactMap { $0 as? UIPageViewController }.first!
89100
let indicator = descendants(viewer.view).compactMap { $0 as? UIPageControl }.first!
101+
precondition(viewer.shouldBeginDismissal(location: CGPoint(x: 195, y: 400), velocity: CGVector(dx: 0, dy: 300)), "Videos must dismiss downward too")
102+
precondition(!viewer.shouldBeginDismissal(location: CGPoint(x: 195, y: 400), velocity: CGVector(dx: 300, dy: 0)), "Video horizontal scrubbing must not dismiss")
90103
let first = pager.viewControllers!.first!
91104
first.loadViewIfNeeded()
105+
let posterView = descendants(first.view).compactMap { $0 as? UIImageView }.first!
106+
let oldPoster = posterView.image
107+
viewer.updateFrame(data, at: 0)
108+
precondition(posterView.image !== oldPoster, "A late poster updates the selected video")
109+
let selectedPoster = posterView.image
110+
viewer.updateFrame(data, at: 2)
111+
precondition(posterView.image === selectedPoster, "A stale poster cannot replace the selected video")
92112
let playerController = first.children.compactMap { $0 as? AVPlayerViewController }.first!
93113
let player = playerController.player!
94114
let item = player.currentItem!
115+
let firstSpinner = descendants(first.view).compactMap { $0 as? NativeGalleryLoadingIndicator }.first!
95116
let deadline = Date().addingTimeInterval(5)
96-
while item.status == .unknown && Date() < deadline {
117+
while (item.status == .unknown || !firstSpinner.isHidden) && Date() < deadline {
97118
RunLoop.current.run(until: Date().addingTimeInterval(0.01))
98119
}
99120
precondition(item.status == .readyToPlay, "Real MP4 fixture must become playable")
121+
precondition(firstSpinner.isHidden, "Playable video removes the loader")
100122
precondition(requested == [0], "Only the selected video may download")
123+
precondition(feedback.clicks == 0, "Opening must not trigger selection haptics")
124+
viewer.pageViewController(pager, willTransitionTo: [first])
125+
precondition(feedback.clicks == 1, "Swipe feedback fires before the page changes")
126+
viewer.pageViewController(pager, willTransitionTo: [first])
127+
precondition(feedback.clicks == 1, "Repeated start callbacks do not duplicate swipe feedback")
128+
viewer.pageViewController(pager, didFinishAnimating: true, previousViewControllers: [first], transitionCompleted: false)
129+
precondition(feedback.clicks == 1, "Cancellation does not add a second haptic")
101130
indicator.currentPage = 2; sendValueChanged(indicator)
102131
precondition(player.currentItem == nil && playerController.player == nil, "Paging releases the old player before Dart deletes its file")
103132
precondition(requested == [0, 2], "Paging requests the selected video")
133+
precondition(feedback.clicks == 2, "Changing to a video triggers one selection click")
134+
let loadingPage = pager.viewControllers!.first!
135+
loadingPage.loadViewIfNeeded()
136+
let spinner = descendants(loadingPage.view).compactMap { $0 as? NativeGalleryLoadingIndicator }.first!
137+
precondition(!spinner.isHidden && spinner.accessibilityLabel == "Loading video", "Show the shared arc treatment while loading")
138+
precondition(!descendants(loadingPage.view).compactMap { $0 as? UILabel }.contains { $0.text == "Loading video…" }, "No visible loading copy")
139+
lateVideo?(nil)
140+
precondition(spinner.isHidden, "Failure stops the loader")
141+
let retry = descendants(loadingPage.view).compactMap { $0 as? UIButton }.first { $0.currentTitle == "Retry video" }!
142+
precondition(!retry.isHidden, "Failure retains retry")
143+
for target in retry.allTargets {
144+
for action in retry.actions(forTarget: target, forControlEvent: .touchUpInside) ?? [] {
145+
_ = (target as AnyObject).perform(NSSelectorFromString(action), with: retry)
146+
}
147+
}
148+
precondition(!spinner.isHidden && requested == [0, 2, 2], "Retry restarts the loader")
104149
indicator.currentPage = 0; sendValueChanged(indicator)
105150
lateVideo?(path)
106151
let current = pager.viewControllers!.first!
107152
current.loadViewIfNeeded()
108153
let active = current.children.compactMap { $0 as? AVPlayerViewController }.first!
109154
precondition(active.player != nil, "Returning to a video restores its player")
155+
precondition(spinner.isHidden, "Leaving a loading video stops its loader")
156+
let next = viewer.pageViewController(pager, viewControllerAfter: current)!
157+
viewer.pageViewController(pager, willTransitionTo: [next])
158+
precondition(feedback.clicks == 4, "Next swipe fires before the new photo is selected")
159+
pager.setViewControllers([next], direction: .forward, animated: false)
160+
viewer.pageViewController(pager, didFinishAnimating: true, previousViewControllers: [current], transitionCompleted: true)
161+
precondition(feedback.clicks == 4, "Completed swipe does not add a delayed haptic")
162+
viewer.pageViewController(pager, didFinishAnimating: true, previousViewControllers: [current], transitionCompleted: true)
163+
precondition(feedback.clicks == 4, "Repeated callbacks for the same page do not duplicate haptics")
110164
viewer.viewDidDisappear(false)
111165
precondition(active.player == nil, "Closing releases the player before returning to Flutter")
112166
print("PASS: native video decoding, selected-only loading, paging and teardown")
113167
}
168+
@MainActor static func loaderJourney() {
169+
let window = UIWindow(frame: CGRect(x: 0, y: 0, width: 100, height: 100))
170+
let root = UIViewController(); window.rootViewController = root
171+
let loader = NativeGalleryLoadingIndicator(frame: CGRect(x: 28, y: 28, width: 44, height: 44))
172+
root.view.addSubview(loader); window.isHidden = false
173+
loader.setLoading(true); loader.layoutIfNeeded()
174+
if !UIAccessibility.isReduceMotionEnabled {
175+
precondition(loader.layer.animation(forKey: "loading")?.duration == 0.5, "Match the shared spinner cadence")
176+
}
177+
loader.setLoading(false)
178+
precondition(loader.layer.animation(forKey: "loading") == nil && loader.isHidden)
179+
loader.setLoading(true); loader.removeFromSuperview()
180+
precondition(loader.layer.animation(forKey: "loading") == nil, "Detached loaders must stop animating")
181+
window.isHidden = true
182+
}
114183
@MainActor static func sendValueChanged(_ control: UIControl) {
115184
// A command-line UIKit fixture has no UIApplication action dispatcher.
116185
for target in control.allTargets {
@@ -123,3 +192,9 @@ struct NativePhotoZoomTests {
123192
[view] + view.subviews.flatMap { descendants($0) }
124193
}
125194
}
195+
196+
@MainActor
197+
private final class GalleryFeedbackSpy: UISelectionFeedbackGenerator {
198+
var clicks = 0
199+
override func selectionChanged() { clicks += 1 }
200+
}
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
import Foundation
2+
3+
@MainActor
4+
private final class ExportGate {
5+
var pending: [Int: CheckedContinuation<String, Error>] = [:]
6+
var started: [Int] = []
7+
let directory = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
8+
9+
init() throws { try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) }
10+
11+
func export(_ index: Int) async throws -> String {
12+
started.append(index)
13+
return try await withCheckedThrowingContinuation { pending[index] = $0 }
14+
}
15+
16+
func waitFor(_ count: Int) async {
17+
while started.count < count { await Task.yield() }
18+
}
19+
20+
func finish(_ index: Int) throws {
21+
let file = directory.appendingPathComponent("\(index).mov")
22+
try Data([UInt8(index)]).write(to: file)
23+
pending.removeValue(forKey: index)!.resume(returning: file.path)
24+
}
25+
}
26+
27+
@main
28+
struct PickerFilePreparationTests {
29+
@MainActor static func main() async throws {
30+
let gate = try ExportGate()
31+
defer { try? FileManager.default.removeItem(at: gate.directory) }
32+
let task = Task { try await PickerFilePreparation.prepare(count: 5, export: gate.export) }
33+
await gate.waitFor(3)
34+
precondition(gate.pending.count == 3, "Exports overlap but stay bounded")
35+
try gate.finish(2)
36+
await gate.waitFor(4)
37+
precondition(gate.pending.count == 3)
38+
try gate.finish(0)
39+
await gate.waitFor(5)
40+
for index in [4, 3, 1] { try gate.finish(index) }
41+
let paths = try await task.value
42+
precondition(paths.map { URL(fileURLWithPath: $0).lastPathComponent } == (0..<5).map { "\($0).mov" }, "Completion order must not reorder the selection")
43+
44+
for cancel in [false, true] {
45+
let gate = try ExportGate()
46+
defer { try? FileManager.default.removeItem(at: gate.directory) }
47+
let task = Task { try await PickerFilePreparation.prepare(count: cancel ? 5 : 3, export: gate.export) }
48+
await gate.waitFor(3)
49+
if cancel { task.cancel() }
50+
else { gate.pending.removeValue(forKey: 0)!.resume(throwing: CocoaError(.fileReadUnknown)) }
51+
if cancel { try gate.finish(0) }
52+
// Providers may return files even after cancellation or another export fails.
53+
try gate.finish(1)
54+
try gate.finish(2)
55+
do {
56+
_ = try await task.value
57+
preconditionFailure("Failed/cancelled batches must not become attachments")
58+
} catch {}
59+
precondition(gate.started.count == 3, "Stop scheduling after failure or cancellation")
60+
let remaining = try FileManager.default.contentsOfDirectory(atPath: gate.directory.path)
61+
precondition(remaining.isEmpty, "Remove every completed temporary file")
62+
}
63+
print("Picker export batching: order, concurrency, failure and cancellation passed")
64+
}
65+
}

‎mobile/ios/Runner/InlinePhotoPicker.swift‎

Lines changed: 51 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -90,10 +90,10 @@ final class InlinePhotoPickerPlatformView: NSObject, FlutterPlatformView {
9090
@available(iOS 17.0, *)
9191
private func installPicker() {
9292
var configuration = PHPickerConfiguration(photoLibrary: .shared())
93-
configuration.filter = .images
93+
configuration.filter = .any(of: [.images, .videos])
9494
configuration.selectionLimit = 0
9595
configuration.selection = .continuousAndOrdered
96-
configuration.preferredAssetRepresentationMode = .compatible
96+
configuration.preferredAssetRepresentationMode = .current
9797
configuration.disabledCapabilities = [
9898
.search,
9999
.stagingArea,
@@ -129,7 +129,7 @@ final class InlinePhotoPickerPlatformView: NSObject, FlutterPlatformView {
129129
guard
130130
let typeIdentifier = provider.registeredTypeIdentifiers.first(where: {
131131
guard let type = UTType($0) else { return false }
132-
return type.conforms(to: .image)
132+
return type.conforms(to: .image) || type.conforms(to: .movie)
133133
})
134134
else {
135135
throw InlinePhotoPickerError.unsupportedImage
@@ -200,9 +200,8 @@ extension InlinePhotoPickerPlatformView: PHPickerViewControllerDelegate {
200200
guard let self else { return }
201201
var paths: [String] = []
202202
do {
203-
for result in results {
204-
try Task.checkCancellation()
205-
paths.append(try await self.exportPickerResult(result))
203+
paths = try await PickerFilePreparation.prepare(count: results.count) {
204+
try await self.exportPickerResult(results[$0])
206205
}
207206
try Task.checkCancellation()
208207
await MainActor.run {
@@ -235,3 +234,49 @@ private enum InlinePhotoPickerError: Error {
235234
case missingFile
236235
case unsupportedImage
237236
}
237+
238+
// Keep Photos exports bounded without serializing an entire multi-selection.
239+
// Drain cancelled exports before cleaning up: providers can finish after cancel.
240+
enum PickerFilePreparation {
241+
@MainActor
242+
static func prepare(
243+
count: Int,
244+
export: @escaping @MainActor (Int) async throws -> String
245+
) async throws -> [String] {
246+
var paths = Array<String?>(repeating: nil, count: count)
247+
var failure: Error?
248+
await withTaskGroup(of: (Int, Result<String, Error>).self) { group in
249+
func enqueue(_ index: Int) {
250+
group.addTask {
251+
do {
252+
try Task.checkCancellation()
253+
return (index, .success(try await export(index)))
254+
} catch {
255+
return (index, .failure(error))
256+
}
257+
}
258+
}
259+
var next = min(3, count)
260+
for index in 0..<next { enqueue(index) }
261+
for await (index, result) in group {
262+
switch result {
263+
case .success(let path): paths[index] = path
264+
case .failure(let error):
265+
if failure == nil { failure = error }
266+
group.cancelAll()
267+
}
268+
if failure == nil && !Task.isCancelled && next < count {
269+
enqueue(next)
270+
next += 1
271+
}
272+
}
273+
}
274+
if Task.isCancelled || failure != nil {
275+
for path in paths.compactMap({ $0 }) {
276+
try? FileManager.default.removeItem(atPath: path)
277+
}
278+
throw failure ?? CancellationError()
279+
}
280+
return paths.compactMap { $0 }
281+
}
282+
}

‎mobile/ios/Runner/NativeAttachmentPopover.swift‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -180,10 +180,10 @@ final class NativeAttachmentPopoverViewController:
180180
stopCamera()
181181

182182
var configuration = PHPickerConfiguration(photoLibrary: .shared())
183-
configuration.filter = .images
183+
configuration.filter = .any(of: [.images, .videos])
184184
configuration.selectionLimit = 0
185185
configuration.selection = .continuousAndOrdered
186-
configuration.preferredAssetRepresentationMode = .compatible
186+
configuration.preferredAssetRepresentationMode = .current
187187
configuration.disabledCapabilities = [
188188
.search,
189189
.stagingArea,
@@ -505,9 +505,8 @@ final class NativeAttachmentPopoverViewController:
505505
guard let self else { return }
506506
var paths: [String] = []
507507
do {
508-
for result in results {
509-
try Task.checkCancellation()
510-
paths.append(try await self.exportPickerResult(result))
508+
paths = try await PickerFilePreparation.prepare(count: results.count) {
509+
try await self.exportPickerResult(results[$0])
511510
}
512511
try Task.checkCancellation()
513512
await MainActor.run {
@@ -541,7 +540,7 @@ final class NativeAttachmentPopoverViewController:
541540
} else if count == 0 {
542541
title = "All Photos"
543542
} else {
544-
title = "Add \(count) \(count == 1 ? "photo" : "photos")"
543+
title = "Add \(count) \(count == 1 ? "item" : "items")"
545544
}
546545
guard let button = photoActionButton else { return }
547546
let canInteract = !preparing
@@ -576,7 +575,7 @@ final class NativeAttachmentPopoverViewController:
576575
guard
577576
let typeIdentifier = provider.registeredTypeIdentifiers.first(where: {
578577
guard let type = UTType($0) else { return false }
579-
return type.conforms(to: .image)
578+
return type.conforms(to: .image) || type.conforms(to: .movie)
580579
})
581580
else {
582581
throw NativeAttachmentPopoverError.unsupportedImage

‎mobile/ios/Runner/NativeAttachmentPopoverCoordinator.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ final class NativeAttachmentPopoverCoordinator: NSObject {
265265
}
266266

267267
enum NativeAttachmentMenuLayout {
268-
static let itemCount: CGFloat = 5
268+
static let itemCount: CGFloat = 4
269269
static let contentPadding: CGFloat = 16
270270
static let minimumItemHeight: CGFloat = 52
271271
static let itemSpacing: CGFloat = 8

‎mobile/ios/Runner/NativeAttachmentPopoverMenu.swift‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -62,15 +62,6 @@ extension NativeAttachmentPopoverViewController {
6262
action: { [weak self] in self?.showPhotos() }
6363
)
6464
)
65-
stack.addArrangedSubview(
66-
makeNativeAttachmentMenuButton(
67-
title: "Video",
68-
symbol: "video",
69-
action: { [weak self] in
70-
self?.finish(method: "pickVideo")
71-
}
72-
)
73-
)
7465
stack.addArrangedSubview(
7566
makeNativeAttachmentMenuButton(
7667
title: "Voice note",

0 commit comments

Comments
 (0)