This is a camera app that continuously detects objects (bounding boxes, classes and confidence) in the frames seen by your device's camera, with Google's MediaPipe Object Detector. It can also detect objects in images and videos from your device or computer. One Flutter app runs on Android, iOS, macOS, Linux, Windows and the web.
It is the Flutter counterpart of Google's Android object detection sample, with the same screens and settings:
- Camera: live detection in
LIVE_STREAMmode. The detector runs one frame at a time and drops the frames that arrive meanwhile, as Google's live stream does. - Gallery: pick a photo (
IMAGEmode) or a video (VIDEOmode, played frame by frame at the file's own speed). - Settings: the score threshold, the maximum number of results and the delegate (CPU or GPU), plus the inference time of the last frame. Changing a setting recreates the detector, as in Google's sample.
lib/detector_helper.dart holds everything MediaPipe-specific, like
ObjectDetectorHelper.kt in Google's sample. The rest is camera, file and
drawing code.
- Flutter 3.44 or newer (the latest stable release is recommended), with the toolchain for the platform you build: Android SDK, Xcode, Visual Studio, or the Linux desktop packages. iOS and macOS builds with Xcode 27 need Flutter 3.47 or later.
- A clone of this repository. The sample builds against the packages in the
checkout, which can be ahead of pub.dev. Your own app adds
mediapipe_visionfrom pub.dev; see its installation guide. - Network access on the first build: the build hook downloads Google's MediaPipe vision library for the target and verifies its SHA-256.
| Platform | Requirement | Camera |
|---|---|---|
| Android | API 28 or newer; arm64 for the GPU | camera |
| iOS | iOS 15 or newer; a device or an arm64 Simulator | camera |
| macOS | Apple Silicon, macOS 14 or newer | camera_desktop |
| Linux | x64 | camera_desktop |
| Windows | x64 | camera_desktop |
| Web | A browser with WebAssembly; WebGL 2 for the GPU | getUserMedia |
From this directory:
flutter pub get
dart run mediapipe_core:bundle_models
flutter run -d <device>bundle_models downloads the model listed under
hooks.user_defines.mediapipe_vision.models in pubspec.yaml, verifies it and
writes it to assets/mediapipe/, so the app downloads nothing at run time.
Run it again after changing that list.
Use flutter devices to find the device name, or one of macos, linux,
windows or chrome. For the web, allow camera access when the browser asks;
browsers offer the camera only on HTTPS or localhost.
flutter testThe tests cover the drawing and pixel code, and run Google's detector on the
host in image, video and live stream mode against the repository's
gallery/samples/dog.jpg.
The sample bundles EfficientDet-Lite0 (float 32), the model Google's
sample uses by default, as VisionModels.objectDetector. It detects the 80
COCO classes. To try another model, replace model: in
ObjectDetectorHelper._create with modelPath: (a .tflite file) or
modelBytes:.
- GPU: the settings offer GPU only where
queryObjectDetectorCapabilities()reports it. If the GPU fails when the detector opens, the helper says so and switches to the CPU, as Google's sample does; the package itself never falls back. - macOS GPU: not available for now, so macOS runs on the CPU.
- iOS GPU: offered on iPhones and iPads, and tested on an iPhone 15 Pro (iOS 27) with Google's 1.2.2 library. iPads were not tested.
- Camera rotation: phone frames arrive in sensor layout. The sample passes their rotation to MediaPipe and turns the boxes upright with the preview.
- Web camera frames are read only for the frames the detector runs
(
VisionImage.deferred), so dropped frames cost nothing.
Apache 2.0, as the rest of this repository.