munim-pencilkit

Works with Expo  •  Read the Documentation  •  Report Issues

Follow Munim Technologies

   

Introduction

munim-pencilkit is a React Native library for Apple PencilKit integration with comprehensive Apple Pencil support. This library provides advanced stroke analysis, raw sensor data collection, haptic feedback, and professional drawing tools using well-established iOS APIs.

Fully compatible with Expo! Works seamlessly with both Expo managed and bare workflows.

Comprehensive Apple Pencil Support! Includes pressure sensitivity, tilt detection, predicted touches, coalesced touches, estimated properties, and haptic feedback using verified iOS APIs.

Table of contents

• 📚 Documentation
• 🚀 Features
• 📦 Installation
• ⚡ Quick Start
• 🔧 API Reference
• 📖 Usage Examples
• 🎨 Apple Pencil Pro Features
• 🔍 Troubleshooting
• 👏 Contributing
• 📄 License

📚 Documentation

Learn about building PencilKit apps in our documentation!

• Getting Started
• API Reference
• Usage Examples
• Apple Pencil Pro Features
• Troubleshooting

🚀 Features

Core PencilKit Integration

• 🎨 Full PencilKit Framework - Complete Apple PencilKit framework support
• ✍️ Dual Drawing Engines - Use Apple PencilKit (PKCanvasView) or a custom stylus engine (UITouch renderer)
• ✏️ Professional Drawing Tools - Pen, pencil, marker, eraser, and lasso tools
• 📱 Native Performance - Built with Turbo modules for optimal performance
• 🎯 TypeScript Support - Full TypeScript definitions included
• 🔄 Undo/Redo Support - Built-in drawing history management
• 🎛️ Configurable - Customizable drawing policies and tool settings
• 🚀 Expo Compatible - Works seamlessly with Expo managed and bare workflows

Drawing Engine Modes

• Apple PencilKit Engine: Set useCustomStylusView to false (native PKCanvasView rendering/input)
• Custom Stylus Engine: Set useCustomStylusView to true (custom UITouch stylus renderer)
• Coalesced Touches: onApplePencilCoalescedTouches is emitted in both engines when Apple Pencil capture is enabled

Apple Pencil Data Capture

• 📊 Raw Sensor Data - Pressure, tilt, azimuth, force, and location data
• 🎯 Precise Location - High-precision touch coordinates
• ⚡ Real-time Streaming - Live Apple Pencil data streaming
• 🔄 Coalesced Touches - High-fidelity input for smooth drawing
• 🔮 Predicted Touches - Latency compensation for responsive drawing
• 📈 Property Tracking - Estimated properties and refinement updates

Apple Pencil Advanced Features

• 📊 Pressure Sensitivity - Full pressure detection with Apple's recommended curves
• 🎯 Tilt Detection - Altitude and azimuth angle tracking
• 🧭 Azimuth Unit Vector - Efficient azimuth direction vector for transforms
• 🔄 Coalesced Touches - Optimized high-fidelity input for smooth drawing
• 🔮 Predicted Touches - Latency compensation for responsive drawing
• 📈 Estimated Properties - Real-time property refinement with touchesEstimatedPropertiesUpdated
• 📱 Haptic Feedback - Tactile responses for interactions
• 🎢 Motion Tracking - Core Motion gyroscope data for barrel roll detection
• 🧭 Device Orientation - Roll, pitch, and yaw angle tracking
• ⚡ Velocity & Acceleration - Real-time stroke velocity and acceleration tracking
• 📐 Stroke Analysis - Advanced stroke smoothness and consistency analysis

Advanced Capabilities

• 🧮 Perpendicular Force - Computed perpendicular force for accurate pressure
• 📊 Estimated Properties - Track and handle property refinements
• 🎨 Custom Brush Logic - Advanced brush behavior based on sensor data
• 🔧 Dynamic Updates - Update drawing configuration while active
• 📱 Cross-Platform - Works on all iOS devices with Apple Pencil support
• 📐 Stroke Analysis - Real-time stroke smoothness, consistency, and quality metrics
• ⚡ Performance Optimized - Apple's recommended coalesced touches handling
• 🎯 Pressure Curves - Natural pressure response using Apple's recommended algorithms

📦 Installation

React Native CLI

npm install munim-pencilkit react-native-nitro-modules
# or
yarn add munim-pencilkit react-native-nitro-modules

Expo

npx expo install munim-pencilkit react-native-nitro-modules

Note: This library requires Expo SDK 52+ (React Native 0.75+) and a development build (npx expo run:ios or EAS Build); it is not available in Expo Go.

iOS Setup

The pod's minimum deployment target is iOS 17.5. React Native and Expo projects default to a lower target, so raise it or pod install fails with "required a higher minimum deployment target".

React Native CLI (ios/Podfile):

platform :ios, '17.5'

Expo (app.json, with npx expo install expo-build-properties):

{
  "expo": {
    "plugins": [
      ["expo-build-properties", { "ios": { "deploymentTarget": "17.5" } }]
    ]
  }
}

PencilKit.framework is linked automatically and no Info.plist keys are needed.

⚡ Quick Start

Basic Usage

import React, { useRef } from 'react';
import { View, StyleSheet } from 'react-native';
import { PencilKitView, type PencilKitConfig } from 'munim-pencilkit';

export default function DrawingScreen() {
  const pencilKitRef = useRef<any>(null);

  const config: PencilKitConfig = {
    allowsFingerDrawing: true,
    allowsPencilOnlyDrawing: false,
    isRulerActive: false,
    drawingPolicy: 'default',
  };

  return (
    <View style={styles.container}>
      <PencilKitView
        ref={pencilKitRef}
        style={styles.canvas}
        config={config}
        enableApplePencilData={true}
        enableToolPicker={true}
        onApplePencilData={(data) => {
          console.log('Apple Pencil Data:', data);
        }}
        onDrawingChange={(drawing) => {
          console.log('Drawing changed:', drawing);
        }}
      />
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
  canvas: {
    flex: 1,
  },
});

Apple Pencil Advanced Usage

import React, { useRef } from 'react';
import { View, StyleSheet } from 'react-native';
import {
  PencilKitView,
  type PencilKitConfig,
  type ApplePencilData,
  type ApplePencilMotionData,
} from 'munim-pencilkit';

export default function AdvancedDrawingScreen() {
  const pencilKitRef = useRef<any>(null);

  const config: PencilKitConfig = {
    allowsFingerDrawing: true,
    allowsPencilOnlyDrawing: false,
    isRulerActive: false,
    drawingPolicy: 'default',
    enableHapticFeedback: true,
  };

  return (
    <View style={styles.container}>
      <PencilKitView
        ref={pencilKitRef}
        style={styles.canvas}
        config={config}
        enableApplePencilData={true}
        enableToolPicker={true}
        enableHapticFeedback={true}
        enableMotionTracking={true}
        onApplePencilData={(data: ApplePencilData) => {
          console.log('Advanced Apple Pencil Data:', {
            pressure: data.pressure,
            perpendicularForce: data.perpendicularForce,
            azimuth: data.azimuth,
            azimuthUnitVector: data.azimuthUnitVector,
            rollAngle: data.rollAngle,
            preciseLocation: data.preciseLocation,
            estimatedProperties: data.estimatedProperties,
          });
        }}
        onApplePencilMotion={(data: ApplePencilMotionData) => {
          console.log('Motion data:', {
            rollAngle: data.rollAngle,
            pitchAngle: data.pitchAngle,
            yawAngle: data.yawAngle,
          });
          // Handle motion - adjust brush orientation, barrel roll effects
        }}
        onApplePencilCoalescedTouches={(data) => {
          console.log('Coalesced touches:', data.touches.length);
          // Handle high-fidelity input for smooth drawing
        }}
        onApplePencilPredictedTouches={(data) => {
          console.log('Predicted touches:', data.touches.length);
          // Handle predicted touches for latency compensation
        }}
        onDrawingChange={(drawing) => {
          console.log('Drawing changed:', drawing);
        }}
      />
    </View>
  );
}

🔧 API Reference

PencilKitView Component

Main React Native component for PencilKit integration with full Apple Pencil Pro support.

Props:

Basic Props

• config (PencilKitConfig): PencilKit configuration
• style (ViewStyle): Component styling
• onViewReady (function): View ready callback

Apple Pencil Data Props

• enableApplePencilData (boolean): Enable Apple Pencil data capture
• onApplePencilData (function): Apple Pencil data callback
• onApplePencilCoalescedTouches (function): Coalesced touches callback
• onApplePencilPredictedTouches (function): Predicted touches callback
• onApplePencilEstimatedProperties (function): Estimated properties callback

Apple Pencil Pro Props

• enableSqueezeInteraction (boolean): Enable squeeze gesture detection
• enableDoubleTapInteraction (boolean): Enable double-tap detection
• enableHoverSupport (boolean): Enable hover pose detection
• enableHapticFeedback (boolean): Enable haptic feedback
• onApplePencilSqueeze (function): Squeeze gesture callback
• onApplePencilDoubleTap (function): Double-tap callback
• onApplePencilHover (function): Hover pose callback
• onApplePencilPreferredSqueezeAction (function): Preferred squeeze action callback

Drawing Props

• enableToolPicker (boolean): Show tool picker
• onDrawingChange (function): Lightweight drawing change callback ({ viewId, revision, canUndo, canRedo, dirty, bounds }); fired on every change without serializing the drawing
• onDrawingSnapshot (function): Full serialized drawing, emitted after the drawing settles; enable it with config.snapshotDebounceMs > 0
• onDrawingBegin / onDrawingEnd (function): Stroke phase callbacks ({ viewId, revision, canUndo, canRedo, phase, timestamp, timestampClock })
• onHistoryChange (function): Undo/redo availability callback ({ viewId, revision, canUndo, canRedo })
• onToolPickerChange (function): Tool picker visibility/selection callback ({ viewId, visible, selectedTool })
• onToolPickerItemChange (function, iOS 18+): The selected tool picker item changed, including custom items ({ viewId, identifier, itemType, reselected, selectedTool? | color?, width? })
• onToolPickerAccessoryPress (function, iOS 18+): config.toolPickerAccessoryItem was tapped ({ viewId, identifier })
• onDidFinishRendering (function): PencilKit finished rendering all visible content (canvasViewDidFinishRendering). Only wired up when you pass it.
• onSelectionChange (function, iOS 27+): the lasso selection changed ({ viewId, revision, strokeIds }). Only wired up when you pass it.
• onApplePencilHover events now carry phase ('began' | 'changed' | 'ended' | 'cancelled'); ended/cancelled fire once when the pencil leaves hover range.

The tool picker needs the canvas to be first responder, so showing it takes focus. That now happens only when the picker goes from hidden to shown (or on setToolPickerVisible(true)), and never while a text input is focused. Starting a stroke brings the picker back after focus has moved away. Set config.autoFocusToolPicker: false to leave focus alone until the user draws or you call setToolPickerVisible(true).

PencilKitUtils

Utility functions for advanced PencilKit operations.

Methods:

Feature Detection

• isSupported(): true when PencilKit is available (iOS)
• getCapabilities(): Detailed PencilKitCapabilities object with OS version, supported document formats, ink/eraser tools, telemetry support, native import/export size limits, and features (which iOS 26/27 APIs work here)
• getRecognitionInfo(): handwriting recognition support, languages and model version (supported: false before iOS 27)

View Management

• createView(): Create a new PencilKit view
• destroyView(viewId): Release the native state behind a view id (motion tracking, tool picker, pending snapshots). The UIView belongs to React Native and is left in place
• setConfig(viewId, config): Configure PencilKit view

Documents and Tools

• exportDocument(viewId, options): Export the drawing. Options (PencilKitExportOptions): format ('archive' | 'png' | 'jpeg' | 'pdf'), output ('base64' | 'fileUrl'), crop ('drawingBounds' | 'canvas' | 'custom' with cropRect), scale (0.1 to 8), backgroundColor (#RRGGBB/#RRGGBBAA), and quality (JPEG, 0 to 1). Returns a PencilKitExportResult ({ format, output, mimeType, byteLength, width, height, dataBase64? | fileUrl? }). fileUrl exports are written to the app's temporary directory.
• importDocument(viewId, options): Import a document (PencilKitImportOptions). archive restores a PKDrawing into the PencilKit engine. png/jpeg are only accepted when config.useCustomStylusView is enabled, where the image becomes the raster canvas base — PKCanvasView cannot convert images into strokes. Invalid or mismatched imports throw and leave the current drawing untouched.
• setTool(viewId, tool): Set the active tool (PencilKitToolState): inks (pen, pencil, marker, monoline, fountainPen, watercolor, crayon, and reed on iOS 26+), eraser (bitmap/vector with optional width), or lasso. getCapabilities().tools.ink lists the inks the running OS supports.
• getTool(viewId): Read the active tool back as PencilKitToolState
• setToolPickerVisible(viewId, visible): Programmatically show/hide the tool picker, overriding the enableToolPicker prop until the prop next changes

All of the above are also available on PencilKitViewRef (exportDocument, importDocument, setTool, getTool, setToolPickerVisible).

Export also accepts appearance ('light' default, 'dark', or 'view' for the canvas's current style). PencilKit adapts black/white ink to the interface style, and exports render off the main thread, so the style is pinned explicitly. PDF exports are raster: PencilKit has no vector export, so the page holds the drawing rendered at scale, and width/height report the page size in points.

On iOS 18+ setTool also moves the tool picker's selection to the matching item (or to itemIdentifier when given), so the picker no longer resets the tool the next time it is used.

Async API

The PencilKitView ref runs its calls in call order, and getDrawing, setDrawing, exportDocument, importDocument, and the stroke calls run natively off the JS thread. PencilKitUtils has the same as getDrawingAsync, setDrawingAsync, exportDocumentAsync, and importDocumentAsync. The synchronous getDrawing, setDrawing, exportDocument, and importDocument utils still work but are deprecated: they block the JS thread until the main thread is free.

Strokes

PencilKit engine only (not useCustomStylusView). Available on the ref and as PencilKitUtils.*(viewId, ...):

• getStrokes(): every stroke as PencilKitStroke
• setStrokes(strokes): replace all strokes
• appendStrokes(strokes): add strokes on top
• removeStrokes(indices): resolves the number removed
• transformStrokes(indices, { a, b, c, d, tx, ty }): applies the transform in canvas space; resolves the number changed

Every mutation is one undo step. A PencilKitStroke keeps the legacy fields (points, tool, color, width) and adds ink (inkType, color), transform, mask (SVG path data), randomSeed, creationDate, renderBounds, and requiredContentVersion. Points are the path's control points in stroke space (apply transform for canvas coordinates) with timeOffset, size, opacity, force, azimuth, altitude, secondaryScale, and threshold (iOS 26+). Output round-trips through setStrokes. For input, only points[].location is required. See the PencilKitStroke JSDoc for the defaults.

getDrawing() leaves strokes empty unless you pass getDrawing({ includeStrokes: true }) or set config.includeStrokesInDrawing (which also covers onDrawingSnapshot), because stroke JSON is much larger than the archive. setDrawing still prefers dataBase64; without it, a non-empty strokes array is drawn instead.

iOS 27: stroke ids, selection, erasing, handwriting

These need iPadOS/iOS 27 at runtime and an app built with the iOS 27 SDK (Xcode 27). Elsewhere the calls reject with an error that says so (stroke ids are simply absent), so check first:

const { features } = PencilKitUtils.getCapabilities()
if (features?.handwritingRecognition) {
  // iOS 27 APIs are available
}

• Stable stroke ids: on iOS 27 every PencilKitStroke from getStrokes() or getDrawing({ includeStrokes: true }) has an id (PKStroke.id, a UUID string that survives edits and undo), plus pathId, renderGroupId, renderState and per-point lateralJitter. setStrokes, appendStrokes and setDrawing keep the ids you pass; an id that is already on the canvas (or repeated in the same call) gets a fresh one so ids stay unique.
• Selection: getSelection() resolves the ids of the lasso-selected strokes, setSelection(ids) selects strokes (unknown ids are ignored), and onSelectionChange reports changes.
• Erase along a path: erasePath({ points, width?, mask?, transform? }) erases like the vector-precise eraser dragged along points (canvas space, { x, y } or stroke points; width defaults to 20). Strokes it crosses are trimmed or split. Resolves { changed, strokeCountBefore, strokeCountAfter } and is one undo step. It runs off the main thread.
• Bézier paths: getStrokes({ includeBezierPaths: true }) adds each stroke's bezierPath (PKStrokePath.bezierRepresentation) as SVG path data in stroke space.
• Handwriting recognition (PKStrokeRecognizer):

const info = await PencilKitUtils.getRecognitionInfo()
// { supported, supportedLanguages: ['en', ...], recognitionVersion }

const { text } = await canvasRef.current.recognizeText()
// Only some strokes, and preferred languages:
await canvasRef.current.recognizeText({
  strokeIds: selectedIds,
  preferredLanguages: ['en', 'fr'],
  includeIndexableContent: true, // all candidate text, for search indexes
})

const matches = await canvasRef.current.searchText('invoice', {
  fullWordsOnly: true,
})
// [{ strokeIds: [...], bounds: { x, y, width, height } }]

Each view keeps one recognizer and updates it incrementally with the drawing. text is null when nothing was recognized. PencilKitUtils has the same calls taking a viewId first.

config.toolPickerColorMaximumLinearExposure (iOS 26+) sets PKToolPicker.colorMaximumLinearExposure: values above 1 let people pick HDR ink colors on HDR displays (null restores the default).

Content version

config.maximumSupportedContentVersion (1-5 or 'latest', iOS 17+) caps the PKContentVersion that new strokes may use, on both the canvas and the tool picker. Use it when drawings must open on older OS versions. Drawing payloads and archive exports report requiredContentVersion, and getCapabilities().contentVersion.maximum is the newest version the OS renders (5 on iOS 27, which adds per-stroke render state).

Custom tool picker (iOS 18+)

config={{
  // ...
  toolItems: [
    { type: 'ink', inkType: 'pen', color: '#111827', width: 3, identifier: 'pen' },
    { type: 'ink', inkType: 'marker', color: '#FACC15', width: 20, identifier: 'highlighter' },
    { type: 'eraser', eraserType: 'vector' },
    { type: 'lasso' },
    { type: 'ruler' },
    { type: 'custom', identifier: 'stamp', name: 'Stamp', systemImage: 'star.fill' },
  ],
  toolPickerAccessoryItem: { systemImage: 'ellipsis.circle', identifier: 'more' },
}}

Tool items are fixed when a picker is created, so changing them rebuilds the picker. null restores the system picker. iOS 17 ignores both options. PencilKit does not draw with custom items: handle onToolPickerItemChange and the pencil input yourself.

Drawing Operations

• getDrawingAsync(viewId, options?) / getDrawing(viewId) (deprecated): Get current drawing data
• setDrawingAsync(viewId, drawing) / setDrawing(viewId, drawing) (deprecated): Set drawing data; rejects/throws if the import fails validation
• clearDrawing(viewId): Clear the drawing
• undo(viewId): Undo last action
• redo(viewId): Redo last action
• canUndo(viewId): Check if undo is available
• canRedo(viewId): Check if redo is available

Drawing Import Limits

Native iOS validation applies the following fixed limits to imported configuration and drawing data:

• JSON UTF-8 size: 16,777,216 bytes (16 MiB)
• JSON nesting: 32 collection levels
• JSON complexity: 10,000 total object entries and array elements (4,000,000 for drawing and stroke payloads)
• Strokes: 50,000 per call, 100,000 points per stroke, 500,000 points total
• Base64 text size: 12,582,912 UTF-8 bytes (12 MiB)
• Base64-decoded data: 8,388,608 bytes (8 MiB)
• PencilKit drawing archive: 8,388,608 bytes before PKDrawing construction
• Custom-canvas image: one frame, at most 8,192 pixels on either axis and 16,777,216 total pixels

Image dimensions are read from metadata and checked before the image is handed to UIKit for decompression or rendering. Malformed or oversized imports throw without changing the current drawing. PencilKitUtils.setDrawing surfaces the native exception directly; PencilKitViewRef.setDrawing returns a rejected promise.

Apple Pencil Data Capture

• startApplePencilCapture(viewId): Start capturing Apple Pencil data
• stopApplePencilCapture(viewId): Stop capturing Apple Pencil data
• isApplePencilCaptureActive(viewId): Check if capture is active

Event Listeners

Every add*Listener(callback, viewId?) returns an unsubscribe function and optionally filters by viewId. Every remove*Listener(callback?) removes only the given callback; calling it without arguments keeps the old behavior of clearing all listeners for that event.

• addApplePencilListener(callback): Add Apple Pencil data listener
• removeApplePencilListener(callback?): Remove Apple Pencil data listener
• addDrawingChangeListener(callback): Add drawing change listener
• removeDrawingChangeListener(callback?): Remove drawing change listener
• addHistoryChangeListener(callback): Add undo/redo history listener
• removeHistoryChangeListener(callback?): Remove history listener
• addToolPickerChangeListener(callback): Add tool picker listener
• removeToolPickerChangeListener(callback?): Remove tool picker listener
• addApplePencilSqueezeListener(callback): Add squeeze listener
• removeApplePencilSqueezeListener(): Remove squeeze listener
• addApplePencilDoubleTapListener(callback): Add double-tap listener
• removeApplePencilDoubleTapListener(): Remove double-tap listener
• addApplePencilHoverListener(callback): Add hover listener
• removeApplePencilHoverListener(): Remove hover listener
• addApplePencilCoalescedTouchesListener(callback): Add coalesced touches listener
• removeApplePencilCoalescedTouchesListener(): Remove coalesced touches listener
• addApplePencilPredictedTouchesListener(callback): Add predicted touches listener
• removeApplePencilPredictedTouchesListener(): Remove predicted touches listener
• addApplePencilEstimatedPropertiesListener(callback): Add estimated properties listener
• removeApplePencilEstimatedPropertiesListener(): Remove estimated properties listener
• addApplePencilPreferredSqueezeActionListener(callback): Add preferred squeeze action listener
• removeApplePencilPreferredSqueezeActionListener(): Remove preferred squeeze action listener

Types

ApplePencilData

Enhanced interface for Apple Pencil data with all advanced properties:

interface ApplePencilData {
  // Basic properties
  pressure: number; // 0.0 to 1.0
  altitude: number; // 0.0 to 1.0
  azimuth: number; // 0.0 to 2π radians
  azimuthUnitVector: {
    x: number;
    y: number;
  }; // Unit vector pointing in azimuth direction
  force: number; // 0.0 to 1.0
  maximumPossibleForce: number;
  timestamp: number;

  // Location data
  location: {
    x: number;
    y: number;
  };
  previousLocation: {
    x: number;
    y: number;
  };
  preciseLocation: {
    x: number;
    y: number;
  };

  // Apple Pencil Pro properties
  perpendicularForce: number; // Computed perpendicular force
  rollAngle: number; // Barrel roll angle (Apple Pencil Pro)

  // Touch properties
  isApplePencil: boolean;
  phase: 'began' | 'moved' | 'ended' | 'cancelled';
  hasPreciseLocation: boolean;

  // Advanced properties
  estimatedProperties: string[];
  estimatedPropertiesExpectingUpdates: string[];
}

Apple Pencil Pro Event Types

interface ApplePencilSqueezeData {
  viewId: number;
  phase: 'began' | 'changed' | 'ended' | 'cancelled';
  timestamp: number;
  preferredAction:
    | 'ignore'
    | 'switchEraser'
    | 'showColorPalette'
    | 'showInkAttributes'
    | 'showContextualPalette'
    | 'switchPrevious'
    | 'runSystemShortcut';
  hoverPose?: {
    location: { x: number; y: number };
    zOffset: number;
    azimuth: number;
    azimuthUnitVector: { x: number; y: number };
    altitude: number;
    rollAngle: number;
  };
}

interface ApplePencilDoubleTapData {
  viewId: number;
  phase: 'ended';
  timestamp: number;
  preferredAction:
    | 'ignore'
    | 'switchEraser'
    | 'showColorPalette'
    | 'showInkAttributes'
    | 'showContextualPalette'
    | 'switchPrevious'
    | 'runSystemShortcut';
  hoverPose?: {
    location: { x: number; y: number };
    zOffset: number;
    azimuth: number;
    azimuthUnitVector: { x: number; y: number };
    altitude: number;
    rollAngle: number;
  };
}

interface ApplePencilHoverData {
  viewId: number;
  location: {
    x: number;
    y: number;
  };
  altitude: number; // radians
  azimuth: number; // radians
  azimuthUnitVector: {
    x: number;
    y: number;
  };
  zOffset?: number; // 1.0 far from screen, 0.0 near screen
  rollAngle?: number; // Apple Pencil Pro (iOS 17.5+)
  timestamp: number;
}

interface ApplePencilCoalescedTouchesData {
  viewId: number;
  touches: ApplePencilData[];
  timestamp: number;
}

interface ApplePencilPredictedTouchesData {
  viewId: number;
  touches: ApplePencilData[];
  timestamp: number;
}

interface ApplePencilEstimatedPropertiesData {
  viewId: number;
  touchId: number;
  updatedProperties: string[];
  newData: ApplePencilData;
  timestamp: number;
}

interface ApplePencilPreferredSqueezeActionData {
  preferredAction:
    | 'ignore'
    | 'switchEraser'
    | 'showColorPalette'
    | 'showInkAttributes'
    | 'showContextualPalette'
    | 'switchPrevious'
    | 'runSystemShortcut';
}

PencilKitConfig

Configuration interface for PencilKit:

interface PencilKitConfig {
  // Basic configuration
  allowsFingerDrawing: boolean;
  allowsPencilOnlyDrawing: boolean;
  isRulerActive: boolean;
  drawingPolicy: 'default' | 'anyInput' | 'pencilOnly';

  // Input / capture configuration
  enableApplePencilData?: boolean;
  enableToolPicker?: boolean;
  enableHapticFeedback?: boolean;
  enableMotionTracking?: boolean;
  enableSqueezeInteraction?: boolean;
  enableDoubleTapInteraction?: boolean;
  enableHoverSupport?: boolean;
  useCustomStylusView?: boolean; // false = Apple PencilKit engine, true = custom stylus engine
  squeezeEraserBehavior?: 'alwaysOn' | 'switchEraserOnly' | 'toggle' | 'none'; // custom stylus squeeze behavior, default: 'alwaysOn'
  customStylusRenderMode?: 'incremental' | 'replay'; // default: 'incremental'
  customStylusEraserMode?: 'clear' | 'paint'; // default: 'clear'
  customStylusOpaqueCanvas?: boolean; // default: false
  customStylusSurfaceColor?: string; // default: system background
  showHoverPreview?: boolean; // Hover ring used by custom stylus engine
  strokeColor?: string;
  baseLineWidth?: number;
  snapshotDebounceMs?: number; // > 0 emits onDrawingSnapshot after drawing settles
  scrollEnabled?: boolean; // default: false — let a parent ScrollView own finger pans; true restores PKCanvasView's own panning
  includeStrokesInDrawing?: boolean; // default: false — fill `strokes` in getDrawing()/onDrawingSnapshot
  maximumSupportedContentVersion?: 1 | 2 | 3 | 4 | 5 | 'latest' | null; // iOS 17+
  toolItems?: PencilKitToolItem[] | null; // iOS 18+ custom tool picker
  toolPickerAccessoryItem?: { systemImage?: string; title?: string; identifier?: string } | null; // iOS 18+
  autoFocusToolPicker?: boolean; // default: true — see "Drawing Props"
  customStylusHistoryLimit?: number; // custom stylus undo depth, default 20
  customStylusHistoryMemoryMB?: number; // custom stylus undo+redo bitmap budget, default 96
}

Drawing Data Types

interface PencilKitDrawingData {
  strokes: PencilKitStroke[];
  bounds: {
    x: number;
    y: number;
    width: number;
    height: number;
  };
  dataBase64?: string; // PencilKit engine: PKDrawing archive
  imageBase64?: string; // custom stylus engine: PNG
  requiredContentVersion?: number;
}

interface PencilKitStroke {
  points: PencilKitPoint[];
  tool: PencilKitTool;
  color: string;
  width: number;
  ink?: { inkType: PencilKitInkType; color: string };
  transform?: { a: number; b: number; c: number; d: number; tx: number; ty: number };
  mask?: string; // SVG path data
  randomSeed?: number;
  creationDate?: number; // ms since epoch
  renderBounds?: PencilKitRect; // output only
  requiredContentVersion?: number; // output only
}

interface PencilKitPoint {
  location: {
    x: number;
    y: number;
  };
  pressure: number; // = force in native output
  azimuth: number;
  altitude: number;
  timestamp: number; // = timeOffset in native output
  timeOffset?: number;
  size?: { width: number; height: number };
  opacity?: number;
  force?: number;
  secondaryScale?: number;
  threshold?: number; // iOS 26+
}

interface PencilKitTool {
  type: 'pen' | 'pencil' | 'marker' | 'eraser' | 'lasso';
  width: number;
  color: string;
}

📖 Usage Examples

Professional Drawing App with Apple Pencil Pro

import React, { useRef, useState } from 'react';
import { View, StyleSheet, Button, Text } from 'react-native';
import {
  PencilKitView,
  PencilKitUtils,
  type ApplePencilData,
  type ApplePencilSqueezeData,
} from 'munim-pencilkit';

const ProfessionalDrawingApp = () => {
  const pencilKitRef = useRef<any>(null);
  const [isDrawing, setIsDrawing] = useState(false);
  const [brushSize, setBrushSize] = useState(5);

  const config = {
    allowsFingerDrawing: false,
    allowsPencilOnlyDrawing: true,
    isRulerActive: true,
    drawingPolicy: 'pencilOnly',
    enableSqueezeInteraction: true,
    enableDoubleTapInteraction: true,
    enableHoverSupport: true,
    enableHapticFeedback: true,
  };

  const handleApplePencilData = (data: ApplePencilData) => {
    // Use advanced properties for sophisticated brush behavior
    const newBrushSize = data.perpendicularForce * 20;
    setBrushSize(newBrushSize);

    // Use roll angle for brush orientation
    const brushAngle = data.rollAngle;

    // Use precise location if available
    const location = data.hasPreciseLocation
      ? data.preciseLocation
      : data.location;

    console.log('Advanced brush data:', {
      size: newBrushSize,
      angle: brushAngle,
      location,
      pressure: data.pressure,
    });
  };

  const handleSqueeze = (data: ApplePencilSqueezeData) => {
    if (data.phase === 'ended') {
      // Show contextual palette on squeeze
      console.log('Showing contextual palette');
    }
  };

  const handleSave = async () => {
    const drawing = await pencilKitRef.current.getDrawing();
    console.log('Saving drawing:', drawing);
  };

  return (
    <View style={styles.container}>
      <PencilKitView
        ref={pencilKitRef}
        style={styles.canvas}
        config={config}
        enableApplePencilData={true}
        enableToolPicker={true}
        enableSqueezeInteraction={true}
        enableDoubleTapInteraction={true}
        enableHoverSupport={true}
        enableHapticFeedback={true}
        onApplePencilData={handleApplePencilData}
        onApplePencilSqueeze={handleSqueeze}
        onApplePencilDoubleTap={(data) => {
          // Toggle between pen and eraser on double tap
          console.log('Double tap - switching tool');
        }}
        onApplePencilHover={(data) => {
          // Show brush preview on hover
          console.log('Hover preview at:', data.location);
        }}
        onDrawingChange={(drawing) => {
          console.log('Drawing changed:', drawing);
        }}
      />
      <View style={styles.controls}>
        <Text>Brush Size: {brushSize.toFixed(1)}</Text>
        <Button title="Save" onPress={handleSave} />
      </View>
    </View>
  );
};

Real-time Data Collection with Advanced Features

import React, { useEffect, useState } from 'react';
import { View, Text, StyleSheet, ScrollView } from 'react-native';
import {
  PencilKitUtils,
  type ApplePencilData,
  type ApplePencilPredictedTouchesData,
  type ApplePencilEstimatedPropertiesData,
} from 'munim-pencilkit';

const AdvancedDataCollectionApp = () => {
  const [pencilData, setPencilData] = useState<ApplePencilData | null>(null);
  const [predictedTouches, setPredictedTouches] =
    useState<ApplePencilPredictedTouchesData | null>(null);
  const [estimatedProperties, setEstimatedProperties] =
    useState<ApplePencilEstimatedPropertiesData | null>(null);

  useEffect(() => {
    // Apple Pencil data listener
    const handlePencilData = (data: ApplePencilData) => {
      setPencilData(data);
    };

    // Predicted touches listener
    const handlePredictedTouches = (data: ApplePencilPredictedTouchesData) => {
      setPredictedTouches(data);
    };

    // Estimated properties listener
    const handleEstimatedProperties = (
      data: ApplePencilEstimatedPropertiesData
    ) => {
      setEstimatedProperties(data);
    };

    PencilKitUtils.addApplePencilListener(handlePencilData);
    PencilKitUtils.addApplePencilPredictedTouchesListener(
      handlePredictedTouches
    );
    PencilKitUtils.addApplePencilEstimatedPropertiesListener(
      handleEstimatedProperties
    );

    return () => {
      PencilKitUtils.removeApplePencilListener();
      PencilKitUtils.removeApplePencilPredictedTouchesListener();
      PencilKitUtils.removeApplePencilEstimatedPropertiesListener();
    };
  }, []);

  return (
    <ScrollView style={styles.container}>
      <Text style={styles.title}>Advanced Apple Pencil Data</Text>

      {pencilData && (
        <View style={styles.dataContainer}>
          <Text style={styles.sectionTitle}>Basic Data</Text>
          <Text>Pressure: {pencilData.pressure.toFixed(3)}</Text>
          <Text>Altitude: {pencilData.altitude.toFixed(3)}</Text>
          <Text>Azimuth: {pencilData.azimuth.toFixed(3)}</Text>
          <Text>Force: {pencilData.force.toFixed(3)}</Text>

          <Text style={styles.sectionTitle}>Apple Pencil Pro Data</Text>
          <Text>
            Perpendicular Force: {pencilData.perpendicularForce.toFixed(3)}
          </Text>
          <Text>Roll Angle: {pencilData.rollAngle.toFixed(3)}</Text>
          <Text>
            Has Precise Location: {pencilData.hasPreciseLocation ? 'Yes' : 'No'}
          </Text>

          <Text style={styles.sectionTitle}>Location Data</Text>
          <Text>
            Location: ({pencilData.location.x.toFixed(1)},{' '}
            {pencilData.location.y.toFixed(1)})
          </Text>
          <Text>
            Precise: ({pencilData.preciseLocation.x.toFixed(1)},{' '}
            {pencilData.preciseLocation.y.toFixed(1)})
          </Text>

          <Text style={styles.sectionTitle}>Advanced Properties</Text>
          <Text>Estimated: {pencilData.estimatedProperties.join(', ')}</Text>
          <Text>
            Expecting Updates:{' '}
            {pencilData.estimatedPropertiesExpectingUpdates.join(', ')}
          </Text>
        </View>
      )}

      {predictedTouches && (
        <View style={styles.dataContainer}>
          <Text style={styles.sectionTitle}>Predicted Touches</Text>
          <Text>Count: {predictedTouches.touches.length}</Text>
          <Text>Timestamp: {predictedTouches.timestamp}</Text>
        </View>
      )}

      {estimatedProperties && (
        <View style={styles.dataContainer}>
          <Text style={styles.sectionTitle}>Estimated Properties Update</Text>
          <Text>Touch ID: {estimatedProperties.touchId}</Text>
          <Text>
            Updated: {estimatedProperties.updatedProperties.join(', ')}
          </Text>
          <Text>Timestamp: {estimatedProperties.timestamp}</Text>
        </View>
      )}
    </ScrollView>
  );
};

🎨 Apple Pencil Pro Features

Squeeze Gestures

Apple Pencil Pro squeeze gestures allow users to perform actions by squeezing the pencil:

<PencilKitView
  enableSqueezeInteraction={true}
  onApplePencilSqueeze={(data) => {
    if (data.phase === 'ended') {
      // Show contextual palette
      showContextualPalette(data.preferredAction);
    }
  }}
/>

Double Tap Gestures

Double-tap gestures for quick tool switching:

<PencilKitView
  enableDoubleTapInteraction={true}
  onApplePencilDoubleTap={(data) => {
    if (data.phase === 'ended') {
      // Toggle between pen and eraser
      toggleTool();
    }
  }}
/>

Hover Effects

Hover pose detection for previews and visual feedback:

<PencilKitView
  onApplePencilHover={(data) => {
    // Show brush preview at hover location/angle
    showBrushPreview(data.location, data.altitude, data.azimuth);
  }}
/>

Hover pose telemetry (altitude, azimuth, azimuthUnitVector) requires iPadOS 16.4+ on supported iPad hardware. zOffset requires iPadOS 16.1+, and rollAngle requires iOS/iPadOS 17.5+ with supported Apple Pencil hardware.

Barrel Roll Support

Use barrel roll for brush angle control:

<PencilKitView
  onApplePencilData={(data) => {
    // Use roll angle for brush orientation
    const brushAngle = data.rollAngle;
    updateBrushOrientation(brushAngle);
  }}
/>

Predicted Touches

Use predicted touches for latency compensation:

<PencilKitView
  onApplePencilPredictedTouches={(data) => {
    // Pre-render predicted strokes for smoothness
    preRenderStrokes(data.touches);
  }}
/>

Haptic Feedback

Enable haptic feedback for interactions:

<PencilKitView
  enableHapticFeedback={true}
  onApplePencilSqueeze={(data) => {
    // Haptic feedback is automatically triggered
    console.log('Squeeze with haptic feedback');
  }}
/>

🔍 Troubleshooting

Common Issues

1. PencilKit Not Loading: Ensure PencilKit.framework is properly linked in your iOS project
2. Apple Pencil Data Not Captured: Check that enableApplePencilData is set to true
3. Drawing Not Appearing: Verify that the PencilKitView has proper dimensions and styling
4. Apple Pencil Pro Features Not Working: Ensure you're using Apple Pencil Pro and iOS/iPadOS 17.5+

Expo-Specific Issues

1. Development Build Required: This library requires a development build in Expo. Use npx expo run:ios
2. Framework Not Found: Ensure you're using Expo SDK 52+ and have the latest Expo CLI
3. "required a higher minimum deployment target": Set the iOS deployment target to 17.5 (see iOS Setup)

Apple Pencil Pro Issues

1. Squeeze Not Working: Ensure enableSqueezeInteraction is true and using Apple Pencil Pro
2. Hover Not Detected: Ensure onApplePencilHover is registered and test on supported iPad hardware/iPadOS versions
3. Haptic Feedback Missing: Verify enableHapticFeedback is true and device supports haptics

Known issue: Xcode 27 builds on iOS 17

Any app using react-native-nitro-modules (0.36 and 0.37) that is built with Xcode 27 crashes at launch on iOS older than 18 (dyld cannot find std::exception_ptr::__from_native_exception_pointer; margelo/nitro#1652, fix in margelo/nitro#1666, not released yet). Until Nitro ships the fix, build with Xcode 26 if you support iOS 17, or apply the fix from that PR with patch-package (ios/utils/RuntimeError.hpp: throw and catch the error, then use std::current_exception() instead of std::make_exception_ptr).

Apps built with Xcode 27 must also adopt the UIScene lifecycle or they crash at launch on iOS 27 (Apple TN3187). The example app shows a minimal SceneDelegate; Expo apps set expo-build-properties ios.enableSceneSupport: true.

Requirements

• iOS/iPadOS 17.5+ (iOS 27 for the stroke id, selection, erase-path and handwriting APIs)
• React Native 0.75+ (the example uses 0.87.1)
• Xcode 16+ (Xcode 27 for the iOS 27 APIs)
• Apple Pencil (for full functionality)
• Apple Pencil Pro (for advanced features)
• Expo SDK 52+ (for Expo projects; development build, not Expo Go)

👏 Contributing

We welcome contributions! Please see our Contributing Guide for details on how to submit pull requests, report issues, and contribute to the project.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

---