Skip to content

Gamepad plugin: on-screen touch controls, reported as a gamepad - #1799

Merged
obiot merged 9 commits into
masterfrom
claude/modest-johnson-ao5snz
Oct 30, 2026
Merged

obiot merged 9 commits into
masterfrom
claude/modest-johnson-ao5snz

Conversation

@obiot

@obiot obiot commented Oct 30, 2026

Copy link
Copy Markdown
Member

Description

A new official plugin, @melonjs/gamepad-plugin: on-screen sticks and buttons drawn over the game and read by the engine as a pad in a gamepad slot. A game's input.bindGamepad bindings work with them unchanged, so one set of bindings covers a real pad and a touch screen.

the virtual gamepad

plugin.register(GamepadPlugin, "gamepad", { buttons: [{ label: "A" }, { label: "B" }] });

Plugin (packages/gamepad-plugin)

  • one or two analog sticks with a deadzone, fixed or floating; the left one also presses the D-pad in 8 directions
  • any number of buttons (FACE_1, FACE_2… or a key), which a game can relabel, dim or hide at run time
  • each finger followed on its own (stick and button at once); a touch that starts on the controls never reaches the game
  • drawn over the whole frame after every camera: above any overlay whatever its z, never inside a minimap, kept across stage changes
  • shown on touch screens by default (enabled: device.touch), hidden while a real pad is in use and back on the next touch; visible lets a game word its prompts
  • sizes and positions in game pixels, proportional to the viewport and laid out again on resize
  • input.vibrateGamepad vibrates the phone where navigator.vibrate exists
  • README, CHANGELOG, browser tests, and a developer test page (pnpm -F @melonjs/gamepad-plugin visual)

Engine (packages/melonjs/src/input/gamepad.ts)

  • an internal hook (@ignore @internal, so not in the docs or the published types) that puts a virtual pad in a gamepad slot; the poll reads it together with a real pad in the same slot. The plugin needs melonJS 20.12 for it, checked at registration
  • two fixes in the same poll, reproduced on master first:
    • a pad unplugged while a button or the stick was held left those keys pressed for good
    • a stick bound on one side only never let go of its key when it came back to the centre or rested just past it
  • a key is now released with the same source id it was pressed with

Wiring and docs: CI test and publish jobs, root README and API docs plugin lists, llms.txt, the melonjs, melonjs-plugins, melonjs-input and AGENTS skills.

Two design choices worth a look:

  • the plugin listens to DOM pointer events itself (capture phase on the canvas's parent), because engine pointer regions track one pointer each and drop it when it leaves their bounds, which a stick can't live with
  • it draws by wrapping Stage.prototype.draw (restored on destroy()), so the controls are drawn inside the frame after every camera, before the flush

Type of change

  • Bug fix
  • New feature
  • Documentation update
  • Performance improvement
  • Refactoring (no functional changes)

Checklist

  • I have read the Contributing Guide
  • My code follows the existing code style (pnpm lint passes)
  • I have tested my changes locally (pnpm test passes): the gamepad, input and keyboard specs and the plugin's own suite pass. In the full suite, 2 tests failed in my sandbox: a text glyph pixel check that fails the same way on master's code, and two timeouts that pass on a rerun
  • I have added tests that cover my changes (if applicable)
  • The build succeeds (pnpm build)

Checked by hand in headless Chromium with real touch events on Canvas and WebGL. WebGPU is not verified (no GPU adapter in my sandbox): the visual test page has a renderer picker (#webgpu) for that.

Related issues

Part of #1719. Still to do from that issue: the platformer, isometric dungeon and alex4 examples moved onto the plugin (the platformer's entities/controls.ts removed).


Generated by Claude Code

`input.addVirtualGamepad(pad, index)` puts a pad the game drives itself,
such as on-screen touch controls, in a gamepad slot. The poll reads it with
the pad the browser reports in that slot, so everything bound with
`bindGamepad` answers to either, and `activeGamepad` / vibration see the
virtual pad when the slot holds no real one. No GAMEPAD_CONNECTED is sent
for it, so a game can still use that event to hide its touch controls when
a real pad shows up. This is what the gamepad plugin will build on.

Two fixes in the same loop, both reproduced on master first:
- a pad gone while a button or the stick is held now lets go of its keys
  (a slot with no pad was skipped, leaving them pressed); taking out a
  virtual pad relies on this
- a stick bound on one side lets go of its key when it comes to rest just
  past the centre on the unbound side (the poll returned before the
  release)

A key is released with the same source it was pressed with, now kept on
the binding, as a slot can change between its real and virtual pad.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
A new package, @melonjs/gamepad-plugin, built from the plugin template
and following the in-repo plugins. It draws sticks and buttons and puts
a virtual pad in a gamepad slot (input.addVirtualGamepad), so a game's
bindGamepad bindings answer to the touch controls as to a real pad.

- one or two analog sticks with a deadzone; the left one presses the
  D-pad in 8 directions. Fixed or floating (centered where the thumb
  lands). Keys can be pressed instead, for keyboard-only games
- any number of buttons, FACE_1, FACE_2... by default or a key; a game
  can relabel, dim or hide them at run time
- each finger followed by its own pointer id from DOM pointer events,
  caught in the capture phase on the canvas's parent: a touch starting
  on the controls never reaches the engine or the game, and a stick
  keeps its finger when the thumb drifts off its base
- drawn after the stage, over every camera and every renderable,
  whatever their z, and so never inside a minimap; nothing to load,
  only the renderer's shapes; kept across stage changes
- `enabled` (device.touch by default) and `visible`; hidden while a
  real pad is in use (hideOnGamepad), back when it is unplugged or the
  screen is touched
- input.vibrateGamepad vibrates the phone through navigator.vibrate
- the minimum engine version comes from peerDependencies (>= 20.12.0,
  the release with addVirtualGamepad)

33 browser tests on the Canvas renderer, including that touches are
kept from the game and the controls are drawn over a full-screen
overlay; checked by hand on WebGL. Also: README, CHANGELOG, the API
docs entry, the melonjs-input skill (with symptom entries), CI test
and publish wiring, and the root README plugin list.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
`pnpm -F @melonjs/gamepad-plugin visual` serves visual/, a page with the
controls over a small scene, to try on a desktop or a phone (Vite serves
it on the network too). Not part of the published package.

- a dot the left stick moves (analog through LX/LY, digital through the
  D-pad), bound as a game binds a real pad
- a readout of the plugin's state and of what the engine receives: the
  axes on GAMEPAD_UPDATE, the actions held, the active slot, the real
  pads, and how many taps reached the game (a tap off the controls
  leaves a ring)
- toggles for enabled, two sticks, floating, hide on pad, dim / hide /
  relabel a button, a shade at a very high z, a minimap camera, phone
  vibration, and the renderer (?renderer=canvas|webgl|webgpu)

Checked in headless Chromium with real touch events on Canvas and WebGL:
two fingers on the stick and A at once reach the engine, the game gets
none of those touches but gets a tap in the open, the controls stay over
the shade, and the minimap draws the world without them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
Plugin
- touches are placed against this application's canvas from clientX/Y,
  so they land right on a scrolled page (globalToLocal takes page
  coordinates) and on a page with several applications
- hidden for a real pad again when the player goes back to it after
  touching the screen: the pads are watched each frame while one is
  connected, not only on GAMEPAD_CONNECTED
- the version, an initialized app and a free slot are checked before
  anything is hooked; a second set of controls on a slot throws even
  while switched off
- `ready` checks isInitialized and the viewport (a destroyed app keeps
  both), instead of a guard that was always true
- alpha and deadzone clamped with math.clamp: Canvas ignored an alpha
  above 1 when opacity was over 0.5
- a label keeps one Text, resized with setFont on layout and relabel
- a new vibration effect stops the one still running
- the control classes are exported as types; trimmed comments, a split
  CHANGELOG, README rows; the test page lets #hash win over ?renderer=

Engine
- one virtual pad cannot go in two slots; @throws lists the RangeError
- the connect listeners reuse hasGamepadApi(); reworded comments and
  CHANGELOG entries (a one-sided stick also stuck when it came back to
  exactly 0)
- tests: a negative-side stick back to 0, a keyboard key and a pad on
  one action, a remapped pad unplugged mid-press next to a virtual pad

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
…he README

As the debug plugin's README: the banner and badges under the title, and
a screenshot of the controls (a stick held and leaning, A pressed), taken
from the visual test page and shipped in the package.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
addVirtualGamepad, removeVirtualGamepad and VirtualGamepad stay in
input/gamepad.ts but are @ignore @internal: out of the docs and the
published types, there for the gamepad plugin only. The plugin declares
the two calls itself and reaches them through one cast. Their changelog
entry and skill paragraph go; the two gamepad fixes stay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
In the API docs home page, llms.txt, and the melonjs, melonjs-plugins
(with its own section and trigger words) and AGENTS skills.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JnzwA4xBNBw7cEZzG1bUru
…on-ao5snz

# Conflicts:
#	packages/melonjs/CHANGELOG.md
Copilot AI balanced review requested due to automatic review settings October 30, 2026 01:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@obiot
obiot merged commit 33bfc96 into master Oct 30, 2026
6 checks passed
@obiot
obiot deleted the claude/modest-johnson-ao5snz branch October 30, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants