PortalCamera

An off-axis (“physical”) perspective camera for head-coupled / portal rendering in Three.js.

You define a screen plane (monitor) and an eye (head position). The camera builds an asymmetric frustum that makes the 3D scene line up with the physical screen.

Internally, PortalCamera extends THREE.PerspectiveCamera for ecosystem compatibility (post-processing, controls, helpers). It overrides the projection math. Properties like fov/aspect are ignored as inputs and exposed only as read-only effective values for UI/inspection.

This repo provides a standalone Three.js demo that requires a separate tracker (see below) and useful helpers/tools for visualizing and configuring.

Reference: Robert Kooima, "Generalized Perspective Projection"

Model attribution

Model: Littlest Tokyo by Glen Fox — CC Attribution. Artstation Page

Project Inspiration

Blog: Interactive-Frame-Head-Tracking by Charlie Gerard.

Quickstart (local)

npm i
npm run dev

Open the printed URL in the terminal.

Build

npm run build
npm run preview

All-in-one DEMO

This is a single static page that combines the tracking solution and renderer together. Performance is worse with this method (probably due to everything living in the lil-gui) but a quick way to experience the rendering method.

PortalCamera-Demo

Minimal PortalCamera Template

No helpers, gui or model assets. This template contains the core logic required to run all of the main components for this rendering method

PortalCamera-minimal

Head tracking

Here's the standalone tracker I made to pair with this PortalCamera Example.

portal-head-tracker

The app sends WebSocket pose stream if available:

• VITE_LUCI_WS_URL=ws://localhost:8787 (or wss://…)

I recommend dialing in Offsets and Gains in the tracker first to properly align your eyes(Pe) relative to the monitor and then you can use the tuning knobs in the PortalCamera demo for finer control.

If not set, the demo runs with defaults and the GUI controls downstream X/Y/Z gain and smoothing.

We manage this by creating the websocket connection with /net/HeadPoseWS.js and applying those coordinates with "applyPose" from /controls/HeadCoupledController.js.

---

Table of Contents

• Mental model
• Quick start
• Constructor
• Methods
  • Screen setup
  • Eye & clipping
  • Recompute projection + placement
• Read-only “effective” values
• Gotchas & best practices
• Interop tips
• Validation checklist
• FAQ
• Minimal example (full loop)

---

Mental model

• Screen plane = 3 points in world space: pa (lower-left), pb (lower-right), pc (upper-left).
  From these we form a screen basis r,u,n (right, up, normal).
• Eye = pe (head position).
• We compute near-plane bounds l, r, b, t by projecting (pa−pe), (pb−pe), (pc−pe) onto r and u and scaling by near / d, where d is the signed distance from the eye to the screen along n.
• The camera’s world transform is set so its local axes line up with r,u,n and its position is pe.

---

Quick start

import * as THREE from 'three';
import { PortalCamera } from './PortalCamera.js';

// Define a screen by frame (center, right, up, width, height) and initial eye:
const camera = new PortalCamera({
  near: 0.02,
  far:  50,
  screenByFrame: {
    center: new THREE.Vector3(0, 0, 0),
    right:  new THREE.Vector3(1, 0, 0),
    up:     new THREE.Vector3(0, 1, 0),
    width:  0.60,  // meters
    height: 0.34,  // meters
  },
  eye: new THREE.Vector3(0, 0, 0.6),
});

// On each head-pose update:
camera.setEyeXYZ(px, py, pz);
camera.updateMatrices(); // also runs if something calls camera.updateProjectionMatrix()

Mirror the browser window (for demos):

camera.fitToViewport({ width: canvas.width, height: canvas.height }, 1.0);

---

Constructor

new PortalCamera(opts?: {
  near?: number;              // default 0.01
  far?: number;               // default 1000
  screenByFrame?: {           // optional: define screen by frame
    center: THREE.Vector3;
    right:  THREE.Vector3;
    up:     THREE.Vector3;
    width:  number;
    height: number;
  };
  screenByCorners?: {         // optional: define screen by corners (virtual world)
    pa: THREE.Vector3;        // lower-left
    pb: THREE.Vector3;        // lower-right
    pc: THREE.Vector3;        // upper-left
  };
  eye?: THREE.Vector3;        // initial eye
  autoFlipNormal?: boolean;   // default true (flip if eye behind screen)
  epsilon?: number;           // default 1e-6 (numeric guard)
})

Why extend PerspectiveCamera?
We keep isPerspectiveCamera === true so post-processing, controls, and helpers don't break (with time it'd be more appropriate to build the types and a solid class that only extends camera.js) We override updateProjectionMatrix() to delegate to our physical math.

---

Methods

Screen setup

setScreenByFrame(
  center: THREE.Vector3,
  right: THREE.Vector3,
  up: THREE.Vector3,
  width: number,
  height: number
): this

Define by a frame. right and up are orthonormalized internally. (Recommended method, reflecting your physical monitor dimensions in fullscreen provides the most accurate representation)

setScreenByCorners(pa: THREE.Vector3, pb: THREE.Vector3, pc: THREE.Vector3): this

Define the screen by three world-space corners (pa=LL, pb=LR, pc=UL). (This is basically like the portal addon camera utils, can arbitrarily be set in an existing virtual world)

fitToViewport(
  size: { width: number; height: number },
  unitsPerShortEdge = 1
): this

Convenience: centers a screen at the origin sized to the viewport’s aspect. (Once again, for our preferred "fullscreen" version this is very useful, and differentiates this technique from the CameraUtilsPortal)

---

Eye & clipping

setEye(v: THREE.Vector3): this
setEyeXYZ(x: number, y: number, z: number): this
setNearFar(near: number, far: number): this

---

Recompute projection + placement

updateMatrices(): this

• Builds the asymmetric projection matrix (projectionMatrix, projectionMatrixInverse).
• Sets position = eye and quaternion from the screen basis.
• Caches friendly inspection values (see below).

updateProjectionMatrix() is overridden to call updateMatrices(), so external libs that invoke it are safe. Any loose functions that updateMatrices() might break our Portal render otherwise.

---

Read-only “effective” values

For inspector UIs and debugging, we cache:

• camera.fov → effective vertical FOV at the near plane (degrees)
• camera.aspect → effective near-plane aspect ratio = (right−left)/(top−bottom)

Note: Setters for fov, aspect, filmGauge, filmOffset, and setFocalLength() warn and are ignored.
The projection is controlled exclusively by the screen geometry and eye.

---

Gotchas & best practices

• Call updateMatrices() after changing any of: screen corners/frame, eye, near, or far.
  (If another lib calls updateProjectionMatrix(), ours forwards correctly.)
• autoFlipNormal (default: true): Helpful when the eye crosses the plane; flip it off for strict one-sided behavior.
• Units: Treat screen width/height in meters (or consistent world units) for intuitive tuning.
• Numerical safety: We guard tiny distances and degenerate edges with epsilon. If your screen corners are collinear or identical, the update is a no-op.
• Helpers: A visual “screen frame” helper is invaluable—draw the rectangle through pa/pb/pc and rays from pe for debugging.

---

Interop tips

• Post-processing / EffectComposer: Works as with any perspective camera (we keep isPerspectiveCamera).
• Controls: OrbitControls will technically work but may fight your intent. Prefer your own eye driver (e.g., head tracker) and keep OrbitControls disabled or limited. To cleanly swap between you want to dispose the former camera.
• CameraHelper: Renders asymmetric frusta correctly since we provide a valid perspective projection matrix. *The included helper from the demo is a work in progress.
• FrameHelper: This is useful for properly placing assets in the scene and constructs itself from the defined near plane. It also helps for tweaking the tracker settings to line up the edges of your monitor.

---

Validation checklist

• Move eye left/right/up/down: scene should “stick” to the screen edges (fish-tank VR effect).
• Extreme positions: with autoFlipNormal=true, the view remains valid when crossing the screen plane. In Kooima's formula this orientation is required for proper rendering. The handedness of the world is key for the correct effect.
• Fullscreen toggling: ensure your renderer resizes, then call fitToViewport() or recompute your screen as needed, followed by updateMatrices().

---

FAQ

Q: Why ignore fov/aspect?
They’re artifacts of the symmetric camera API. In a physical off-axis camera, the frustum is dictated by the screen geometry and eye, not a single FOV number. We still expose effective values for UI/readout.

Q: Can I construct from only monitor size + pose?
Yes—use setScreenByFrame(center,right,up,width,height) with your monitor plane in world space, plus setEye(...). Reccommended Method:

Q: Can I use world units other than meters?
Yes—just be consistent across screen sizes, eye positions, and scene scale.

---

Minimal example (full loop)

import * as THREE from 'three';
import { PortalCamera }          from './cameras/PortalCamera.js';
import { HeadCoupledController } from './controls/HeadCoupledController.js';
import { createHeadPoseWS }      from './net/HeadPoseWS.js';

/* === Mount & renderer === */
const container = document.getElementById('container') || document.body;
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setPixelRatio(Math.min(window.devicePixelRatio || 1, 2));
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.outputColorSpace = THREE.SRGBColorSpace;
container.appendChild(renderer.domElement);

/* === Scene === */
const scene = new THREE.Scene();
scene.background = new THREE.Color(0xbfe3dd);

/* === Portal camera (physical screen + eye) === */
const camera = new PortalCamera({ near: 0.01, far: 10.0 });
scene.add(camera);

// Example: ~27" 16:9 monitor in meters
const SCREEN_W = 0.59, SCREEN_H = 0.34;
const screenCenter = new THREE.Vector3(0, SCREEN_H * 0.5, 0);
camera.setScreenByFrame(
  screenCenter,
  new THREE.Vector3(1, 0, 0), // right
  new THREE.Vector3(0, 1, 0), // up
  SCREEN_W,
  SCREEN_H
);

// Start eye ~60 cm in front of the screen plane
camera.setEyeXYZ(0, 0.15, 0.6).updateMatrices();

/* === Demo geometry: a simple cube === */
const cube = new THREE.Mesh(
  new THREE.BoxGeometry(0.1, 0.1, 0.1),
  new THREE.MeshNormalMaterial()
);
// Updated position for first-time users
cube.position.set(0, 0.18, -0.1);
scene.add(cube);

/* === Head-coupled controller === */
const headCtrl = new HeadCoupledController(camera, {
  smoothing: 1.0,
  trackerZScale: 1.0,
  gain: { x: 1, y: 1, z: 1 },
  deadzone: 0,
});

/* === WebSocket pose hookup (minimal) === */
const WS_URL =
  new URLSearchParams(location.search).get('ws') ||
  (import.meta.env && import.meta.env.VITE_LUCI_WS_URL) ||
  `${location.protocol === 'https:' ? 'wss' : 'ws'}://${location.hostname}:8787`;

console.log('[WS] connecting →', WS_URL);
createHeadPoseWS({
  url: WS_URL,
  onPose: (x, y, z) => headCtrl.applyPose(x, y, z),
  onState: (s) => console.log('[WS]', s),
});

/* === Resize === */
function onResize() {
  renderer.setSize(window.innerWidth, window.innerHeight);
  camera.updateMatrices();
}
window.addEventListener('resize', onResize);

/* === Render loop === */
function animate() {
  requestAnimationFrame(animate);
  renderer.render(scene, camera);
}
animate();

HTML MOUNT

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
    <title>PortalCamera Minimal</title>
    <style>
      html, body, #container { margin:0; padding:0; width:100%; height:100%; background:#0b0b0b; }
      canvas { display:block; }
    </style>
  </head>
  <body>
    <div id="container"></div>
    <script type="module" src="/src/main.js"></script>
  </body>
</html>