Floor tracking | LightBuzz

What is Floor Tracking?

The LightBuzz Body Tracking SDK provides advanced floor tracking capabilities by utilizing depth sensors such as Apple LiDAR, Intel RealSense, Orbbec, OAK-D, and Structure Core. Floor tracking enables accurate detection and tracking of the floor plane within a 3D environment. The SDK computes a geometric description of the floor plane as coordinates (X, Y, Z, W), along with valuable utility metrics such as the device’s height from the ground, the camera’s field of view, tilt, and the distance from any 3D point to the floor.

Requirements

To use the Floor Tracking module, your setup must fulfill the following requirements:

A compatible depth camera, such as:
- Apple LiDAR
- Intel RealSense
- Orbbec
- OAK-D
- Structure Core
Ensure the environment is relatively clear of clutter and avoid placing obstacles directly in front of the camera.

Features

Creating a Floor Instance

Initialize a Floor plane instance using a FrameData object:

Floor floor = Floor.Create(frame);

For optimized performance, apply a downsampling factor (recommended sweet spot: 4):

Floor floor = Floor.Create(frame, 4);

Floor Instance Properties

A Floor instance provides the following details:

Coordinates (X, Y, Z, W): Geometric description of the floor plane. (W represents the vertical distance from the sensor to the floor.)
Normal: The normal vector perpendicular to the detected floor plane.
Origin: Reference origin point on the plane.
FieldOfView: Indicates the quality of camera’s view—either OK (optimal) or Limited (view is obstructed or cluttered).
Height: The absolute vertical distance between the sensor and the floor.
IsValid: Boolean indicating if the floor data has sufficient depth information for accurate measurements.
Tilt: Tilt angle of the camera in degrees:
- Positive value: Camera faces upwards.
- Negative value: Camera faces downwards.
- Zero: Camera is straight.
- NaN: Invalid accelerometer data or severely limited view.

Utility Methods

The SDK provides additional utility methods for floor tracking:

Distance(Vector3D point): Calculates the distance (in meters) between the floor plane and a specified 3D point.
Distance(Vector3D point, Axis axis): Calculates the distance between the floor plane and a specified 3D point along a specified axis. (Use Axis.Y for vertical measurements.)
Average(List floors): Computes an average floor plane from multiple floor instances, useful for smoothing and enhancing accuracy.

Frequently Asked Questions (FAQ)

Avoid shaking or moving the device rapidly during capture to ensure accurate tracking.

Does Floor Tracking support multiple planes?

Floor Tracking currently supports only one plane at a time, selecting the largest detected plane as the floor.

Can Floor Tracking detect other surfaces like walls or tables?

No, Floor Tracking specifically focuses on detecting the floor plane and does not detect other surfaces.

Does it work with RGB-only cameras?

No, a depth camera is required. RGB-only cameras are not supported.

How should the camera or device be held for optimal tracking?

Hold the camera, phone, or tablet vertically for the best tracking results.

Is it okay to move the device during capture?

Avoid shaking or moving the device rapidly during capture to ensure accurate tracking.

Do I need to enable Body Tracking to use Floor Tracking?

No! Body-tracking does not need to be enabled for Floor Tracking. You only need a depth sensor streaming data.

Example

Here’s a simple example on using Floor Tracking when a frame instance is available:

private void Update()
{
    FrameData frame = _sensor.Update();
    if (frame == null) return;
    Floor floor = Floor.Create(frame, 4);
    if (floor == null) return;
    FieldOfView fov = floor.FieldOfView;
    float height = floor.Height;
    float tilt = floor.Tilt;
    Debug.Log($"Field of view: {fov}");
    Debug.Log($"Approximate device height: {height:N2}m");
    Debug.Log($"Device tilt: {tilt:N0}°");
    if (fov == FieldOfView.OK)
    {
        Body body = frame.BodyData?.Default();
        if (body != null)
        {
            // Calculate the vertical distance from
            // the floor to the selected joint.
            Vector3D point = body.Joints[JointType.Neck].Position3D;
            float distance = floor.Distance(point, Axis.Y);
            Debug.Log($"Floor - Neck distance: {distance:N2}m");
        }
    }
}

General Information