Does Asset Rotation Affect Scale Values in PoseWriter?

Omniverse Isaac Sim
,
1

I am creating synthetic data using PoseWriter in isaaasim 4.2.
[isaacsim.replicator.writers] Isaac Sim Replicator Writers — Isaac Sim

When rotating an asset, the scale value changes. Is this behavior correct?

I tried the two codes below. In code 1, the scale remains (1,1,1), which seems correct, but in code 2, the cubes rotated around the Z-axis no longer have a scale of (1,1,1).

code 1

from omni.replicator.isaac.scripts.writers import PoseWriter, YCBVideoWriter
import omni.replicator.core as rep

camera = rep.create.camera(
    position = (5,5,5),
    look_at = (0,0,0),
    clipping_range=(0.001, 100000)
    )

light = rep.create.light(light_type = "dome",intensity = 250)

render_product = rep.create.render_product(camera, resolution=(512, 512))

cube_1 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,0,0),rotation=(0,0,0))
cube_2 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,-2,0),rotation=(0,0,0))
cube_3 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,2,0),rotation=(0,0,0))

# Setup writer
writer = rep.WriterRegistry.get("PoseWriter")
writer.initialize(output_dir="pose_writer_cube_move_camera",write_debug_images = True, format="centerpose")
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=10, rt_subframes=30):
    with camera :
        rep.modify.pose(
            position=rep.distribution.uniform((-5, -5, 6), (5, 5, 10)),
            look_at = (0,0,0)
                        )

image
image551×554 51.8 KB

image
image1953×666 237 KB

pose_writer_cube_move_camera.zip (3.2 MB)

code 2

from omni.replicator.isaac.scripts.writers import PoseWriter, YCBVideoWriter
import omni.replicator.core as rep

camera = rep.create.camera(
    position = (5,5,5),
    look_at = (0,0,0),
    clipping_range=(0.001, 100000)
    )

light = rep.create.light(light_type = "dome",intensity = 250)

render_product = rep.create.render_product(camera, resolution=(512, 512))

cube_1 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,0,0),rotation=(0,0,0))
cube_2 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,-2,0),rotation=(0,0,45))
cube_3 = rep.create.cube(scale=(1, 1, 1),semantics = [('class', 'cube')],position=(0,2,0),rotation=(0,0,-45))

# Setup writer
writer = rep.WriterRegistry.get("PoseWriter")
writer.initialize(output_dir="pose_writer_cube_rotate_move_camera",write_debug_images = True, format="centerpose")
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=10, rt_subframes=30):
    with camera :
        rep.modify.pose(
            position=rep.distribution.uniform((-5, -5, 6), (5, 5, 10)),
            look_at = (0,0,0)
                        )

image
image551×550 53.6 KB

image
image1922×663 251 KB

pose_writer_cube_rotate_move_camera.zip (2.9 MB)

Isaac Sim Version

4.2.0

Operating System

Ubuntu 22.04

2

How is the situation?
I apologize for reaching out during your busy time, but I would appreciate your response.

5

Hi there,

thank you for the report, it seems the PoseWriter was using AABB for computing the sizes, the code has been changed to use oriented bounding box (OBB). Here are the changes you need to apply to the writer to use this until the fix is out:

            # Size of the object before scale (NOTE: scale is not applied yet to objects in local frame)
            min_local = np.array([bbox["x_min"], bbox["y_min"], bbox["z_min"], 1])
            max_local = np.array([bbox["x_max"], bbox["y_max"], bbox["z_max"], 1])
            size_local = np.abs(max_local - min_local)[:3]
            center_local = min_local + (max_local - min_local) / 2
            if self._write_debug_images:
                self._debug_frame_data["size_local"].append(size_local.tolist())
                self._debug_frame_data["center_local"].append(center_local[:3].tolist())

            # Cuboid keypoints in local frame
            keypoints_local = {
                [..]
            }

            # Calculate the oriented bounding box (OBB) size by combining local size with world scale
            local_to_world_tf_gf = Gf.Transform()
            local_to_world_tf_gf.SetMatrix(Gf.Matrix4d(local_to_world_tf.tolist()))
            world_scale = local_to_world_tf_gf.GetScale()
            size_obb = (size_local * world_scale).tolist()
            if self._format is None:
                obj["size"] = size_obb
            elif self._format == "centerpose":
                obj["scale"] = size_obb