Questions about watertight ray-triangle intersection

Gaming and Visualization Technologies Visualization OptiX
1

Hi,

I am testing watertight ray–triangle intersection and have the following three cases:

  1. A ray hits an edge of a single triangle.

  2. A ray hits an edge shared by two triangles.

  3. A ray hits an edge shared by three triangles.

I observed that Case 1 consistently produces a single hit. However, for Cases 2 and 3, the number of reported hits appears to depend on the arrangement of the triangles. If all triangles lie on the same side of ray, they will be counted.

Is there any way in OptiX to reliably determine how many triangles share the edge that the ray intersects? More generally, when a ray passes exactly through a shared edge, is there a robust method to count all triangles incident to that edge?

Thank you.

2

Hi @whix , trying to guess your goal.

Short answer: not from OptiX intersections alone, the OptiX API doesn’t expose topology for hits.

OptiX watertight triangle intersection is meant to avoid cracks. For a ray exactly on an edge shared by two well-formed built-in triangles, the intended behavior is to choose one triangle, not report both. This is a default tie-breaker for shared edges, with best guarantees when triangles share vertex indices and have consistent winding (Tie breaker for when ray hits 2 triangles exactly on edge?)). For three triangles sharing one edge, that is non-manifold topology, and OptiX does not define “all incident triangles” reporting for that case.

Using an any-hit shader is not a robust solution for edge incidence counting. It can gather candidate intersections, especially if you call optixIgnoreIntersection(), but OptiX traversal is BVH-order, not ray-order, and the programming guide notes that any-hit is not guaranteed as an “all possible mathematical hits” enumerator once accepted intersections update the ray interval ([OptiX Programming Guide]( 13.4 - Reporting intersections and attribute access)). OPTIX_GEOMETRY_FLAG_REQUIRE_SINGLE_ANYHIT_CALL only prevents duplicate any-hit calls for the same primitive; it does not force all triangles sharing an edge to be reported (OptiX API).

So what can you do? One possible solution is:

  1. Precompute an edge-adjacency table:
    canonical_edge(min(vertex_i, vertex_j), max(vertex_i, vertex_j)) -> list/count of incident triangle IDs.

  2. In the hit program, get the hit triangle ID and barycentrics:
    float2 uv = optixGetTriangleBarycentrics();
    w = 1 - u - v.

  3. If one barycentric coordinate is zero or within your chosen tolerance, identify the edge:
    w == 0 → edge (v1, v2)
    u == 0 → edge (v0, v2)
    v == 0 → edge (v0, v1)

  4. Look up that edge in your adjacency table. That gives 1, 2, 3, etc., independent of triangle arrangement.

If your mesh is a triangle soup with duplicated vertices, first canonicalize/deduplicate vertices or build the edge table using a deliberate positional tolerance. OptiX can tell you “which primitive won the ray query”; your mesh data must tell you “how many triangles share this edge.”

3

Hi, I am trying to count the number of triangles that share a common edge in order to classify edges. The input mesh is assumed to be clean and may contain boundary edges, manifold edges, or non-manifold edges. In addition, I would also like to identify the IDs of the incident triangles when the edge is shared by exactly two triangles.

I am investigating whether the watertight ray-triangle intersection can be used for this purpose. Here is part of my test code:

// --- Case 1: ray hits the boundary edge of a single triangle (1 face) ---
    printf("=== Case 1: single triangle, hit boundary edge ===\n");
    {
        RayMesh::TriangleMesh mesh;
        mesh.vertices = {
            make_float3(0.f, 0.f, 0.f),
            make_float3(1.f, 0.f, 0.f),
            make_float3(0.f, 1.f, 0.f),
        };
        mesh.indices = {0, 1, 2};
        mesh.num_vertices = 3;
        mesh.num_triangles = 1;

        RayMesh::DeviceMesh d_mesh = RayMesh::OptixIndex::UploadToDevice(mesh);
        RayMesh::OptixIndex index({makeProbeDescriptor()});
        index.BuildAccel(d_mesh, build_opt);

        // Shoot at edge (v0, v1) from slightly above the mesh
        int hits = shootRay(index,
                            make_float3(0.5f, 0.f, 0.1f),   // origin: near edge midpoint, above
                            make_float3(0.f, 0.f, -1.f),       // direction: down toward edge
                            0.2f);                              // tmax
        printf("  hits: %d\n\n", hits);

        RayMesh::OptixIndex::FreeFromDevice(d_mesh);
    }

    // --- Case 2: ray hits the shared edge of two triangles (2 faces) ---
    printf("=== Case 2: two triangles, hit shared edge ===\n");
    {
        RayMesh::TriangleMesh mesh;
        mesh.vertices = {
            make_float3(0.f, 0.f, 0.f),     // v0
            make_float3(1.f, 0.f, 0.f),     // v1
            make_float3(0.5f, 1.f, 0.f),    // v2
            make_float3(0.5f, 1.f, 0.5f),   // v3
        };
        mesh.indices = {0, 1, 2, 0, 3, 1};
        mesh.num_vertices = 4;
        mesh.num_triangles = 2;

        RayMesh::DeviceMesh d_mesh = RayMesh::OptixIndex::UploadToDevice(mesh);
        RayMesh::OptixIndex index({makeProbeDescriptor()});
        index.BuildAccel(d_mesh, build_opt);

        // Shoot at shared edge (v0, v1) from slightly above
        int hits = shootRay(index,
                            make_float3(0.5f, 0.f, 0.1f),
                            make_float3(0.f, 0.f, -1.f),
                            0.2f);
        printf("  hits: %d\n\n", hits);

        RayMesh::OptixIndex::FreeFromDevice(d_mesh);
    }

    // --- Case 3: ray hits the shared edge of three triangles (3 faces) ---
    printf("=== Case 3: three triangles sharing one edge ===\n");
    {
        RayMesh::TriangleMesh mesh;
        mesh.vertices = {
            make_float3(0.f, 0.f, 0.f),      // v0
            make_float3(1.f, 0.f, 0.f),      // v1
            make_float3(0.5f, 1.f, 0.f),     // v2
            make_float3(0.5f, 1.f, -0.5f),    // v3
            make_float3(0.5f, 1.f, -0.5f),    // v4
        };
        mesh.indices = {0, 2, 1, 0, 1, 3, 0, 4, 1};
        mesh.num_vertices = 5;
        mesh.num_triangles = 3;

        RayMesh::DeviceMesh d_mesh = RayMesh::OptixIndex::UploadToDevice(mesh);
        RayMesh::OptixIndex index({makeProbeDescriptor()});
        index.BuildAccel(d_mesh, build_opt);

        // Shoot at shared edge (v0, v1) from slightly above
        int hits = shootRay(index,
                            make_float3(0.5f, 0.01f, 0.1f),
                            make_float3(0.f, 0.f, -1.f),
                            0.2f);
        printf("  hits: %d\n\n", hits);

        RayMesh::OptixIndex::FreeFromDevice(d_mesh);
    }

There is the output and for different triangle arrangement the hit count is different:

=== Case 1: single triangle, hit boundary edge ===
[2026-06-03 16:20:21.958] [info] Enable timing
Hit pridIdx = 0
  hits: 1

=== Case 2: two triangles, hit shared edge ===
[2026-06-03 16:20:22.045] [info] Enable timing
Hit pridIdx = 1
Hit pridIdx = 0
  hits: 2

=== Case 3: three triangles sharing one edge ===
[2026-06-03 16:20:22.126] [info] Enable timing
Hit pridIdx = 1
Hit pridIdx = 2
Hit pridIdx = 0
  hits: 3

Done.
4

Is there a specific reason to use ray tracing for that? One way of doing that, is to use connectivity (i.e. topology) only (I assume you start from a mesh, not a raw point cloud): iterate your data structure (i.e. iterate the faces or the edges, depending on the data struct) and find out for each oriented edge (a,b) how many faces share it. If edge[i].numFaces>2 then the edge is non-manifold.

5

Thanks for the suggestion. The topology-based approach is the standard solution. I am exploring the problem from a different perspective.

6

I’d try my previous answer - or a similar approach - but to me it looks numerically fragile, in general. Moreover the result could depend on internal implementation details.