Convert from one coordinate system to another?

Questions
, ,
1

Math is my weakness.

Suppose I receive data from somewhere, and I want to render it in a Three.js scene.

The data assumes that X is forward, Y is left, and Z is up, while Three.js assumes (based on the default camera view) that Z is backward, X is right, and Y is up.

The data that I get are positions, and quaternions (XYZW) in that other coordinate system.

Can we derive a transform matrix that can convert everything into the Three.js coordinate system?

In the data coordinate system, for example, a car drives forward moving along +X, then when the car turns left, it rotates toward the +Y axis.

In Three, the result would be the car drives forward along -Z, and when it turns left the car would rotate toward the -X axis.

Is there some general way to convert from any coordinate system to another? I have a feeling that the answer involves applying a single conversion matrix between the camera matrix, and the rest of the scene, but thatā€™s just a hunch.

1 Like
2

WestLangely and I worked at this topic in 2017. However, the PR was not merged since we have come to the conclusion that exporters should perform the conversion of coordinate systems (and not the engine when importing 3D models). You can probably reuse BasisConverter for your use case.

2 Likes
3

Interesting. I took a brief look at the conversation, and it seems similar to what I do now, which is to modify the vectors and rotations, but to me this seems like the wrong way to do it.

I am not just importing objects, but I am receiving a continuous stream of points/vertices and quaternions, all in another ā€œbasisā€.

But I am imagining that instead of converting the incoming data, I can keep it as is, and instead place a whole-world transform either between the camera matrices and the rest of the scene graph, or as the root-most matrix.

Not sure how this would impact user input for OrbitControls and other controls.

If this is possible, it would greatly simplify things, because we would need to transform data from every loader or every input stream.

Do you think this is possible?

1 Like
4

Thatā€™s not practical. We canā€™t just tell every external library to arbitrarily support Three.js. Itā€™s better if Three has a simple tool to let anyone convert the basis of imported data to Threeā€™s rendering basis. Without some particular external tool doing that conversion, and Three not having a utility, then people are stuck.

1 Like
5

In my opinion converting every vertex being rendered is needlessly complicated and impractical. I think thereā€™s value to preserving the data in itā€™s original basis to operate on as much as possible. A lot of my work takes place in a non-Y up coordinate system, as well, though. If you want a Z up coordinate system you can add an additional object below the scene that rotates 90 degrees about X and voila! Z up coordinate system. Just add all your objects to that object instead of the scene.

1 Like
6

In general, I agree. However that doesnā€™t account for every case. For example, Autodesk Forge viewer works with many difference formats, such as Max, Maya, Revit, etc. that send data in multiple coordinate systems. I havenā€™t delved that deeply into the inner working, but while working with Revit (Z-UP) models, it seems like they have done exactly what @trusktr is suggesting here, that is, modified the internal three.js UP vector depending on which application the data comes from.
I guess their reasoning is that they want people to be able to work interoperatively between these applications and the viewer without having to switch coordinate systems.
Itā€™s not a decision to be taken lightly though because it probably will cause problems elsewhere. The forge viewer is still using r71, which might have something to do with that.

Iā€™ve dealt with this in the LWOLoader and FBXLoader and agree with you, regarding importers specifically. However, it doesnā€™t account for times when users need to do this conversion manually. In these cases itā€™s unneccesary mental overhead that should be stuck in a resuable method somewhere, although not necessarily on the three.js repo :slight_smile:

As another example aside from @trusktrā€™s, Iā€™m currently dealing with matching GPS data from drone photos to 3D models, and Iā€™m having to do loads of conversion between coordinate systems. Itā€™s quite a headache actually, converting from whatever coordinate system the GPS data is in (unknown, because I just have lists of lat/lon/alt positions and kappa/yaw/phi orientations), through Revitā€™s coordinate systems into the Forge Viewer system and then comparing it to the actual photos. Position is correct now but orientation still seems to be off.

Iā€™m finding myself wishing I had an little app to generate the transforms between each coordinate system, so I could throw in my current and desired systems and get back a transformation matrix. Maybe Iā€™ll write on someday.

Itā€™s better if Three has a simple tool to let anyone convert the basis of imported data to Threeā€™s rendering basis.

The problem here is that there are loads of possible coordinate systems. Left handed, right handed, Z-up, Y-UP, Z-Down etc etc. @Mugen87ā€™s example would cover the most common case for 3D apps, but not every case.

7

However that doesnā€™t account for every case. For example, Autodesk Forge viewer works with many difference formats, such as Max, Maya, Revit, etc. that send data in multiple coordinate systems. I havenā€™t delved that deeply into the inner working, but while working with Revit (Z-UP) models, it seems like they have done exactly what @trusktr is suggesting here, that is, modified the internal three.js UP vector depending on which application the data comes from.

Itā€™s true there are other cases, but Iā€™d think they are more rare. I donā€™t know a lot about forge but it looks similar to a modeling tool and maybe it makes most sense to convert all data in that context when geometry has to interact with each other more directly. I think itā€™s something to be avoided, especially if youā€™re dealing with data transit itā€™s a pain because you have to be certain that every piece of data has been transformed everywhere in the app on the way in and on the way out and it makes comparing data in different applications a nightmare. Itā€™s basically asking for bugs. I would opt for just rotating the root objects from different sources appropriately if you can.

The problem here is that there are loads of possible coordinate systems. Left handed, right handed, Z-up, Y-UP, Z-Down etc etc. @Mugen87ā€™s example would cover the most common case for 3D apps, but not every case.

Oh man left handed coordinate systems make this insanely frustrating ā€“ I had to do this for so long when I was using Unity. Honestly I donā€™t think Iā€™ll use a left handed engine for that type of work again if given a choice. Rotation order complicates things further.

Years ago I wrote one for Unity that would create a coordinate system based on the UP, RIGHT, and FORWARD directions along with the order in which to apply rotations on each axis and then you could convert euler angles and positions between them. Something like this:

// left handed system means rotations happen the other way
const unityFrame = new Frame( '+Y+X-Z', '-Y-Z-X' );
const threejsFrame = new Frame( '+Y+X+Z', '+X+Y+Z' );

unityFrame.convertPosition( position, threejsFrame );
unityFrame.convertEuler( angles, threejsFrame );

This would actually be much easier to implement in three.js considering it already has an euler order converter built in.

1 Like
8

I meant specifically the Forge Viewer, which is a web viewer for 3D models built on top of three.js. Thereā€™s no modelling functionality, it takes models from other Autodesk apps, converts them to an intermediate format called SVF and them displays them in browser.

Itā€™s true there are other cases, but Iā€™d think they are more rare.

Well, sure. But somehow I seem to run into the rare cases more often than the ā€œnormalā€ cases :sweat_smile:

1 Like
9

Can we make something like that, but instead of having it convert Vector3s or Eulerā€™s, it just outputs a transformation matrix that converts from one basis to another?

That way, someone could take that matrix and apply it to a root-level object that encapsulates all the objects of a scene from another basis, without having to transform every vector and rotation?

Would this only work if everything is right handed? If there is a basis with a left handed rotation around some axis, would this not work? Or can a single transformation matrix encapsulate a basis-to-basis conversion including arbitrary rotation directions?

So far I think I got lucky with my current use case with the fact the the incoming dataā€™s basis is also right handed, so only the axes need to be transposed.

10

You find the relevant code in the mentioned PR. For example the following matrix:

var matrix = new Matrix4().set(
    1,   0,   0,   0,
    0,   0,   1,   0,
    0, - 1,   0,   0,
    0,   0,   0,   1
);

is a so called change-of-basis matrix. It converts from right-handed Z-up to right-handed Y-up. You can convert a given local or world matrix of a 3D object like so:

var matrixInverse = matrix.clone().transpose();

object.matrix.premultiply( matrix ).multiply( matrixInverse );

This approach also allows to convert from left to right handed coordinates systems (and vice versa) with arbitrary up directions. Itā€™s just a matter of how you compose the change-of-basis matrix.

2 Likes
11

Is that Z-up as in Z-up with X forward, Y left? Or Z-up with X forward, Y right? Or Z-up with -X forward, Y left? Or Z-up with -X forward, Y right?

Plus, for each of those permutate right handed or left handed rotation for each axis. I think that is 32 possible variants of Z-up.

Then thereā€™s Y-up and X-up. So, 96 possible basises?

If we arbitrarily pick two basises, how do we determine the transform matrix to convert between them? What were the steps that you used to create your previous transform matrix? And do those steps include everything needed to come up with the matrix needed to convert from any arbitrary basis to other arbitrary basis?

12

In this particular example, y and z axis are just swapped. X is still the right vector.

I donā€™t know the total amount of possible combinations but you need for each one a respective matrix.

Well, itā€™s not possible to derive the change-of-basis matrix from two given matrices. If you need for some reason all possible combinations, you have to define them once by hand and store them in some sort of map.

13

@Mugen87 I didnā€™t mean programmatically, I meant, how do you by hand, come up with a basis-to-basis conversion matrix given two basises (it is supposed to be ā€œbasesā€ plural, but that can be confused with ā€œbaseā€ singular).

How did you come up with the above matrix? I want to know what your thought process was.

14

@Mugen87 Looks like features like Matrix4.lookAt require an up argument. It seems that simply having a transformation matrix to convert from any arbitrary basis to the Three.js basis is not enough. Any parts of the API that depend on the up values also need to be affected.

I would love a solution to this problem. I think there could be two features:

  1. Telling Three.js what basis to render in by default (f.e. Y is perpendicular to the screen instead of Z).
  2. Being able to somehow render any object in another basis in the chosen Three.js basis.

This is daunting for me though. Iā€™m not comfortable enough with the math yet. I donā€™t have a clear understanding of all the math in order to then think about how to implement a low level thing like that.

As for notation, I think something similar to your Unity example @gkjohnson would be nice.

To set the basis for Three.js, maybe it would be something like

THREE.setBasis({
  up: {axis: 'z', handedness: 'left'},
  left: {axis: '-x', handedness: 'right'},
  forward: {axis: '-y', handedness: 'left'},
})

However, Iā€™m not clear on how to import objects and use them in their original basis, when the Three.js APIs depend on up and handedness across different APIs. Maybe the only way is to convert their data like @Mugen87 suggested, rather than using a transform.

15

Hmmmm, maybe each class can have a basis option, that allows it to have a certain basis.

F.e.

const o = new Object3D
o.basis = { /*similar to THREE.setBasis*/ }

And this object, along with it underlying Vector3s, Matrix4s, Eulers, Quaternions, etc, would all inherit the same basis.

16

To be honest I think involving basis transformation matrices may over complicate things for your use case if youā€™re just trying to change which axis is up. Technically you can have an infinite amount of basis transformations because the axes donā€™t necessarily have to be normalized or 90 degrees to each other. One basis can be skewed and rotated relative to another but I suppose thatā€™s besides the point. Even when just dealing with a orthogonal axes itā€™s a non trivial problem to solve in the general case. If you have data from two different sources that need to display correctly together you have to make sure that they wind up being oriented relative to eachother correctly and that notional ā€œrightā€, ā€œforwardā€, and ā€œupā€ is the same in both frames.

If you just want to display data from a right handed coordinate system with Z up you can apply a 90 degree rotation about X which will point Z to be up:

world.rotation.x = - Math.PI / 2;

The matrix field for world is now a basis vector that can be used to transform vertices if you want to. Another option if you need to transform position vectors is to manually apply the transform to the vector:

vector.set( vector.x, vector.z, - vector.y );

Hereā€™s a fiddle that has a bit of the code working:

I donā€™t mean to discourage work towards a helper class but I just feel that solving right for the general case is complicated and and there are simpler mechanisims that can be used most of the time.

Regarding your new comment:

  1. Telling Three.js what basis to render in by default (f.e. Y is perpendicular to the screen instead of Z).
  2. Being able to somehow render any object in another basis in the chosen Three.js basis.

You can do this by creating a root object that is rotated to the correct orientation as I demonstrated above. This wonā€™t handle every case but should work just fine when displaying data from a right handed coordinate system. You can treat everything under that root as thought itā€™s rendering in the new basis.

17

Well, sure. But somehow I seem to run into the rare cases more often than the ā€œnormalā€ cases :sweat_smile:

Maybe youā€™re just lucky :smiley:. I may have been too strong in my statements. There are clearly plenty of cases where you need to resolve data into the same frame in order to operate on it and merge it. It seems questionable to do it to geometry just for display, though. Iā€™d be curious to understand Forge Viewers reasoning.

18

Well, like I said maybe Iā€™m wrong about Forge. I havenā€™t delved that deeply into the internals, Iā€™m just trying to match an outside coordinate system to the viewer coords and what I have found is that it works when I try to match the Revit system, not the normal three.js system.

19

@gkjohnson Sidenote, it is better to post JSFiddles that link to Three.js using version tags instead of pointing to master to that future people landing here can view working examples.

I run into different basises in practice, and having a tool would greatly simplify this. For example, DOM CSS 3D transforms are in one basis, data from the AI team where I work is another basis, and Three.js is in yet a third basis. Plus Iā€™m sure other tools and frameworks out there have yet more basises.

Yeah, I think those are more rare cases, and anyone with those crazy coordinate systems can probably also do the crazy math.

I think that, for a great API that covers most cases, we can limit it to 3 perpendicular axes, with left/right handedness for each axis, and assuming the same units along each axis. Someone with more complicated cases can convert to a same-unit perpendicular-axes system prior to using the API.

I think I miscalculated the number of possible permutations earlier. If we have this API (for example):

THREE.setBasis({
  up: {axis: 'z', handedness: 'left'},
  left: {axis: '-x', handedness: 'right'},
  forward: {axis: '-y', handedness: 'left'},
})

up means up on the display device, left means to the left on the display device, and forward means pointing into the display device.

For the first up option, we can have six possible values: z, -z, y, -y, x, -x, and two possible handedness values, left and right. This means, for up we can have 6 * 2, or 12, permutations.

For the next one (letā€™s say y), we can only use y or x axis variants. This means 8 possible permutations for the second options (because x, -x, y, and -y for axes and left, right for handedness which means 4 * 2 = 8.

For the last option, weā€™re left with 2*2 = 4 permutations.

So the total permutations is 12 * 8 * 4, or 384.

If we can provide a generic API for this, no one would ever have any more questions (well, they would, but weā€™d simply point them to the API and theyā€™d be on their way).

Not all systems are right-handed. CSS 3D is left-handed, with -Z forward, -X left, -Y up. Hereā€™s a fiddle: Edit fiddle - JSFiddle - Code Playground

I bet there must exist tools that have a mix of left and right handedness per axis, but I havenā€™t had to use any yet.

I donā€™t think weā€™d need to manually write matrices in a map for the solution. I think we can generate a transform to get a sub-tree rendering as expected in Three-space.

However, Iā€™m not sure that all Three.js APIs would be able operate on those objects as we expect. For example, certain APIs expect Y to be up and operate on world transforms, so my guess is there may be problems when the APIs try to operate on any objects in a sub-tree with the root basis transform applied (maybe trying to rotate something manually might go the wrong direction, or etc).

Hmmmmā€¦ speaking of CSS 3D transforms, the THREE.CSS3DRenderer does do some sort of transform to output Three.js matrices into the DOM basis. I wonder what we can learn from that example.

I think this makes one thing obvious: it is east to work in one basis, where all tools operate in the same basis, then export to another basis. F.e., work in Three.js space, then finally export to CSS 3D. But the problems are in importing objects from foreign basis, then working with them using tools that expect the basis of the environment we imported into.

20

I think Iā€™ll just use basii for the plural of basis, to not confuse with the plural of base.