This repository was archived by the owner on Nov 17, 2023. It is now read-only.
[2.0] Precission loss when exporting a model from 2.0 #20420
Description
Here's a simple model which just has one operator:
def forward(self, a):
return a - 37.38088245350211
This model will export this symbol file:
{
"nodes": [
{
"op": "null",
"name": "data",
"attrs": {"__profiler_scope__": "<unk>:"},
"inputs": []
},
{
"op": "_npi_subtract_scalar",
"name": "node_0",
"attrs": {
"is_int": "0",
"scalar": "37.3809"
},
"inputs": [[0, 0, 0]]
}
],
"arg_nodes": [0],
"node_row_ptr": [0, 1, 2],
"heads": [[1, 0, 0]],
"attrs": {
"is_np_shape": ["int", 1],
"mxnet_version": ["int", 20000]
}
}
as we can see here the scalar only has 4 digits precision after the decimal point.