It seems proto2 descriptor encodes these two types as string, and they fail with type errors like
TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)
We are trying to migrate from insertAll method but we faced this issue.
For now we edited the schema and changed TIMESTAMP type to STRING and NUMERIC to FLOAT as a workaround.
Environment details
- OS: Mac
- Node.js version: 20
- npm version: 9.6.7
@google-cloud/bigquery-storage version: 4.2.1
Steps to reproduce
Trying to write this example event to a BQ table returns errors,
Schema Example:
"fields": [
{
"name": "total_val",
"type": "NUMERIC",
"mode": "REQUIRED"
},
{
"name": "test_timestamp",
"type": "TIMESTAMP",
"mode": "REQUIRED"
}
]
const storageSchema = adapt.convertBigQuerySchemaToStorageTableSchema(schema);
const protoDescriptor = adapt.convertStorageSchemaToProto2Descriptor(
storageSchema,
'root'
);
const connection = await writerClient.createStreamConnection({
destinationTable: `projects/foo-project-id/datasets/test_dataset/tables/test-table`
});
const writer = new JSONWriter({
connection: connection,
protoDescriptor: protoDescriptor,
});
const event = {
test_timestamp: '2024-02-15T09:22:33Z',
total_val: 12.4
};
const result: IAppendRowsResponse = await writer.appendRows([event]).getResult();
Error for the timestamp field : Error: interior hyphen at Function.fromString
For the numeric field : TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)
It seems proto2 descriptor encodes these two types as string, and they fail with type errors like
TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)We are trying to migrate from insertAll method but we faced this issue.
For now we edited the schema and changed
TIMESTAMPtype toSTRINGandNUMERICtoFLOATas a workaround.Environment details
@google-cloud/bigquery-storageversion: 4.2.1Steps to reproduce
Trying to write this example event to a BQ table returns errors,
Schema Example:
Error for the timestamp field :
Error: interior hyphen at Function.fromStringFor the numeric field :
TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)