Skip to content
This repository was archived by the owner on Mar 5, 2026. It is now read-only.
This repository was archived by the owner on Mar 5, 2026. It is now read-only.

Using adapt convertStorageSchemaToProto2Descriptor Converts TIMESTAMP and NUMERIC fields to string #424

@Ashenn1

Description

@Ashenn1

It seems proto2 descriptor encodes these two types as string, and they fail with type errors like
TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)
We are trying to migrate from insertAll method but we faced this issue.
For now we edited the schema and changed TIMESTAMP type to STRING and NUMERIC to FLOAT as a workaround.

Environment details

  • OS: Mac
  • Node.js version: 20
  • npm version: 9.6.7
  • @google-cloud/bigquery-storage version: 4.2.1

Steps to reproduce

Trying to write this example event to a BQ table returns errors,

Schema Example:

    "fields": [
      {
        "name": "total_val",
        "type": "NUMERIC",
        "mode": "REQUIRED"
      },
      {
        "name": "test_timestamp",
        "type": "TIMESTAMP",
        "mode": "REQUIRED"
      }
    ]
  const storageSchema = adapt.convertBigQuerySchemaToStorageTableSchema(schema);
  const protoDescriptor = adapt.convertStorageSchemaToProto2Descriptor(
      storageSchema,
      'root'
  );
  const connection = await writerClient.createStreamConnection({
      destinationTable: `projects/foo-project-id/datasets/test_dataset/tables/test-table`
  });
  const writer = new JSONWriter({
      connection: connection,
      protoDescriptor: protoDescriptor,
  });
  const event = {
      test_timestamp: '2024-02-15T09:22:33Z',
      total_val: 12.4
  };
  const result: IAppendRowsResponse = await writer.appendRows([event]).getResult();

Error for the timestamp field : Error: interior hyphen at Function.fromString
For the numeric field : TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received type number (12.4)

Metadata

Metadata

Assignees

Labels

api: bigquerystorageIssues related to the googleapis/nodejs-bigquery-storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions