Skip to content
This repository was archived by the owner on Mar 5, 2026. It is now read-only.
This repository was archived by the owner on Mar 5, 2026. It is now read-only.

adapt.convertStorageSchemaToProto2Descriptor converts Record column names to lower case #358

@brantian

Description

@brantian

adapt.convertStorageSchemaToProto2Descriptor converts the names of RECORD type fields to lower case, which causes data not to be inserted if the data schema keeps the casing of the table

Environment details

  • OS: any
  • Node.js version: v18.6.0
  • npm version: 9.6.6
  • @google-cloud/bigquery-storage version: 3.4.0

Steps to reproduce

  1. Create a table with a RECORD type column with any upper case letters.
    Example:
[
  {
    "name": "stringField",
    "type": "STRING",
    "mode": "NULLABLE"
  },
  {
    "name": "recordField",
    "type": "RECORD",
    "mode": "NULLABLE",
    "fields": [
      {
        "name": "recoredSubField",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  },
  {
    "name": "repeatedField",
    "type": "RECORD",
    "mode": "REPEATED",
    "fields": [
      {
        "name": "repeatedSubField",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  }
]
  1. Adapt its schema to proto descriptor using adapt.convertStorageSchemaToProto2Descriptor and check the output:
  const {  adapt } = require('@google-cloud/bigquery-storage');
  const { BigQuery } = require('@google-cloud/bigquery');

  const bigquery = new BigQuery({ projectId});
  const dataset = bigquery.dataset(datasetId);
  const table = await dataset.table(tableId);
  const [metadata] = await table.getMetadata();

  const storageSchema = adapt.convertBigQuerySchemaToStorageTableSchema(
    metadata.schema
  );

  const protoDescriptor = adapt.convertStorageSchemaToProto2Descriptor(
    storageSchema,
    'InputData'
  );
  
  console.log(protoDescriptor)

Ouput:

  DescriptorProto {
  field: [
    FieldDescriptorProto {
      name: 'stringField',
      number: 1,
      type: 9,
      label: 1,
      options: [Object]
    },
    FieldDescriptorProto {
      name: 'recordfield',
      number: 2,
      type: 11,
      typeName: 'InputData_recordField',
      label: 1
    },
    FieldDescriptorProto {
      name: 'repeatedfield',
      number: 3,
      type: 11,
      typeName: 'InputData_repeatedField',
      label: 3
    }
  ],
...
}

Note that the STRING field stringField keeps its original casing in FieldDescriptorProto name, while recordField and repeteadField are converted to recordfield and repeatedfield.

Thanks!

Metadata

Metadata

Assignees

Labels

api: bigquerystorageIssues related to the googleapis/nodejs-bigquery-storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions