Skip to content

bigquery: Timestamps no longer returned in UTC by default #9407

@bombsimon

Description

@bombsimon

Hi!

Breaking change in v1.59.0 not listed as breaking/intentional.

I'm not sure if this was intentional or if you follow semver for >= v1.0.0 but after bumping a minor version of bigquery our code stopped working as expected because the client no longer returns data the same way. This change is not noted as breaking in the release log.

With the code noted belove, this is my result:

› go get cloud.google.com/go/[email protected]
go: downgraded cloud.google.com/go/bigquery v1.59.0 => v1.58.0

› go run .
1991-01-01 01:01:01 +0000 UTC
› go get cloud.google.com/go/[email protected]
go: upgraded cloud.google.com/go/bigquery v1.58.0 => v1.59.0

› go run .
1991-01-01 02:01:01 +0100 CET

I'm aware of the DATETIME function that allows me to specify timezone, however not all queries is easy enough to manipulate by replacing the selection set to use a function.

Was this intentional? Should I invest time in figuring out a way to update my queries or is there other ways to use UTC by default? Should I convert my timestamps to UTC after fetching them as Go types?

Client

e.g. BigQuery

Environment

Local installation / Any

Go Environment

› go version
go version go1.22.0 darwin/arm64
› go env
GO111MODULE='auto'
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/simon/Library/Caches/go-build'
GOENV='/Users/simon/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/simon/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/simon/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
AR='ar'
CC='/usr/bin/clang'
CXX='/usr/bin/clang++'
CGO_ENABLED='1'
GOMOD='/Users/simon/tmp/test-go/bq/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/yx/19hjwx4n2m9f5453rs5tz1cm0000gn/T/go-build3786632168=/tmp/go-build -gno-record-gcc-switches -fno-common'

Code

package main

import (
	"context"
	"errors"
	"fmt"

	"cloud.google.com/go/bigquery"
	"google.golang.org/api/iterator"
)

func main() {
	ctx := context.Background()
	project := "some-project"

	client, err := bigquery.NewClient(ctx, project)
	if err != nil {
		panic(err)
	}

	datasetName := "some-ds"
	tableName := "some-table"
	columnName := "ts"

	// Create the table and insert data if needed, however this might take up to several minutes
	// to be visible to the client.
	/*
		table := client.Dataset(datasetName).Table(tableName)
		testSchema := []*bigquery.FieldSchema{
			{Name: columnName, Type: bigquery.TimestampFieldType},
		}

		if err := table.Create(ctx, &bigquery.TableMetadata{
			Name:   tableName,
			Schema: testSchema,
		}); err != nil {
			panic(err)
		}

		if err := table.Inserter().Put(ctx, []*bigquery.ValuesSaver{
			{
				Schema:   testSchema,
				InsertID: uuid.New().String(),
				Row: []bigquery.Value{
					time.Date(1991, 1, 1, 1, 1, 1, 0, time.UTC),
				},
			},
		}); err != nil {
			panic(err)
		}
	*/

	query := client.Query(fmt.Sprintf("SELECT %s FROM %s.%s LIMIT 1", columnName, datasetName, tableName))

	it, err := query.Read(ctx)
	if err != nil {
		panic(err)
	}

	var bqRow map[string]bigquery.Value
	if err := it.Next(&bqRow); errors.Is(err, iterator.Done) {
		return
	} else if err != nil {
		panic(err)
	}

	fmt.Println(bqRow[columnName])
}

Expected behavior

The data returned by the client to be the same in v1.58.0 and v1.59.0

Actual behavior

The data is no longer the same, it now holds a local timezone instead of UTC.

Screenshots

n/a

Additional context

Started after upgrading to v1.59.0. Most likely in #9368 and this line.

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions