Bigtable python raises InvalidChunk: possible loss of microsecond timestamp precision? #2397
Description
The smallest reproducible test case I could get to is here (small diff off hello world):
destijl/python-docs-samples@cc074c1
I write a new value to the same row with an older timestamp that is 200us after epoch, and now I can't read the row anymore. My current best guess is that we end up storing a 0 timestamp because microseconds are lost somewhere and there is code that treats not chunk.timestamp_micros as an error as you can see in the backtrace.
The only reason I came across this is I'm implementing a bigtable datastore and our unit tests do weird stuff like this to test timestamp handling of the various databases.
backtrace:
$ python main.py grr-test-demo bigtabletesting
Creating the Hello-Bigtable table.
Writing some greetings to the table.
Getting a single greeting by row key.
Traceback (most recent call last):
File "main.py", line 136, in <module>
main(args.project_id, args.instance_id, args.table)
File "main.py", line 98, in main
row = table.read_row(key.encode('utf-8'), filter_=row_filter)
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/table.py", line 236, in read_row
rows_data.consume_all()
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/row_data.py", line 324, in consume_all
self.consume_next()
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/row_data.py", line 276, in consume_next
self._validate_chunk(chunk)
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/row_data.py", line 391, in _validate_chunk
self._validate_chunk_row_in_progress(chunk)
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/row_data.py", line 371, in _validate_chunk_row_in_progress
_raise_if(not chunk.timestamp_micros or not chunk.value)
File "/usr/local/google/home/gcastle/VE/release/lib/python2.7/site-packages/gcloud/bigtable/row_data.py", line 442, in _raise_if
raise InvalidChunk(*args)