Skip to content

Conversation

@DanielMahu
Copy link
Contributor

This fixes #77.

I tried to cover the basic cases in a style similar to the other tests

  • all rows, including a large key and large value
  • a subset of rows
  • no rows
  • error - inexistent table.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 5, 2018
@DanielMahu
Copy link
Contributor Author

During testing I found a couple of corner cases with the empty RowSet that seem broken (they're not in the code right now):

This one failed against the emulator, but succeeded in production, when added to TableReadRowsAllRows:

auto read4 =
  table->ReadRows(bigtable::RowSet(), bigtable::Filter::PassAllFilter());
CheckEqualUnordered(created,  MoveCellsFromReader(read4));

it's testing that the default RowSet matches all rows, which I think is the documented behaviour.

This failed in both, in TableReadRowsNoRows:

ASSERT_TRUE(bigtable::RowSet(bigtable::RowRange::Empty()).IsEmpty());
auto read3 = table->ReadRows(bigtable::RowSet(bigtable::RowRange::Empty()),
                             bigtable::Filter::PassAllFilter());
CheckEqualUnordered(expected, MoveCellsFromReader(read3));

and was broken because we'd expect that reading an empty RowSet should produce no results, but instead it returned all the rows. Maybe we should test for IsEmpty() and skip the actual read if it is true?

@DanielMahu DanielMahu requested review from coryan and mbrukman April 5, 2018 21:55
@coryan
Copy link
Contributor

coryan commented Apr 6, 2018

Thanks for the tests! For read4, if the emulator does not match production please file a bug with the emulator (check #151 for an existing list), skip the test when running against the emulator, and add a TODO for it.

For read3, is there a way to request an empty row set to the production server? If so, what do you think if we issue that request when IsEmpty() returns true?

Copy link
Contributor

@coryan coryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider adding the tests for the corner cases you described, and using std::move(). Otherwise looks pretty good.

bigtable::RowReader& reader) {
std::vector<bigtable::Cell> result;
for (auto const& row : reader) {
std::copy(row.cells().begin(), row.cells().end(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using std::move() (the function in <algorithm> not the function in <utility>)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if you think it is not a good idea or not possible tell me here!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@DanielMahu
Copy link
Contributor Author

I tried replacing the RowRange::Empty() implementation to return ("", "\0") and the integration test passed, but other tests failed because we have some bugs in the RowRange class. I filed #404, and I could take care of it next week.

@coryan
Copy link
Contributor

coryan commented Apr 16, 2018

@DanielMahu any news?

table.ReadRows(bigtable::RowSet(bigtable::RowRange::InfiniteRange()),
bigtable::Filter::PassAllFilter());
#if GOOGLE_CLOUD_CPP_HAVE_EXCEPTIONS
EXPECT_THROW(read1.begin(), std::runtime_error);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I know why this fails with no exceptions enabled. Calling read1.begin() is an error, but we no longer crash the application when the iterator first fails. The application needs to call read1.Finish() to get the error status, and only if they fail to retrieve the status does the code crash (to avoid silent errors).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing that out. But I actually got confused by the line here https://github.com/GoogleCloudPlatform/google-cloud-cpp/blob/dc5c782d69c8f455f075ba602aa90183fbb427c5/bigtable/client/table.cc#L74
that always sets raise_on_error to true... and I still am, it seems to me it's still going down that code path in the noex:: namespace?

I reran the test, with

  EXPECT_EQ(read1.begin(), read1.end());

and got this:

Aborting because exceptions are disabled: Unretriable error: table "projects/emulated-1524060748/instances/data-test/tables/table-read-rows-wrong-table" not found

so the program does die.

Is the no-exception code path ready? otherwise we could submit without this test and fix it separately together with that.

Copy link
Contributor Author

@DanielMahu DanielMahu Apr 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed noex::Table::ReadRows to use raise_on_error=false, I think that should do the trick. PTAL I was obviously wrong, sadly the unit tests seem to pass on my Mac.

Copy link
Contributor

@coryan coryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@coryan coryan merged commit 798f283 into googleapis:master Apr 20, 2018
@DanielMahu DanielMahu deleted the integration-readrows branch May 10, 2018 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement integration test for ReadRows() against the Cloud Bigtable Emulator

3 participants