Skip to content

Problem: New collections are not created in existing DB.#2520

Merged
vrde merged 4 commits intobigchaindb:masterfrom
ldmberman:create-collections-if-do-not-exist
Sep 12, 2018
Merged

Problem: New collections are not created in existing DB.#2520
vrde merged 4 commits intobigchaindb:masterfrom
ldmberman:create-collections-if-do-not-exist

Conversation

@ldmberman
Copy link
Copy Markdown
Contributor

Solution: Do not abort the initialisation if a collection exists. Unify the index creation.

@ldmberman ldmberman requested review from kansi and vrde September 6, 2018 09:04
@codecov-io
Copy link
Copy Markdown

codecov-io commented Sep 6, 2018

Codecov Report

Merging #2520 into master will decrease coverage by 0.17%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #2520      +/-   ##
=========================================
- Coverage   93.37%   93.2%   -0.18%     
=========================================
  Files          43      44       +1     
  Lines        2583    2547      -36     
=========================================
- Hits         2412    2374      -38     
- Misses        171     173       +2

try:
logger.info(f'Create `{table_name}` table.')
conn.conn[dbname].create_collection(table_name)
create_indexes(conn, dbname, table_name, INDEXES[table_name])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we just add new index?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can. But we would need to change logging then. Unconditional Creating indexes for .. messages are disorienting.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we should change logging and ensure all the indexes exist too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Contributor

@kansi kansi Sep 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the code above, if the collection already exists then the exception is generated and the code jumps to the except clause. So if there is merely an addition of a new index then that wouldn't be updated in the existing collection. The point here would be to ensure that the collection and indexes are the way we expect them to be

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about it. Most of our indexes come with a unique constraint. Unique constraints are not backwards-compatible and thus can not be introduced by a new bigchaindb start or bigchaindb init.

On the other hand, we are losing a good means of adding those rare indexes without constraints.. So, I think it's better to arrange it how you suggest.



@pytest.mark.tendermint
def test_bigchain_run_init_when_db_exists(mocker, capsys):
Copy link
Copy Markdown
Contributor

@kansi kansi Sep 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this test could be re-purposed to asset a success message?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This particular test does not use a real database but asserts that one function catches an exception raised by the other.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove the mock and execute the actual code. But I would recommend skipping this comment if its too much work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a pattern of not using the real database in this file so I would leave it to be handled by future test (re)organisation efforts.

Copy link
Copy Markdown
Contributor

@kansi kansi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good, please resolve the conflicts though

@ldmberman ldmberman force-pushed the create-collections-if-do-not-exist branch from bc3b13a to 479d6d7 Compare September 11, 2018 13:22
Solution: Do not abort the initialisation if a collection exists. Unify the index creation.
@ldmberman ldmberman force-pushed the create-collections-if-do-not-exist branch from 479d6d7 to 3a0e1af Compare September 12, 2018 09:09
Copy link
Copy Markdown
Contributor

@vrde vrde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job!

I see you added/removed some markers from the tests, is it something specific to this PR? If not, they should live in another one.

return create_tx.sign([alice.private_key])


@pytest.mark.abci
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it something related to this PR?

register_schema = module_dispatch_registrar(backend.schema)


INDEXES = {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this!

@vrde vrde merged commit 35e35ec into bigchaindb:master Sep 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants