Skip to content

Separate pending and effective validator updates.#2556

Merged
kansi merged 8 commits intobigchaindb:masterfrom
ldmberman:coclude-multiple-elections-adjustments-2
Sep 21, 2018
Merged

Separate pending and effective validator updates.#2556
kansi merged 8 commits intobigchaindb:masterfrom
ldmberman:coclude-multiple-elections-adjustments-2

Conversation

@ldmberman
Copy link
Copy Markdown
Contributor

@ldmberman ldmberman commented Sep 18, 2018

  • Pending validator updates do not prevent elections from concluding.
  • ValidatorElection overrides has_concluded to not conclude when there is a pending update for the matching height.
  • Effective validator updates deem past elections inconclusive.

- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.
@ldmberman ldmberman requested review from kansi and vrde September 18, 2018 14:39
@codecov-io
Copy link
Copy Markdown

codecov-io commented Sep 18, 2018

Codecov Report

Merging #2556 into master will decrease coverage by 0.01%.
The diff coverage is 96.66%.

@@            Coverage Diff             @@
##           master    #2556      +/-   ##
==========================================
- Coverage   93.65%   93.63%   -0.02%     
==========================================
  Files          45       45              
  Lines        2630     2623       -7     
==========================================
- Hits         2463     2456       -7     
  Misses        167      167

Solution: Record placed elections, update the records upon election conclusion.
@ldmberman ldmberman force-pushed the coclude-multiple-elections-adjustments-2 branch from 7c8ee8c to 4a96216 Compare September 19, 2018 13:15
elections = OrderedDict()
for tx in txns:
if isinstance(tx, Election):
tx.store(bigchain, new_height, is_concluded=False)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend to aggregate and store

(votes_committed + votes_current >= (2/3)*total_votes):
return True
if self.has_validator_set_changed(bigchain):
return False
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to move this check to the beginning of the function so that the votes are not counted unnecessarily.

Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.
@ldmberman
Copy link
Copy Markdown
Contributor Author

Election records are inserted in bulk now.

There is also the crash recovery issue (already in master) which is not only connected with election records but all database artifacts we create during end_block. In order to address it, we need to extend the crash recovery which happens on startup to account for all the new data that was introduced. I will make it in a separate PR.

(votes_committed + votes_current >= (2/3)*total_votes):
return True
current_validators = self.get_validators(bigchain)
total_votes = sum(current_validators.values())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It occured to me that one could sum the "amount" of the outputs of the election to get the total_votes and avoid the query.

cursor = conn.run(
conn.collection('elections')
.find(query, projection={'_id': False})
.sort([('height', DESCENDING)])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to use find_one and sort as an argument to the query.

@kansi
Copy link
Copy Markdown
Contributor

kansi commented Sep 21, 2018

This code is a dependency for fixing crash recovery so I will merge it. @vrde Please leave your comments and @ldmberman will address them in a separate a PR.

@kansi kansi merged commit 24ca0b3 into bigchaindb:master Sep 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants