Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Conversation

@davxy
Copy link
Member

@davxy davxy commented Aug 19, 2022

Closes #432


This PR prevents the StateDbError::TooManySiblingBlocks error from being triggered by eagerly removing leaves from the backend on block import and before the error condition is met.


⚠ SUPERSEEDS: #1544

The main difference is this is a portable solution WRT the leaves order read from the backend.

@davxy davxy requested review from a team, bkchr and skunert August 19, 2022 16:46
@davxy davxy added B0-silent Changes should not be mentioned in any release notes A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. labels Aug 19, 2022
@davxy davxy self-assigned this Aug 19, 2022
@davxy
Copy link
Member Author

davxy commented Aug 22, 2022

(Updated)

Following #1559 (comment), let's recap the problems to be solved:

  • issue 1. We need to fetch the list of all the blocks at the same level of the imported block.
  • issue 2. if this list length is >= the threshold then we have to choose which block to remove.
  • issue 3. The block to remove may not be a leaf (e.g. because what has been said in the linked comment). In this case we also have to remove all its descendants.

I don't see a direct support from the Backend to solve the issues, let's see if we can somehow work around the limitations.

  • issue 1. This is probably the most annoying. Is there already an easy and possibly low cost way to get all blocks at one level?
  • issue 2. The choice will be again driven by the "age". Probably the best thing to do is: for each block at the same height of the imported one its "age" is set as the age of its most recent descendant leaf (for example if a block is the parent a very old leaf and a very fresh one, then we consider the block age equal to the most fresh leaf).
  • issue 3. We can remove all the block's descendants by starting from the leaves and using the Backend::remove_leaf_block.

A block descendants can be recovered using the leaves can be found using the sp_blockchain::tree_route (is expensive... but also a very sporadic operation).

So the main question here is: how to get the blocks list at one level?

I'm open to more simple suggestions :-)

@davxy
Copy link
Member Author

davxy commented Nov 17, 2022

@davxy what is the status with merging this earlier?

Confident to merge this very soon. I just need to add a small safety net

@sandreim
Copy link
Contributor

sandreim commented Dec 9, 2022

bot rebase

@paritytech-processbot
Copy link

Error: Command 'Command { std: "git" "merge" "origin/master" "--no-ff" "--no-edit", kill_on_drop: false }' failed with status Some(1); output: no output

davxy and others added 3 commits December 9, 2022 17:54
* Implemented explicit recovery for announced candidate blocks

* Removed magic number

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Introduce a random delay before attempting blocks explicit recovery

* Richer recovery request message

* Rename 'pending_candidates' PoVEntry to 'candidates'

* Refactory

* Some comments wrt the chosen recovery delay

* Prevent recovery of imported blocks

* Added test to excercise explicit block recovery request

* Trivial cleanup

* Fix ancestry assumption that may break recovery

* Apply code review suggestion

Co-authored-by: Bastian Köcher <[email protected]>
@bkchr
Copy link
Member

bkchr commented Dec 20, 2022

@davxy can we merge this?

@davxy davxy added the D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit label Dec 20, 2022
@davxy
Copy link
Member Author

davxy commented Dec 20, 2022

bot merge

@davxy davxy closed this Dec 20, 2022
@davxy davxy reopened this Dec 20, 2022
@paritytech-processbot
Copy link

Error: Github API says #1559 is not mergeable

@davxy davxy merged commit 73837c3 into master Dec 20, 2022
@davxy davxy deleted the davxy/remove-leaves-portable branch December 20, 2022 11:13
@mn13 mn13 mentioned this pull request Jan 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit

Projects

Status: done

Development

Successfully merging this pull request may close these issues.

Remove "dead" leaves

6 participants