Skip to content

Conversation

@eserilev
Copy link
Member

Issue Addressed

Partially #6291

Proposed Changes

This PR aims to "modularize" the beacon processor scheduler. The goal is to set up the scheduler in such a way that new scheduling algorithms can be implemented/swapped in.

For now we've decided on exposing an interface that implements the following trait

pub trait Scheduler<E: EthSpec, S: SlotClock> {
    fn new(
        beacon_processor: BeaconProcessor<E>,
        beacon_state: &BeaconState<E>,
        event_rx: mpsc::Receiver<WorkEvent<E>>,
        spec: &ChainSpec,
    ) -> Result<Box<Self>, String>;

    fn run(
        self,
        work_journal_tx: Option<mpsc::Sender<&'static str>>,
        slot_clock: S,
        maximum_gossip_clock_disparity: Duration,
    ) -> Result<(), String>;
}

We then define a SchedulerType enum

pub enum SchedulerType<E: EthSpec, S: SlotClock> {
    PriorityScheduler(priority_scheduler::Scheduler<E, S>),
}

Each variant is a beacon processor scheduling implementation with its own unique scheduling algorithm. Swapping between the variants can be done either at the config level, or with a small code change.

Additional Info

  • The concept of work queues is currently specific to the PriorityScheduler variant. Other scheduling algorithms might not use all these different queues to schedule the different WorkEvents and so these queues aren't exposed outside of the PriorityScheduler impl.
  • The reprocess event channel is no longer externally exposed. All work events are now sent through the single BeaconProcessorSend channel. I've introduced a new Work::Reprocess type which we then use to schedule jobs for reprocess. The reprocess logic itself is very specific to the PriorityScheduler so it seems to make more sense to give other implementations maximum flexibility.
  • There may be some ways to generalize this even further so that different scheduling implementations aren't duplicating similar code/logic... I think we'll find out more once we start actually implementing a new scheduling algorithm

@eserilev eserilev added code-quality work-in-progress PR is a work-in-progress labels Sep 30, 2024
@eserilev eserilev added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Oct 9, 2024
@eserilev eserilev requested a review from jxs as a code owner February 27, 2025 02:58
@eserilev eserilev added work-in-progress PR is a work-in-progress and removed ready-for-review The code is ready for review labels Mar 13, 2025
@mergify
Copy link

mergify bot commented Mar 13, 2025

This pull request has merge conflicts. Could you please resolve them @eserilev? 🙏

@mergify
Copy link

mergify bot commented May 19, 2025

This pull request has merge conflicts. Could you please resolve them @eserilev? 🙏

@mergify
Copy link

mergify bot commented May 19, 2025

Some required checks have failed. Could you please take a look @eserilev? 🙏

@mergify mergify bot added the waiting-on-author The reviewer has suggested changes and awaits thier implementation. label May 19, 2025
@mergify mergify bot closed this Jun 18, 2025
@mergify
Copy link

mergify bot commented Jun 18, 2025

Hi @eserilev, this pull request has been closed automatically due to 30 days of inactivity. If you’d like to continue working on it, feel free to reopen at any time.

@mergify mergify bot added the stale Stale PRs that have been inactive and is now outdated label Jun 18, 2025
mergify bot pushed a commit that referenced this pull request Jun 20, 2025
Partially #6291


  This PR removes the reprocess event channel from being externally exposed. All work events are now sent through the single `BeaconProcessorSend` channel. I've introduced a new `Work::Reprocess` enum variant which we then use to schedule jobs for reprocess. I've also created a new scheduler module which will eventually house the different scheduler impls.

This is all needed as an initial step to generalize the beacon processor

A "full" implementation for the generalized beacon processor can be found here
#6448

I'm going to try to break up the full implementation into smaller PR's so it can actually be reviewed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked code-quality stale Stale PRs that have been inactive and is now outdated waiting-on-author The reviewer has suggested changes and awaits thier implementation. work-in-progress PR is a work-in-progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant