Skip to content

[BUG] Instance health DB sync scheduler can stop after one persistence error #6496

Closed

Description

@Aias00

Current Behavior

InstanceCheckService protects the heartbeat state-check task from exceptions, but the periodic DB sync task is scheduled directly with no exception guard.

executor.scheduleWithFixedDelay(this::scheduled, 30, scheduledTime, TimeUnit.SECONDS);
executor.scheduleWithFixedDelay(this::syncDB, 40, scheduledTime, TimeUnit.SECONDS);

scheduled() catches and logs exceptions, but syncDB() does not:

public void syncDB() {
    instanceHealthBeatInfo.values().forEach(vo -> {
        instanceInfoService.createOrUpdate(vo);
    });
}

If instanceInfoService.createOrUpdate(...) throws once, for example due to a transient database error, the exception escapes the scheduled task. ScheduledThreadPoolExecutor then suppresses subsequent executions of that fixed-delay task.

Expected Behavior

A DB sync failure should be logged and retried in the next scheduled run, rather than stopping all future instance-state persistence.

Impact

After a transient persistence error, in-memory instance health can continue changing while DB state is no longer updated until the admin process restarts.

Code Location

  • shenyu-admin/src/main/java/org/apache/shenyu/admin/service/impl/InstanceCheckService.java
    • setup() schedules syncDB directly.
    • syncDB() has no exception handling around persistence.

Activity

  1. github-actions commented on Jul 31, 2026

    @github-actions
    Contributor

    👋 Hello @yuluo-yx

    A new issue has been opened by @Aias00:

    Please review when you have time. Thank you! 🙏

  2. sunnysabor commented on Oct 2, 2026

    @sunnysabor
    Contributor

    I opened PR #7414 for this member of #6608. It isolates each scheduled persistence call so one failing instance is logged without cancelling the fixed-delay task or preventing the other cached instances from syncing; a later run retries the failed entry. The synchronous syncDB() path used during shutdown is unchanged. The focused InstanceCheckServiceTest and full shenyu-admin suite both pass locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions