Skip to content

fix: etcd watch restart when receive invalid revision#11833

Merged
Revolyssup merged 4 commits into
apache:masterfrom
Revolyssup:revolyssup/etcd-watch-fix
Dec 18, 2024
Merged

fix: etcd watch restart when receive invalid revision#11833
Revolyssup merged 4 commits into
apache:masterfrom
Revolyssup:revolyssup/etcd-watch-fix

Conversation

@Revolyssup

@Revolyssup Revolyssup commented Dec 16, 2024 •
edited by moonming
Loading

Copy link
Copy Markdown
Contributor

Description

 [error] 56#56: *223892756 lua user thread aborted: runtime error: /usr/local/apisix/apisix/core/config_etcd.lua:260: attempt to compare number with nil
stack traceback:
coroutine 0:
    /usr/local/apisix/apisix/core/config_etcd.lua: in function </usr/local/apisix/apisix/core/config_etcd.lua:130>
coroutine 1:
    [C]: in function 'ngx_thread_wait'
    /usr/local/apisix/apisix/core/config_etcd.lua:285: in function </usr/local/apisix/apisix/core/config_etcd.lua:268>, context: ngx.timer
 [error] 56#56: *223892756 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/core/config_etcd.lua:287: attempt to concatenate local 'err' (a boolean value)
stack traceback:
coroutine 0:
    /usr/local/apisix/apisix/core/config_etcd.lua: in function </usr/local/apisix/apisix/core/config_etcd.lua:268>, context: ngx.timer

When the revision contained in the response returned by etcd watch API is empty, it will cause the watch thread to abort. Since the thread exits abnormally, the error object return from ngx.thread.wait is either nil or a boolean value (nil can be reproduced, but boolean cannot, I don't know why yet), causing subsequent log printing to abort too and making it impossible to restart the watch thread.

Fixes # (issue)

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. bug Something isn't working labels Dec 16, 2024
membphis
membphis reviewed Dec 17, 2024
View reviewed changes
Comment thread apisix/core/config_etcd.lua Outdated
if not ok then
log.error("check_worker thread terminates failed, retart checker, error: " .. err)
log.error("run_watch or check_worker thread terminates failed"
.. " restart those threads, error: ".. inspect(err))

@membphis membphis Dec 17, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code style: bad indentation

@membphis membphis Dec 17, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

log.error("run_watch or check_worker thread terminates failed",
                " restart those threads, error: ", inspect(err))

@Revolyssup Revolyssup Dec 17, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Revolyssup
Revolyssup requested a review from membphis December 17, 2024 20:45
@Revolyssup
Revolyssup merged commit 844b396 into apache:master Dec 18, 2024
@Revolyssup
Revolyssup deleted the revolyssup/etcd-watch-fix branch December 18, 2024 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@membphis membphis membphis approved these changes
@nic-6443 nic-6443 nic-6443 approved these changes
@shreemaan-abhishek shreemaan-abhishek shreemaan-abhishek approved these changes

Assignees

No one assigned

Labels

bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Revolyssup @membphis @nic-6443 @shreemaan-abhishek