Skip to content

fix: etcd sync data checker should work#11457

Merged
moonming merged 2 commits into
apache:masterfrom
ronething:fix/etcd_checker
Aug 2, 2024
Merged

fix: etcd sync data checker should work#11457
moonming merged 2 commits into
apache:masterfrom
ronething:fix/etcd_checker

Conversation

@ronething

@ronething ronething commented Aug 1, 2024

Copy link
Copy Markdown
Contributor

Signed-off-by: ashing [email protected]

Description

Fixes #11216

fix part 2 of this issue

Background

the load_full_data function in the file config_etcd.lua(load all exist data from etcd when apisix start), checker does not depend on item_schema to have a value

if data_valid and self.checker then
data_valid, err = self.checker(item.value)
if not data_valid then
log.error("failed to check item data of [", self.key,
"] err:", err, " ,val: ", json.delay_encode(item.value))
end
end

But in the sync_data function (watch new events from etcd when apisix running), checker relies on item_schema should have value.

if data_valid and res.value and self.item_schema then

As a result, the plugin metadata resource cannot reach the checker logic when etcd watch.

apisix/apisix/plugin.lua

Lines 816 to 821 in 507df12

local plugin_metadatas, err = core.config.new("/plugin_metadata",
{
automatic = true,
checker = check_plugin_metadata
}
)

So we need to fix it.

about the test case:

The admin api is not used for the operation, because the admin api will directly fail to verify the schema and cannot go to the etcd checker logic.

error-log-logger-clickhouse.t: The request header sent to clickhouse should be decrypted to be correct.
There was a problem with the previous modification of this PR
https://github.com/apache/apisix/pull/11076/files#diff-2974594095136be865db5d415f3da92d2a435211e30f93f693f1ed46a4a90037

PS: This PR is a partial fork of #11430.

about failed CI

My PR changes have nothing to do with this CI error. The reason for the error is a problem with the image version used by the github runner.

image

ref: https://github.com/apache/apisix/actions/runs/10193727573/job/28202213850?pr=11457

  • The correct github runner image version is as follows,
image

ref: https://github.com/apache/apisix/actions/runs/10193727573/job/28202215238?pr=11457

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@shreemaan-abhishek

Copy link
Copy Markdown
Contributor

I am confused about one thing, in this line:

if data_valid and self.item_schema then

load_full_data validates the item_schema is not falsy.

Then why do you say this:

checker does not depend on item_schema to have a value

@ronething

Copy link
Copy Markdown
Contributor Author

checker does not depend on item_schema to have a value

@shreemaan-abhishek What I mean is that the execution of the checker function does not depend on the item_schema having a value.

ref:

if data_valid and self.checker then
data_valid, err = self.checker(item.value)

@moonming

moonming commented Aug 1, 2024

Copy link
Copy Markdown
Member

@ronething I rerun test cases and passed.

@shreemaan-abhishek shreemaan-abhishek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ronething

Copy link
Copy Markdown
Contributor Author

@moonming can you merge it, thanks.

@moonming
moonming merged commit 8728863 into apache:master Aug 2, 2024
@ronething
ronething deleted the fix/etcd_checker branch August 2, 2024 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: the plugin metadata obtained from etcd has not been validated by schema

4 participants