Skip to content

bug: Unable to get the latest upstream node list by service discovery #2369

Description

@haifeng9414

Issue description

I use consul as the registry center and configuration center of apisix and I implemented a consul service discovery through consul-template. After an application is redeployed, the log shows that the consul service discovery has obtained the latest application IP list, but accessing the application through apisix returns 502, the log shows that apisix uses the old application IP.

I think it is caused by the cache of server_picker:

-- apisix/balancer.lua
local server_picker = lrucache_server_picker(key, version,
                            create_server_picker, up_conf, checker)

When the upstream checker is not enabled, the value of version is equal to route.modifiedIndex .. "&" .. service.modifiedIndex, the change of the application ip address in the registry center will not cause this value to change, this will cause the application to be inaccessible until the server_picker cache is invalidated.

I use consul as service discovery instead of eureka, but I believe that using eureka will also have this problem. I think the version should be obtained when obtaining the service ip address:

-- apisix/balancer.lua
if up_conf.service_name then
    if not discovery then
        return nil, "discovery is uninitialized"
    end
    up_conf.nodes = discovery.nodes(up_conf.service_name)
end

change to:

if up_conf.service_name then
    if not discovery then
        return nil, "discovery is uninitialized"
    end
    up_conf.nodes, up_version = discovery.nodes(up_conf.service_name)
end

Environment

  • apisix version (cmd: apisix version): lastest master(251625d)
  • OS: mac os/linux

Minimal test code / Steps to reproduce the issue

  1. Set discovery: eureka and worker_processes: 1, config eureka address in config.yaml.
  2. Deploy a multi-instance application and registry to eureka.
  3. Confirm that the application can be accessed through apisix.
  4. Redeploy the application and access it through apisix again.

What's the actual result? (including assertion message & call stack if applicable)

access application through apisix returns 502

What's the expected result?

no error

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions