Skip to content

feat(standalone): support revision in API-driven standalone mode like etcd#12214

Merged
AlinsRan merged 11 commits into
apache:masterfrom
AlinsRan:feat/incr-sync-config
May 20, 2025
Merged

feat(standalone): support revision in API-driven standalone mode like etcd#12214
AlinsRan merged 11 commits into
apache:masterfrom
AlinsRan:feat/incr-sync-config

Conversation

@AlinsRan

@AlinsRan AlinsRan commented May 14, 2025

Copy link
Copy Markdown
Contributor

Description

In standalone mode, APISIX’s Admin API currently requires clients to retrieve the entire configuration on every sync. As configurations grow in size or change more often, this full-sync approach can lead to unnecessary network traffic and increased latency in applying updates. Moreover, frequent resource changes force a full rebuild of the internal radixtree, which significantly degrades lookup performance under heavy churn.

In the scenario of service discovery, the upstream will always update frequently, and we hope to only update the upstream without affecting the usage of other resources.

Control the changes of each resource by adding <resource-type>_conf_version:

{
    "consumer_groups_conf_version": 1000,
    "consumers_conf_version": 1000,
    "global_rules_conf_version": 1000,
    "plugin_configs_conf_version": 1000,
    "plugin_metadata_conf_version": 1000,
    "protos_conf_version": 1000,
    "routes_conf_version": 1000,
    "secrets_conf_version": 1000,
    "services_conf_version": 1000,
    "ssls_conf_version": 1000,
    "upstreams_conf_version": 1000
    "routes": [
        {
            "id": "r1",
            "uri": "/hello",
            "upstream_id": "u1"
        }
    ],
    "upstreams": [
        {
            "id": "u1",
            "nodes": {
                "127.0.0.1:1980": 1,
                "127.0.0.1:1980": 1
            },
            "type": "roundrobin"
        }
    ]
    "services":[]
}

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@AlinsRan
AlinsRan marked this pull request as ready for review May 15, 2025 01:01
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels May 15, 2025
@AlinsRan
AlinsRan marked this pull request as draft May 15, 2025 08:01
@AlinsRan
AlinsRan force-pushed the feat/incr-sync-config branch from 1c4a014 to 17927e2 Compare May 15, 2025 23:45
@AlinsRan
AlinsRan marked this pull request as ready for review May 16, 2025 01:05
@AlinsRan AlinsRan changed the title feat(standalone): support incremental synchronization feat(standalone): support change memory data by resource type. May 16, 2025
Comment thread apisix/admin/standalone.lua
Comment thread apisix/core/config_yaml.lua Outdated
Comment thread docs/en/latest/deployment-modes.md
@nic-6443

nic-6443 commented May 16, 2025

Copy link
Copy Markdown
Member

The title of this PR is not quite appropriate, it can be changed to:
support revision in API-driven standalone mode like etcd

Comment thread apisix/admin/standalone.lua
Comment thread apisix/admin/standalone.lua
Comment thread apisix/admin/standalone.lua Outdated
Comment thread apisix/core/config_yaml.lua
@AlinsRan AlinsRan changed the title feat(standalone): support change memory data by resource type. feat(standalone): support revision in API-driven standalone mode like etcd May 16, 2025
nic-6443
nic-6443 previously approved these changes May 16, 2025
@AlinsRan
AlinsRan requested a review from bzp2010 May 16, 2025 09:28
membphis
membphis previously approved these changes May 16, 2025
Comment thread apisix/core/config_yaml.lua Outdated
Comment thread apisix/core/config_yaml.lua Outdated
Comment thread apisix/core/config_yaml.lua
@AlinsRan
AlinsRan dismissed stale reviews from membphis and nic-6443 via efdb387 May 19, 2025 08:24
@AlinsRan
AlinsRan requested a review from bzp2010 May 19, 2025 09:03
@AlinsRan
AlinsRan requested review from membphis and nic-6443 May 19, 2025 09:03
Comment thread docs/en/latest/deployment-modes.md
Comment thread t/admin/standalone.spec.ts
Comment thread t/admin/standalone.spec.ts
@juzhiyuan

Copy link
Copy Markdown
Member

Hi, please also check the checklist items to make the pr clear

@AlinsRan
AlinsRan requested a review from bzp2010 May 20, 2025 01:20
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
Comment thread docs/en/latest/deployment-modes.md Outdated
bzp2010
bzp2010 previously approved these changes May 20, 2025

@bzp2010 bzp2010 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree in principle with the PR and we can merge when you resolve all the issues on the documentation.

Comment thread t/admin/standalone.spec.ts

@kayx23 kayx23 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc LGTM

@AlinsRan
AlinsRan requested a review from bzp2010 May 20, 2025 06:19
@AlinsRan
AlinsRan merged commit 0bb54e0 into apache:master May 20, 2025
@AlinsRan
AlinsRan deleted the feat/incr-sync-config branch May 20, 2025 08:16
laz-xyr pushed a commit to laz-xyr/apisix that referenced this pull request Jun 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants