Current Behavior
When submitting configurations containing stream routes via the standalone API, all configurations except the stream route load correctly, but the stream route does not.
Expected Behavior
The submitted configuration can take effect correctly.
Error Logs
N/A
Steps to Reproduce
- Enable the stream mode in APISIX and create a TCP proxy port for listening.
- Use standalone mode with Admin API enabled.
- Submit a configuration containing the stream route.
- Access that port, and the proxy will not be effective.
The standalone API, as part of the Admin API, operates within the HTTP subsystem. When a worker receives an HTTP request, it notifies other workers via an event broadcasting mechanism. Upon receiving the broadcast, other workers load the new configuration.
The current event broadcasting mechanism correctly notifies the HTTP subsystems in other workers and loads configurations on their Lua VMs. However, the stream subsystem uses a different Lua VM, preventing configuration data sharing.
Additionally, the HTTP subsystem's event broadcasting and the stream subsystem's event broadcasting communicate over separate Unix domain sockets, inherently isolating them.
This results in the stream subsystem failing to receive notifications prompting it to load new configurations, thereby preventing stream routing from taking effect.
To address this issue and further ensure the reliability of the event notification mechanism, I believe we should introduce a background timer loop. This would allow the configuration refresh mechanism to passively take effect even if event broadcasting fails.
Specifically, we should add a timer to the standalone mode. This timer would periodically retrieve the latest configuration from shdict and check whether it is newer than the locally loaded configuration. If it is newer, the configuration would be reloaded.
Given our existing X-Last-Modified record, comparing differences requires no deep inspection—simply matching timestamps suffices.
This mechanism is lightweight and serves solely as a fallback when event notifications fail or when we intentionally update configurations from HTTP to the stream subsystem. We can run it every few hundred milliseconds or once per second. It will operate on both subsystems within each worker.
Environment
- APISIX version (run
apisix version): master
- Operating system (run
uname -a): N/A
- OpenResty / Nginx version (run
openresty -V or nginx -V): N/A
- etcd version, if relevant (run
curl http://127.0.0.1:9090/v1/server_info): N/A
- APISIX Dashboard version, if relevant: N/A
- Plugin runner version, for issues related to plugin runners: N/A
- LuaRocks version, for installation issues (run
luarocks --version): N/A
Current Behavior
When submitting configurations containing stream routes via the standalone API, all configurations except the stream route load correctly, but the stream route does not.
Expected Behavior
The submitted configuration can take effect correctly.
Error Logs
N/A
Steps to Reproduce
The standalone API, as part of the Admin API, operates within the HTTP subsystem. When a worker receives an HTTP request, it notifies other workers via an event broadcasting mechanism. Upon receiving the broadcast, other workers load the new configuration.
The current event broadcasting mechanism correctly notifies the HTTP subsystems in other workers and loads configurations on their Lua VMs. However, the stream subsystem uses a different Lua VM, preventing configuration data sharing.
Additionally, the HTTP subsystem's event broadcasting and the stream subsystem's event broadcasting communicate over separate Unix domain sockets, inherently isolating them.
This results in the stream subsystem failing to receive notifications prompting it to load new configurations, thereby preventing stream routing from taking effect.
To address this issue and further ensure the reliability of the event notification mechanism, I believe we should introduce a background timer loop. This would allow the configuration refresh mechanism to passively take effect even if event broadcasting fails.
Specifically, we should add a timer to the standalone mode. This timer would periodically retrieve the latest configuration from shdict and check whether it is newer than the locally loaded configuration. If it is newer, the configuration would be reloaded.
Given our existing
X-Last-Modifiedrecord, comparing differences requires no deep inspection—simply matching timestamps suffices.This mechanism is lightweight and serves solely as a fallback when event notifications fail or when we intentionally update configurations from HTTP to the stream subsystem. We can run it every few hundred milliseconds or once per second. It will operate on both subsystems within each worker.
Environment
apisix version): masteruname -a): N/Aopenresty -Vornginx -V): N/Acurl http://127.0.0.1:9090/v1/server_info): N/Aluarocks --version): N/A