Skip to content

request help: configure etcd mTLS warn certificate host mismatch #5155

Closed
#5206
Closed
request help: configure etcd mTLS warn certificate host mismatch#5155
#5206

Description

@dyrnq
dyrnq
opened on Sep 29, 2021

Issue description

Apisix use mTLS(self-certification) connect etcd report error when config etcd.tls.verify=true on apisix2.10.0 version，but the same config run ok on apisix2.9

case 1

apisix:
  node_listen: 9080
  enable_ipv6: false
  enable_debug: true
  allow_admin:
    - 0.0.0.0/0
  admin_key:
    - name: "admin"
      key: edd1c9f034335f136f87ad84b625c8f1
      role: admin
  ssl:
    ssl_trusted_certificate: /opt/apisix/pki/etcd/ca.crt
etcd:
  host:
    - "https://192.168.27.11:2379"
    - "https://192.168.27.12:2379"
    - "https://192.168.27.13:2379"
  prefix: "/apisix"
  timeout: 30
  tls:
    cert: /opt/apisix/pki/etcd/etcd-client.crt
    key: /opt/apisix/pki/etcd/etcd-client.key
    verify: true
plugin_attr:
  prometheus:
    export_addr:
      ip: "0.0.0.0"
      port: 9091

error.log

2021/09/29 07:15:32 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] health_check.lua:63: report_failure(): update endpoint: https://192.168.27.11:2379 to unhealthy, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] health_check.lua:63: report_failure(): update endpoint: https://192.168.27.12:2379 to unhealthy, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:34 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.13:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:34 [error] 51#51: *55053 [lua] config_etcd.lua:591: failed to fetch data from etcd: /usr/local/openresty/lualib/resty/core/socket/tcp.lua:209: assertion failed!
stack traceback:
	[C]: in function 'assert'
	/usr/local/openresty/lualib/resty/core/socket/tcp.lua:209: in function 'tls_handshake'
	.../local/apisix//deps/share/lua/5.1/resty/http_connect.lua:239: in function 'connect'
	/usr/local/apisix//deps/share/lua/5.1/resty/http.lua:927: in function 'request_uri'
	/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:72: in function 'http_request_uri'
	/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:146: in function '_request_uri'
	/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:493: in function 'readdir'
	/usr/local/apisix/apisix/core/config_etcd.lua:100: in function 'readdir'
	/usr/local/apisix/apisix/core/config_etcd.lua:296: in function 'sync_data'
	/usr/local/apisix/apisix/core/config_etcd.lua:556: in function </usr/local/apisix/apisix/core/config_etcd.lua:537>
	[C]: in function 'xpcall'
	/usr/local/apisix/apisix/core/config_etcd.lua:537: in function </usr/local/apisix/apisix/core/config_etcd.lua:516>,  etcd key: /apisix/proto, context: ngx.timer

when config etcd.tls.verify=false on apisix2.10.0 the error has gone，is this a bug，and etcd.tls.verify=true on apisix2.9 there is no such problem.

Environment

  • apisix version (cmd: apisix version): 2.10.0
  • OS (cmd: uname -a):
Linux ef2357fe80f7 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • OpenResty / Nginx version (cmd: nginx -V or openresty -V):
nginx version: openresty/1.19.3.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
built with OpenSSL 1.1.1l  24 Aug 2021
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=0.0.0 -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.19 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.9 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../mod_dubbo --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../ngx_multi_upstream_module --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../apisix-nginx-module --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../lua-var-nginx-module --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module
  • etcd version, if have (cmd: run curl http://127.0.0.1:9090/v1/server_info to get the info from server-info API): 3.5.0
  • apisix-dashboard version, if have: 2.8
  • the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
  • luarocks version, if the issue is about installation (cmd: luarocks --version):

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions