Skip to content

bug: ElasticSearch logger not working with error Action/metadata line [1] contains an unknown parameter [_type] #8647

Description

@gaoxingliang

Current Behavior

The plugin eleasticsearch-logger is not working with elasticseachr-8.x and apisix - 3.0.0
and in apisix it has below error log:

2023/01/10 07:51:51 [error] 48#48: *231989 [lua] batch-processor.lua:95: Batch Processor[elasticsearch-logger] failed to process entries: elasticsearch server returned status: 400,
 body: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception",
"reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}, context: ngx.timer, client: , server: 0.0.0.0:9080

the plugin conf is referenced to the doc url:
https://apisix.apache.org/docs/apisix/plugins/elasticsearch-logger/
and my conf is:

{
  "auth": {
    "password": "changeme",
    "username": "elastic"
  },
  "batch_max_size": 1,
  "buffer_duration": 60,
  "disable": false,
  "endpoint_addr": "http://xxxx:9200",
  "field": {
    "index": "services",
    "type": "collector"
  },
  "inactive_timeout": 5,
  "max_retry_count": 10,
  "name": "elasticsearch-logger",
  "retry_delay": 1,
  "ssl_verify": false,
  "timeout": 60
}

I also tried with es 6.0 and same conf as above.
it shows error:

{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: an id must be provided if version type or value are set;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: an id must be provided if version type or value are set;"},"status":400}, context: ngx.timer

Updates:

It seems the _type is deprecated in es: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/mapping-type-field.html#mapping-type-field
After removed the type in the conf. It worked.

{
  "auth": {
    "password": "changeme",
    "username": "elastic"
  },
  "batch_max_size": 1,
  "buffer_duration": 60,
  "disable": false,
  "endpoint_addr": "http://xxxx:9200",
  "field": {
    "index": "services"
  },
  "inactive_timeout": 5,
  "max_retry_count": 10,
  "name": "elasticsearch-logger",
  "retry_delay": 1,
  "ssl_verify": false,
  "timeout": 60
}

Expected Behavior

The elasticsearch logger plugin should work with es 8.x version.

Error Logs

See above.

Steps to Reproduce

  1. create a elk cluster using https://github.com/deviantony/docker-elk#bringing-up-the-stack
  2. add the elasticsearch logger plugin
  3. check the apisix logs.

Environment

  • APISIX version 3.0.0
  • Operating system: centos7

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions