test: elasticsearch_query size not found #11124
Description
Relevant telegraf.conf
n/aLogs from Telegraf
2022/05/17 14:43:06 E! [] metric field 'size' not found on index 'test-elasticsearch_query-1652820179'
2022/05/17 14:43:06 D! [] {"query": {"bool":{"filter":[{"query_string":{"query":"response: 200"}},{"range":{"@timestamp":{"from":"2022-05-17T20:33:06.120173111Z","include_lower":true,"include_upper":true,"to":"2022-05-17T20:43:06.120173111Z"}}}]}}}
2022/05/17 14:43:06 D! [] {"query": {"bool":{"filter":[{"query_string":{"query":"response: 200"}},{"range":{"@timestamp":{"from":"2022-05-17T20:33:06.120516521Z","include_lower":true,"include_upper":true,"to":"2022-05-17T20:43:06.120516521Z"}}}]}}}
2022/05/17 14:43:06 D! [] {"query": {"bool":{"filter":[{"query_string":{"query":"product_2"}},{"range":{"@timestamp":{"from":"2022-05-17T20:33:06.146468039Z","include_lower":true,"include_upper":true,"to":"2022-05-17T20:43:06.146468039Z"}}}]}}}
--- FAIL: TestElasticsearchQuery (7.01s)
elasticsearch_query_test.go:610: [elasticsearch query aggregation measurement2: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement12: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement1: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement13: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement8: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement3: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' elasticsearch query aggregation measurement4: metric field 'size' not found on index 'test-elasticsearch_query-1652820179' ]
elasticsearch_query_test.go:617: []telegraf.Metric
--- expected
+++ actual
[]*testutil.metricDiff(Inverse(cmpopts.SortSlices, []*testutil.metricDiff{
- &{
- Measurement: "measurement1",
- Tags: []*telegraf.Tag{&{Key: "URI_keyword", Value: "/downloads/product_1"}},
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(263)},
- &{Key: "size_avg", Value: float64(202.30038022813687)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
&{
- Measurement: "measurement12",
+ Measurement: "measurement7",
Tags: nil,
Fields: []*telegraf.Field{
- &{Key: "size_avg", Value: float64(0)},
+ &{Key: "doc_count", Value: int64(0)},
},
Type: 3,
... // 1 ignored field
},
- &{
- Measurement: "measurement2",
- Tags: []*telegraf.Tag{&{Key: "URI_keyword", Value: "/downloads/product_1"}},
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(263)}, &{Key: "size_max", Value: float64(3301)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement2",
- Tags: []*telegraf.Tag{&{Key: "URI_keyword", Value: "/downloads/product_2"}},
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(237)}, &{Key: "size_max", Value: float64(3318)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement3",
- Tags: []*telegraf.Tag{&{Key: "response_keyword", Value: "200"}},
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(22)}, &{Key: "size_sum", Value: float64(22790)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement3",
- Tags: []*telegraf.Tag{&{Key: "response_keyword", Value: "304"}},
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(219)}, &{Key: "size_sum", Value: float64(0)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement3",
- Tags: []*telegraf.Tag{&{Key: "response_keyword", Value: "404"}},
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(259)},
- &{Key: "size_sum", Value: float64(86932)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_1"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(3)},
- &{Key: "response_time_min", Value: float64(1514)},
- &{Key: "size_min", Value: float64(490)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_1"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "304"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(113)},
- &{Key: "response_time_min", Value: float64(71)},
- &{Key: "size_min", Value: float64(0)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_1"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "404"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(146)},
- &{Key: "response_time_min", Value: float64(126)},
- &{Key: "size_min", Value: float64(318)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_1"},
- &{Key: "method_keyword", Value: "HEAD"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(1)},
- &{Key: "response_time_min", Value: float64(8479)},
- &{Key: "size_min", Value: float64(0)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_2"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(13)},
- &{Key: "response_time_min", Value: float64(2)},
- &{Key: "size_min", Value: float64(490)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_2"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "304"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(106)},
- &{Key: "response_time_min", Value: float64(134)},
- &{Key: "size_min", Value: float64(0)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_2"},
- &{Key: "method_keyword", Value: "GET"},
- &{Key: "response_keyword", Value: "404"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(113)},
- &{Key: "response_time_min", Value: float64(237)},
- &{Key: "size_min", Value: float64(318)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement4",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_2"},
- &{Key: "method_keyword", Value: "HEAD"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{
- &{Key: "doc_count", Value: int64(5)},
- &{Key: "response_time_min", Value: float64(1059)},
- &{Key: "size_min", Value: float64(0)},
- },
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement5",
- Tags: []*telegraf.Tag{&{Key: "URI_keyword", Value: "/downloads/product_2"}},
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(237)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement6",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_1"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(4)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement6",
- Tags: []*telegraf.Tag{
- &{Key: "URI_keyword", Value: "/downloads/product_2"},
- &{Key: "response_keyword", Value: "200"},
- },
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(18)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement7",
- Fields: []*telegraf.Field{&{Key: "doc_count", Value: int64(22)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
- &{
- Measurement: "measurement8",
- Fields: []*telegraf.Field{&{Key: "size_max", Value: float64(3318)}},
- Type: 3,
- Time: s"2018-06-14 05:51:53.266176036 +0000 UTC",
- },
}))
--- FAIL: TestElasticsearchQuery_getMetricFields (0.04s)
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_1 (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_2 (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_3 (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_4 (0.01s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "response_time": "long",
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_8 (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_9_-_invalid_function (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_12_-_non-existing_timestamp_field (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
--- FAIL: TestElasticsearchQuery_getMetricFields/getMetricFields_query_13_-_non-existing_tag_field (0.00s)
elasticsearch_query_test.go:687: ElasticsearchQuery.getMetricFields() = error = map[string]string{
+ "size": "long",
}
FAIL
FAIL github.com/influxdata/telegraf/plugins/inputs/elasticsearch_query 7.136s
System info
master
Docker
No response
Steps to reproduce
docker-compose upgo test -count=1 -race ./plugins/inputs/elasticsearch_query/...
Expected behavior
test pass
Actual behavior
TestElasticsearchQuery_getMetricFields returns a different response than expected.
This change updated the image from elasticsearch 5 to 7. That prevented these tests from even running since there is a hard-check for versions 5 or 6. My recent change to go back to version 6 started showing these aggregation issues. Going back to elasticsearch version 5 resolves the issue. However, elasticsearch 5 is not even supported anymore.
next steps: revert to 5 and verify tests?
Additional info
No response