With JMX Insights, we have the ability to capture JMX metrics using a simple YAML configuration file.
As found while working on #14067 , some JMX MBeans have some technical details leak into their object names, which forces to capture extra "technical metric attributes" that are not relevant for the metric consumer.
MBeans are published in a tree-like structure with optional key/value pairs, the metric values are extracted from their MBean attributes.
JMX metrics are extracted from those using one or more MBean query that can have wildcards for those key/value pairs
For example, with the Jetty application server, when we need to capture the "select count" metric:
org.eclipse.jetty.io:context=*,type=managedselector,id=* MBean query allows to get all the MBean instances providing the selectCount MBean attribute- by default 10 instances of such MBean exist, each with a distinct value of
id key.
- the current JMX implementation will attempt to capture all of those as 10 distinct metrics with the same identity unless we map the
id key value as a metric attribute, hence producing 10 distinct metrics.
- we need to provide the aggregate sum of
selectCount attribute over all those instances and capture it as a single metric
- in this particular case, in addition the
context key also needs to be discarded because there is often a single transient value (thus potentially high-cardinality over time).
Here is how it looks like with VisualVM:
The context key value is equal to HTTP_1_1@524d6d96, and the id key have 10 distinct values from 0 to 9.
Having the ability to discard some metrics attributes and perform relevant aggregation seems to be possible through the usage of metric views, so I think we should be able to leverage this feature and apply it for JMX metrics.
For the distinct values of id having those as metric attributes is not relevant for the metric consumer.
For the context key, we can consider this as an exception (but one we have to get by) as it goes against the recommendations for MBean names, in particular for this statement:
The name of the same object should not change between different executions of the application, so for example it should not contain items like a "JVM id" or the "MBean Server Id" that will change with each execution
With JMX Insights, we have the ability to capture JMX metrics using a simple YAML configuration file.
As found while working on #14067 , some JMX MBeans have some technical details leak into their object names, which forces to capture extra "technical metric attributes" that are not relevant for the metric consumer.
MBeans are published in a tree-like structure with optional key/value pairs, the metric values are extracted from their MBean attributes.
JMX metrics are extracted from those using one or more MBean query that can have wildcards for those key/value pairs
For example, with the Jetty application server, when we need to capture the "select count" metric:
org.eclipse.jetty.io:context=*,type=managedselector,id=*MBean query allows to get all the MBean instances providing theselectCountMBean attributeidkey.idkey value as a metric attribute, hence producing 10 distinct metrics.selectCountattribute over all those instances and capture it as a single metriccontextkey also needs to be discarded because there is often a single transient value (thus potentially high-cardinality over time).Here is how it looks like with VisualVM:
The
contextkey value is equal toHTTP_1_1@524d6d96, and theidkey have 10 distinct values from0to9.Having the ability to discard some metrics attributes and perform relevant aggregation seems to be possible through the usage of metric views, so I think we should be able to leverage this feature and apply it for JMX metrics.
For the distinct values of
idhaving those as metric attributes is not relevant for the metric consumer.For the
contextkey, we can consider this as an exception (but one we have to get by) as it goes against the recommendations for MBean names, in particular for this statement: