English Translated Version:
Current Behavior
There is a flaw in the metric expiration feature, which causes part of the list that has expired and been re-added to be lost when the metric is exported.
Steps to Reproduce
The https://github.com/api7/nginx-lua-prometheus/blob/release-0.20240201/prometheus.lua indicator processing library that the new version relies on has defects in processing expired metrics, resulting in the loss of some expired metrics when exporting metrics. Re-added list.
You can see the processing logic of the lookup_or_create method self.lookup will cache the full name mapping of the metric and will be reset when self.lookup_size >= self.lookup_max_size.
The metric is added for the first time. Since there is no cache in self.lookup, the mapping cache will be set and self._key_index:add(full_name, ERR_MSG_LRU_EVICTION, self.exptime) will be executed to save the key index relationship.
Because the expiration time is set, the value of self._key_index.keys[i] becomes null after a certain period of time.
When this indicator is added again, because it exists in self.lookup, the full name will be returned directly and self._key_index:add() will not be executed. At this time, self._key_index.keys has lost data.
The self._key_index:list() used when outputting indicators traverses the self._key_index.keys list.
So the metric is lost.
Environment
CHINESE
Current Behavior
指标过期特性存在缺陷，导致指标输出时会丢失部分已过期并重新添加的列表。
Expected Behavior
No response
Error Logs
No response
Steps to Reproduce
新版依赖的https://github.com/api7/nginx-lua-prometheus/blob/release-0.20240201/prometheus.lua指标处理库在处理已过期指标时存在缺陷，导致指标输出时会丢失部分已过期并重新添加的列表。
可以看lookup_or_create方法的处理逻辑
self.lookup会缓存指标的full name映射，当self.lookup_size >= self.lookup_max_size时会进行重置。
指标首次新增，因self.lookup中没有缓存，会设置映射缓存并执行self._key_index:add(full_name, ERR_MSG_LRU_EVICTION, self.exptime)，保存key索引关系
因为设置了过期时间，导致一定时间后self._key_index.keys[i]的值为null。
这个指标再次新增时，因为存在在self.lookup，直接返回full name，不会再执行self._key_index:add()，这个时候self._key_index.keys已经缺失了数据。
输出指标时使用的self._key_index:list(), 遍历了self._key_index.keys列表。
所以指标就丢失了。
Environment
English Translated Version:
Current Behavior
There is a flaw in the metric expiration feature, which causes part of the list that has expired and been re-added to be lost when the metric is exported.
Steps to Reproduce
The https://github.com/api7/nginx-lua-prometheus/blob/release-0.20240201/prometheus.lua indicator processing library that the new version relies on has defects in processing expired metrics, resulting in the loss of some expired metrics when exporting metrics. Re-added list.
You can see the processing logic of the
lookup_or_createmethodself.lookupwill cache the full name mapping of the metric and will be reset whenself.lookup_size >= self.lookup_max_size.The metric is added for the first time. Since there is no cache in
self.lookup, the mapping cache will be set andself._key_index:add(full_name, ERR_MSG_LRU_EVICTION, self.exptime)will be executed to save the key index relationship.Because the expiration time is set, the value of
self._key_index.keys[i]becomes null after a certain period of time.When this indicator is added again, because it exists in self.lookup, the full name will be returned directly and
self._key_index:add()will not be executed. At this time,self._key_index.keyshas lost data.The
self._key_index:list()used when outputting indicators traverses theself._key_index.keyslist.So the metric is lost.
Environment
CHINESE
Current Behavior
指标过期特性存在缺陷，导致指标输出时会丢失部分已过期并重新添加的列表。
Expected Behavior
No response
Error Logs
No response
Steps to Reproduce
新版依赖的https://github.com/api7/nginx-lua-prometheus/blob/release-0.20240201/prometheus.lua指标处理库在处理已过期指标时存在缺陷，导致指标输出时会丢失部分已过期并重新添加的列表。
可以看lookup_or_create方法的处理逻辑
self.lookup会缓存指标的full name映射，当self.lookup_size >= self.lookup_max_size时会进行重置。
指标首次新增，因self.lookup中没有缓存，会设置映射缓存并执行self._key_index:add(full_name, ERR_MSG_LRU_EVICTION, self.exptime)，保存key索引关系
因为设置了过期时间，导致一定时间后self._key_index.keys[i]的值为null。
这个指标再次新增时，因为存在在self.lookup，直接返回full name，不会再执行self._key_index:add()，这个时候self._key_index.keys已经缺失了数据。
输出指标时使用的self._key_index:list(), 遍历了self._key_index.keys列表。
所以指标就丢失了。
Environment