Skip to content

NATS JetStream scaler incorrectly scales to maxReplicaCount when the specified consumer does not exist #7657

Description

@robert-blackman

Report

Hey there 👋 as the title suggests I'm seeing behaviour where the Jetstream scaler goes to maxReplicaCount when a consumer doesn't exist. I suspect this behaviour is a red-herring/symptom, and that the actual problem exists in the scalers getMaxMsgLag() function.

This line:

return s.stream.State.LastSequence

If the consumer doesn't exist, getMaxMsgLag() returns the LastSequence. Which might describe the behaviour I'm seeing.

Expected Behavior

When the consumer name configured in the nats-jetstream trigger does not exist in the specified stream, the scaler should either:

  • Return an error (so fallbacks can kick in if they are defined)
  • or; treat the result as 0 - the consumer doesn't exist if there wasn't a failure in listing the consumers - 0 seems correct.

Actual Behavior

The scaler reports some large number which causes the computed lag to exceed any given threshold - which then scales the workload to max replicas.

Steps to Reproduce the Problem

  1. Create a NATS JetStream stream and have some messages sent through.
  2. Do not create the consumer named in the ScaledObject.
  3. Deploy a ScaledObject using the nats-jetstream scaler with a reasonable lagThreshold (1 for example). 3.
  4. Observe the deployment scaling up (even with no messages or activity).

Logs from KEDA operator

I haven't included any logs as the issue seems to be in the scaler code from a quick look. Happy to do so should more info be required 👍

KEDA Version

2.19.0

Kubernetes Version

Other

Platform

Amazon Web Services

Scaler Details

NATS Jetstream

Would you be open to contributing a fix?

Yes

Anything else?

I've included my suspicions for the root cause in the Report section - happy to attempt a fix, but would like guidance on the expected behaviours.

Metadata

Metadata

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Ready To Ship

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions