Skip to content

Introduce HASH items expiration#2089

Merged
ranshid merged 3 commits intovalkey-io:unstablefrom
ranshid:ttl-poc-new
Aug 5, 2025
Merged

Introduce HASH items expiration#2089
ranshid merged 3 commits intovalkey-io:unstablefrom
ranshid:ttl-poc-new

Conversation

@ranshid
Copy link
Member

@ranshid ranshid commented May 15, 2025

Closes #640

Summary

This PR introduces support for field-level expiration in Valkey hash types, making it possible for individual fields inside a hash to expire independently — creating what we call volatile fields.
This is just the first out of 3 PRs. The content of this PR focus on enabling the basic ability to set and modify hash fields expiration as well as persistency (AOF+RDB) and defrag.
The second PR introduces the new algorithm (volatile-set) to track volatile hash fields is in the last stages of review. The current implementation in this PR (in volatile-set.h/c) is just s tub implementation and will be replaced by The second PR
The third PR which introduces the active expiration and defragmentation jobs.

For more highlevel design details you can track the RFC PR: valkey-io/valkey-rfc#22.


Major decisions

Some highlevel major decisions which are taken as part of this work:

  1. We decided to copy the existing Redis API in order to maintain compatibility with existing clients.

  2. We decided to avoid introducing lazy-expiration at this point, in order to reduce complexity and rely only on active-expiration for memory reclamation. This will require us to continue to work on improving the active expiration job and potentially consider introduce lazy-expiration support later on.

  3. Although different commands which are adding expiration on hash fields are influencing the memory utilization (by allocating more memory for expiration time and metadata) we decided to avoid adding the DENYOOM for these commands (an exception is HSETEX) in order to be better aligned with highlevel keys commands like expire

  4. Some hash type commands will produce unexpected results:

  • HLEN - will still reflect the number of fields which exists in the hash object (either actually expired or not).
  • HRANDFIELD - in some cases we will not be able to randomly select a field which was not already expired. this case happen in 2 cases: 1/ when we are asked to provide a non-uniq fields (i.e negative count) 2/ when the size of the hash is much bigger than the count and we need to provide uniq results. In both cases it is possible that an empty response will be returned to the caller, even in case there are fields in the hash which are either persistent or not expired.
  1. For the case were a field is provided with a zero (0) expiration time or expiration time in the past, it is immediately deleted. We decided that, in order to be aligned with how high level keys are handled, we will emit hexpired keyspace event for that case (instead of hdel). For example:
    for the case:
HSET myhash f1 v1
> 0
HGETEX myhash EX 0 FIELDS 1 f1
> "v1"
HTTL myhash FIELDS 1 f1
>  -2

The reported events are:

1) "psubscribe"
2) "__keyevent@0__*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hset"
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hexpired" <---------------- note this
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:del"
4) "myhash"
  1. We will ALWAYS load hash fields during rdb load. This means that when primary is rebooting with an old snapshot, it will take time to reclaim all the expired fields. However this simplifies the current logic and avoid major refactoring that I suspect will be needed.

New entry type

This PR also modularizes and exposes the internal hashTypeEntry logic as a new standalone entry.c/h module. This new abstraction handles all aspects of field–value–expiry encoding using multiple memory layouts optimized for performance and memory efficiency.

An entry is an abstraction that represents a single field–value pair with optional expiration. Internally, Valkey uses different memory layouts for compactness and efficiency, chosen dynamically based on size and encoding constraints.

The entry pointer is the field sds. Which make us use an entry just like any sds. We encode the entry layout type
in the field SDS header. Field type SDS_TYPE_5 doesn't have any spare bits to
encode this so we use it only for the first layout type.

Entry with embedded value, used for small sizes. The value is stored as
SDS_TYPE_8. The field can use any SDS type.

Entry can also have expiration timestamp, which is the UNIX timestamp for it to be expired.
For aligned fast access, we keep the expiry timestamp prior to the start of the sds header.

 +----------------+--------------+---------------+
 | Expiration     | field        | value         |
 | 1234567890LL   | hdr "foo" \0 | hdr8 "bar" \0 |
 +-----------------------^-------+---------------+
                         |
                         |
                        entry pointer (points to field sds content)

Entry with value pointer, used for larger fields and values. The field is SDS
type 8 or higher.

 +--------------+-------+--------------+
 | Expiration   | value | field        |
 | 1234567890LL | ptr   | hdr "foo" \0 |
 +--------------+--^----+------^-------+
                   |           |
                   |           |
                   |         entry pointer (points to field sds content)
                   |
                  value pointer = value sds

The entry.c/h API provides methods to:

  • Create, read, and write and Update field/value/expiration
  • Set or clear expiration
  • Check expiration state
  • Clone or delete an entry

Supported Commands

This PR introduces new commands and extends existing ones to support field expiration:

Commands

The proposed API is very much identical to the Redis provided API (Redis 7.4 + 8.0). This is intentionally proposed in order to avoid breaking client applications already opted to use hash items TTL.

HSETEX

Synopsis

HSETEX key [NX | XX] [FNX | FXX] [EX seconds | PX milliseconds |
  EXAT unix-time-seconds | PXAT unix-time-milliseconds | KEEPTTL]
  FIELDS numfields field value [field value ...]

Set the value of one or more fields of a given hash key, and optionally set their expiration time or time-to-live (TTL).

The HSETEX command supports the following set of options:

  • NX — Only set the fields if the hash object does NOT exist.
  • XX — Only set the fields if if the hash object doesx exist.
  • FNX — Only set the fields if none of them already exist.
  • FXX — Only set the fields if all of them already exist.
  • EX seconds — Set the specified expiration time in seconds.
  • PX milliseconds — Set the specified expiration time in milliseconds.
  • EXAT unix-time-seconds — Set the specified Unix time in seconds at which the fields will expire.
  • PXAT unix-time-milliseconds — Set the specified Unix time in milliseconds at which the fields will expire.
  • KEEPTTL — Retain the TTL associated with the fields.

The EX, PX, EXAT, PXAT, and KEEPTTL options are mutually exclusive.

HEGTEX

Synopsis

HGETEX key [EX seconds | PX milliseconds | EXAT unix-time-seconds |
  PXAT unix-time-milliseconds | PERSIST] FIELDS numfields field
  [field ...]

Get the value of one or more fields of a given hash key and optionally set their expiration time or time-to-live (TTL).

The HGETEX command supports a set of options:

  • EX seconds — Set the specified expiration time, in seconds.
  • PX milliseconds — Set the specified expiration time, in milliseconds.
  • EXAT unix-time-seconds — Set the specified Unix time at which the fields will expire, in seconds.
  • PXAT unix-time-milliseconds — Set the specified Unix time at which the fields will expire, in milliseconds.
  • PERSIST — Remove the TTL associated with the fields.

The EX, PX, EXAT, PXAT, and PERSIST options are mutually exclusive.

HEXPIRE

Synopsis

HEXPIRE key seconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]

Set an expiration (TTL or time to live) on one or more fields of a given hash key. You must specify at least one field. Field(s) will automatically be deleted from the hash key when their TTLs expire.
Field expirations will only be cleared by commands that delete or overwrite the contents of the hash fields, including HDEL and HSET commands. This means that all the operations that conceptually alter the value stored at a hash key's field without replacing it with a new one will leave the TTL untouched.
You can clear the TTL of a specific field by specifying 0 for the ‘seconds’ argument.
Note that calling HEXPIRE/HPEXPIRE with a time in the past will result in the hash field being deleted immediately.

The HEXPIRE command supports a set of options:

  • NX — For each specified field, set expiration only when the field has no expiration.
  • XX — For each specified field, set expiration only when the field has an existing expiration.
  • GT — For each specified field, set expiration only when the new expiration is greater than current one.
  • LT — For each specified field, set expiration only when the new expiration is less than current one.

HEXPIREAT

Synopsis

HEXPIREAT key unix-time-seconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]

HEXPIREAT has the same effect and semantics as HEXPIRE, but instead of specifying the number of seconds for the TTL (time to live), it takes an absolute Unix timestamp in seconds since Unix epoch. A timestamp in the past will delete the field immediately.

The HEXPIREAT command supports a set of options:

  • NX — For each specified field, set expiration only when the field has no expiration.
  • XX — For each specified field, set expiration only when the field has an existing expiration.
  • GT — For each specified field, set expiration only when the new expiration is greater than current one.
  • LT — For each specified field, set expiration only when the new expiration is less than current one.

HPEXPIRE

Synopsis

HPEXPIRE key milliseconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]

This command works like HEXPIRE, but the expiration of a field is specified in milliseconds instead of seconds.

The HPEXPIRE command supports a set of options:

  • NX — For each specified field, set expiration only when the field has no expiration.
  • XX — For each specified field, set expiration only when the field has an existing expiration.
  • GT — For each specified field, set expiration only when the new expiration is greater than current one.
  • LT — For each specified field, set expiration only when the new expiration is less than current one.

HPEXPIREAT

Synopsis

HPEXPIREAT key unix-time-milliseconds [NX | XX | GT | LT]
  FIELDS numfields field [field ...]

HPEXPIREAT has the same effect and semantics as HEXPIREAT``, but the Unix time at which the field will expire is specified in milliseconds since Unix epoch instead of seconds.

HPERSIST

Synopsis

HPERSIST key FIELDS numfields field [field ...]

Remove the existing expiration on a hash key's field(s), turning the field(s) from volatile (a field with expiration set) to persistent (a field that will never expire as no TTL (time to live) is associated).

HSETEX

Synopsis

HSETEX key [NX] seconds field value [field value ...]

Similar to HSET but adds one or more hash fields that expire after specified number of seconds. By default, this command overwrites the values and expirations of specified fields that exist in the hash. If NX option is specified, the field data will not be overwritten. If key doesn't exist, a new Hash key is created.

The HSETEX command supports a set of options:

  • NX — For each specified field, set expiration only when the field has no expiration.

HTTL

Synopsis

HTTL key FIELDS numfields field [field ...]

Returns the remaining TTL (time to live) of a hash key's field(s) that have a set expiration. This introspection capability allows you to check how many seconds a given hash field will continue to be part of the hash key.

HPTTL

HPTTL key FIELDS numfields field [field ...]

Like HTTL, this command returns the remaining TTL (time to live) of a field that has an expiration set, but in milliseconds instead of seconds.

HEXPIRETIME

Synopsis

HEXPIRETIME key FIELDS numfields field [field ...]

Returns the absolute Unix timestamp in seconds since Unix epoch at which the given key's field(s) will expire.

HPEXPIRETIME

Synopsis

HPEXPIRETIME key FIELDS numfields field [field ...]

HPEXPIRETIME has the same semantics as HEXPIRETIME, but returns the absolute Unix expiration timestamp in milliseconds since Unix epoch instead of seconds.

Keyspace Notifications

This PR introduces new notification events to support field-level expiration:

Event Trigger
hexpire Field expiration was set
hexpired Field was deleted due to expiration
hpersist Expiration was removed from a field
del Key was deleted after all fields expired

Note that we diverge from Redis in the cases we emit hexpired event.
For example:
given the following usecase:

HSET myhash f1 v1
(integer) 0
HGETEX myhash EX 0 FIELDS 1 f1
1) "v1"
 HTTL myhash FIELDS 1 f1
1) (integer) -2

regarding the keyspace-notifications:
Redis reports:

1) "psubscribe"
2) "__keyevent@0__:*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:hset"
4) "myhash2"
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:hdel" <---------------- note this
4) "myhash2"
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:del"
4) "myhash2"

However In our current suggestion, Valkey will emit:

1) "psubscribe"
2) "__keyevent@0__*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hset"
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hexpired" <---------------- note this
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:del"
4) "myhash"

Propagation and Replication

  • Expiration-aware commands (HSETEX, HGETEX, etc.) are not propagated as-is.
  • Instead, Valkey rewrites them into equivalent commands like:
    • HDEL (for expired fields)
    • HPEXPIREAT (for setting absolute expiration)
    • HPERSIST (for removing expiration)

This ensures compatibility with replication and AOF while maintaining consistent field-level expiry behavior.


Performance Comparison

Command Name QPS Standard QPS HFE QPS Diff % Latency Standard (ms) Latency HFE (ms) Latency Diff %
One Large Hash Table
HGET 137988.12 138484.97 +0.36% 0.951 0.949 -0.21%
HSET 138561.73 137343.77 -0.87% 0.948 0.956 +0.84%
HEXISTS 139431.12 138677.02 -0.54% 0.942 0.946 +0.42%
HDEL 140114.89 138966.09 -0.81% 0.938 0.945 +0.74%
Many Hash Tables (100 fields)
HGET 136798.91 137419.27 +0.45% 0.959 0.956 -0.31%
HEXISTS 138946.78 139645.31 +0.50% 0.946 0.941 -0.52%
HGETALL 42194.09 42016.80 -0.42% 0.621 0.625 +0.64%
HSET 137230.69 137249.53 +0.01% 0.959 0.958 -0.10%
HDEL 138985.41 138619.34 -0.26% 0.948 0.949 +0.10%
Many Hash Tables (1000 fields)
HGET 135795.77 139256.36 +2.54% 0.965 0.943 -2.27%
HEXISTS 138121.55 137950.06 -0.12% 0.951 0.952 +0.10%
HGETALL 5885.81 5633.80 -4.28% 2.690 2.841 +5.61%
HSET 137005.08 137400.39 +0.28% 0.959 0.955 -0.41%
HDEL 138293.45 137381.52 -0.65% 0.948 0.955 +0.73%

Accumulated Backlog

[ ] Consider extending HSETEX with extra arguments: NX/XX so that it is possible to prevent adding/setting/mutating fields of a non-existent hash
[ ] Avoid loading expired fields when non-preamble RDB is being loaded on primary. This is an optimization in order to reduce loading unnecessary fields (which are expired). This would also require us to propagate the HDEL to the replicas in case of RDBFLAGS_FEED_REPL. Note that it might have to require some refactoring:
1/ propagate the rdbflags and current time to rdbLoadObject. 2/ consider the case of restore and check_rdb etc...
For this reason I would like to avoid this optimizationfor the first drop.

@codecov
Copy link

codecov bot commented May 18, 2025

Codecov Report

❌ Patch coverage is 76.09302% with 514 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.37%. Comparing base (dceb9f3) to head (4319edb).
⚠️ Report is 11 commits behind head on unstable.

Files with missing lines Patch % Lines
src/vset.c 51.27% 439 Missing ⚠️
src/t_hash.c 94.88% 32 Missing ⚠️
src/aof.c 26.08% 17 Missing ⚠️
src/entry.c 95.40% 8 Missing ⚠️
src/expire.c 95.27% 6 Missing ⚠️
src/module.c 0.00% 5 Missing ⚠️
src/rdb.c 85.18% 4 Missing ⚠️
src/defrag.c 80.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #2089      +/-   ##
============================================
- Coverage     71.49%   71.37%   -0.13%     
============================================
  Files           123      125       +2     
  Lines         67487    69207    +1720     
============================================
+ Hits          48251    49395    +1144     
- Misses        19236    19812     +576     
Files with missing lines Coverage Δ
src/anet.c 72.44% <ø> (ø)
src/commands.def 100.00% <ø> (ø)
src/db.c 90.47% <100.00%> (+0.47%) ⬆️
src/hashtable.c 82.71% <100.00%> (+0.28%) ⬆️
src/lazyfree.c 86.39% <100.00%> (+0.28%) ⬆️
src/object.c 81.86% <100.00%> (+0.43%) ⬆️
src/server.c 88.40% <100.00%> (+0.33%) ⬆️
src/server.h 100.00% <ø> (ø)
src/t_string.c 96.34% <100.00%> (-0.50%) ⬇️
src/util.c 66.21% <100.00%> (+0.41%) ⬆️
... and 9 more

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xbasel xbasel added the run-extra-tests Run extra tests on this PR (Runs all tests from daily except valgrind and RESP) label May 25, 2025
@ranshid ranshid marked this pull request as ready for review June 4, 2025 11:37
Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a partial pass on this. I got to the hashtable callback and the entry abstraction. I didn't get to the actual field expiration logic in t_hash and the volatile set though. Need to continue another day.

@ranshid ranshid force-pushed the ttl-poc-new branch 2 times, most recently from 65eeb1d to 8ecd584 Compare June 18, 2025 15:12
Copy link
Contributor

@rainsupreme rainsupreme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a lot of work you've done! I've only had time for a partial review today, but I had a few comments/questions so far. The command schema and entry memory layout looks good to me. It'll be interesting to see perf testing too! 😀

@ranshid
Copy link
Member Author

ranshid commented Jun 19, 2025

This is a lot of work you've done! I've only had time for a partial review today, but I had a few comments/questions so far. The command schema and entry memory layout looks good to me. It'll be interesting to see perf testing too! 😀

We are just more focused on introducing the functionality and would focus on performance testing as soon as possible.

src/entry.c Outdated
zfree(entryAllocPtr(entry));
}

/* Takes ownership of value, does not take ownership of field */
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will just remove that logic for now. it was meant for sets, but I am not sure it will remain that way.

@JimB123
Copy link
Member

JimB123 commented Jun 24, 2025

First comment - 37 changed files??? Dang!

Copy link
Member

@JimB123 JimB123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still reviewing. Posting Day 1. 😨

@ranshid
Copy link
Member Author

ranshid commented Jun 25, 2025

First comment - 37 changed files??? Dang!

you still have another PR in the oven - it might be bigger than this :(

@ranshid ranshid requested a review from rjd15372 June 30, 2025 15:14
Copy link
Member

@rjd15372 rjd15372 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ranshid , this is just the review of the entry.c code that you can check while I continue to review the rest of the code.

@ranshid
Copy link
Member Author

ranshid commented Jul 1, 2025

Hi @ranshid , this is just the review of the entry.c code that you can check while I continue to review the rest of the code.

Thank you @rjd15372 !

TBH the entry is NOT the main focus of this PR. most of the entry code is taken from the already existing implementation of hashTypeEntry (with indeed some changes).

I think the really interesting part are the new commands themselves. this is were the complex logic is introduced (HSETEX, HGETEX, HEXPIRE etc...)

there is also the new volatile set API in the t_hash.c (that I do not like that much) but we can focus on this in the PR introducing the volatile set.

ranshid added a commit that referenced this pull request Aug 5, 2025
This is needed due to changes presented in
#2089

---------

Signed-off-by: Ran Shidlansik <[email protected]>
@enjoy-binbin
Copy link
Member

that will be great if we next time, when doing the rebase merge, we squash the PR number like #2089 in the commit message title. (I usually locate the PR web page based on the commit message title)

@ranshid ranshid added release-notes This issue should get a line item in the release notes client-changes-needed Client changes may be required for this feature labels Aug 6, 2025
ranshid added a commit that referenced this pull request Aug 11, 2025
Following new API presented in
#2089, we might access out of
bound memory in case of some illegal command input

Signed-off-by: Ran Shidlansik <[email protected]>
allenss-amazon pushed a commit to allenss-amazon/valkey-core that referenced this pull request Aug 19, 2025
This is needed due to changes presented in
valkey-io#2089

---------

Signed-off-by: Ran Shidlansik <[email protected]>
allenss-amazon pushed a commit to allenss-amazon/valkey-core that referenced this pull request Aug 19, 2025
…y-io#2464)

Following new API presented in
valkey-io#2089, we might access out of
bound memory in case of some illegal command input

Signed-off-by: Ran Shidlansik <[email protected]>
zuiderkwast added a commit to valkey-io/valkey-doc that referenced this pull request Sep 5, 2025
related to: valkey-io/valkey#2089

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Viktor Söderqvist <[email protected]>
Co-authored-by: Josh Soref <[email protected]>
Co-authored-by: Madelyn Olson <[email protected]>
hpatro pushed a commit to hpatro/valkey that referenced this pull request Oct 3, 2025
…y-io#2464)

Following new API presented in
valkey-io#2089, we might access out of
bound memory in case of some illegal command input

Signed-off-by: Ran Shidlansik <[email protected]>
Signed-off-by: Harkrishn Patro <[email protected]>
ranshid added a commit that referenced this pull request Dec 24, 2025
In #2089 we added a deferred
logic for HGETALL since we cannot anticipate the size of the output as
it may contain expired hash items which should not be included.
As part of the work of #2022
this would greatly increase the time for HGETALL processing, thus we
introduce this minor improvement to avoid using deferred reply in case
the hash has NO volatile items.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
jdheyburn pushed a commit to jdheyburn/valkey that referenced this pull request Jan 8, 2026
In valkey-io#2089 we added a deferred
logic for HGETALL since we cannot anticipate the size of the output as
it may contain expired hash items which should not be included.
As part of the work of valkey-io#2022
this would greatly increase the time for HGETALL processing, thus we
introduce this minor improvement to avoid using deferred reply in case
the hash has NO volatile items.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
ranshid added a commit that referenced this pull request Jan 29, 2026
In the original implementation of Hash Field Expiration
(#2089), the HSETEX command was
implemented to report keyspace notifications only for performed changes.
This is mostly aligned with other Hash commands (for example, HDEL will
also not report `hdel` event for items which does not exist)
The HSETEX case is somewhat different and is more like the `HSET` case.
During HSETEX, after the command validations pass, items are ALWAYS
"added" to the object, even though they might not actually be added.
This case is the same for when the hash object is empty or when all the
provided fields do not exist in the object (as reported
[here](#2998))

This PR changes the way `HSETEX` will report keyspace notifications so
that:
1. `hset` notification will ALWAYS be reported if all command
validations pass.
2. `hexpire` will be reported in case the command include an expiration
time (even past time)
3. `hxpired` will be reported in case the provided expiration time is in
the past (or 0)
4. `hdel` will be reported in case the hash exists (or created as part
of the command) and following the command execution it was left empty.
5. we will always return '1' as a return value of tHSETEX command which
passed all validations. Before that we returned 1 only if we applied the
change cross ALL the input fields, so in case some of them did not exist
and a past time was set we would return 0.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Jacob Murphy <[email protected]>
ranshid added a commit to ranshid/valkey that referenced this pull request Jan 29, 2026
…-io#3001)

In the original implementation of Hash Field Expiration
(valkey-io#2089), the HSETEX command was
implemented to report keyspace notifications only for performed changes.
This is mostly aligned with other Hash commands (for example, HDEL will
also not report `hdel` event for items which does not exist)
The HSETEX case is somewhat different and is more like the `HSET` case.
During HSETEX, after the command validations pass, items are ALWAYS
"added" to the object, even though they might not actually be added.
This case is the same for when the hash object is empty or when all the
provided fields do not exist in the object (as reported
[here](valkey-io#2998))

This PR changes the way `HSETEX` will report keyspace notifications so
that:
1. `hset` notification will ALWAYS be reported if all command
validations pass.
2. `hexpire` will be reported in case the command include an expiration
time (even past time)
3. `hxpired` will be reported in case the provided expiration time is in
the past (or 0)
4. `hdel` will be reported in case the hash exists (or created as part
of the command) and following the command execution it was left empty.
5. we will always return '1' as a return value of tHSETEX command which
passed all validations. Before that we returned 1 only if we applied the
change cross ALL the input fields, so in case some of them did not exist
and a past time was set we would return 0.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Jacob Murphy <[email protected]>
ranshid added a commit that referenced this pull request Jan 29, 2026
In the original implementation of Hash Field Expiration
(#2089), the HSETEX command was
implemented to report keyspace notifications only for performed changes.
This is mostly aligned with other Hash commands (for example, HDEL will
also not report `hdel` event for items which does not exist)
The HSETEX case is somewhat different and is more like the `HSET` case.
During HSETEX, after the command validations pass, items are ALWAYS
"added" to the object, even though they might not actually be added.
This case is the same for when the hash object is empty or when all the
provided fields do not exist in the object (as reported
[here](#2998))

This PR changes the way `HSETEX` will report keyspace notifications so
that:
1. `hset` notification will ALWAYS be reported if all command
validations pass.
2. `hexpire` will be reported in case the command include an expiration
time (even past time)
3. `hxpired` will be reported in case the provided expiration time is in
the past (or 0)
4. `hdel` will be reported in case the hash exists (or created as part
of the command) and following the command execution it was left empty.
5. we will always return '1' as a return value of tHSETEX command which
passed all validations. Before that we returned 1 only if we applied the
change cross ALL the input fields, so in case some of them did not exist
and a past time was set we would return 0.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Jacob Murphy <[email protected]>
Signed-off-by: Ran Shidlansik <[email protected]>
zuiderkwast pushed a commit that referenced this pull request Jan 30, 2026
In the original implementation of Hash Field Expiration
(#2089), the HSETEX command was
implemented to report keyspace notifications only for performed changes.
This is mostly aligned with other Hash commands (for example, HDEL will
also not report `hdel` event for items which does not exist)
The HSETEX case is somewhat different and is more like the `HSET` case.
During HSETEX, after the command validations pass, items are ALWAYS
"added" to the object, even though they might not actually be added.
This case is the same for when the hash object is empty or when all the
provided fields do not exist in the object (as reported
[here](#2998))

This PR changes the way `HSETEX` will report keyspace notifications so
that:
1. `hset` notification will ALWAYS be reported if all command
validations pass.
2. `hexpire` will be reported in case the command include an expiration
time (even past time)
3. `hxpired` will be reported in case the provided expiration time is in
the past (or 0)
4. `hdel` will be reported in case the hash exists (or created as part
of the command) and following the command execution it was left empty.
5. we will always return '1' as a return value of tHSETEX command which
passed all validations. Before that we returned 1 only if we applied the
change cross ALL the input fields, so in case some of them did not exist
and a past time was set we would return 0.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Jacob Murphy <[email protected]>
Signed-off-by: Ran Shidlansik <[email protected]>
harrylin98 pushed a commit to harrylin98/valkey_forked that referenced this pull request Feb 19, 2026
…-io#3001)

In the original implementation of Hash Field Expiration
(valkey-io#2089), the HSETEX command was
implemented to report keyspace notifications only for performed changes.
This is mostly aligned with other Hash commands (for example, HDEL will
also not report `hdel` event for items which does not exist)
The HSETEX case is somewhat different and is more like the `HSET` case.
During HSETEX, after the command validations pass, items are ALWAYS
"added" to the object, even though they might not actually be added.
This case is the same for when the hash object is empty or when all the
provided fields do not exist in the object (as reported
[here](valkey-io#2998))

This PR changes the way `HSETEX` will report keyspace notifications so
that:
1. `hset` notification will ALWAYS be reported if all command
validations pass.
2. `hexpire` will be reported in case the command include an expiration
time (even past time)
3. `hxpired` will be reported in case the provided expiration time is in
the past (or 0)
4. `hdel` will be reported in case the hash exists (or created as part
of the command) and following the command execution it was left empty.
5. we will always return '1' as a return value of tHSETEX command which
passed all validations. Before that we returned 1 only if we applied the
change cross ALL the input fields, so in case some of them did not exist
and a past time was set we would return 0.

---------

Signed-off-by: Ran Shidlansik <[email protected]>
Co-authored-by: Jacob Murphy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client-changes-needed Client changes may be required for this feature needs-doc-pr This change needs to update a documentation page. Remove label once doc PR is open. release-notes This issue should get a line item in the release notes run-extra-tests Run extra tests on this PR (Runs all tests from daily except valgrind and RESP)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Support field level expire/TTL for hash, set and sorted set