You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6e9a532
Browse filesBrowse the repository at this point in the historyBrowse files
feat: Add optional GLIDE client for Redis online reads (#6870)
* feat: Add optional GLIDE client for Redis online reads
Adds a `client: glide` option to RedisOnlineStoreConfig. When set, the
synchronous read paths (online_read and the batched read behind
get_online_features) issue their HMGET commands as one non-atomic GLIDE
batch, so the fetch runs off the GIL instead of in Python per command.
Default behavior is unchanged: redis-py remains the default client, all
writes and async reads keep using redis-py, and GLIDE reads the same
entity keys and hashed feature fields Feast already writes.
valkey-glide-sync is an optional dependency behind the new `feast[glide]`
extra. Configuring `client: glide` without it installed raises a Feast
extras import error naming the extra to install.
This does not include the HGETALL threshold change from the same issue.
Related to #6856.
Signed-off-by: Chandler King <[email protected]>
* fix: Release the GLIDE client and cache its module per store
Addresses review feedback on the GLIDE read path:
- teardown() now closes the native GLIDE client and clears the cached
reference, so its Rust-side resources and file descriptors are released.
- The glide_sync module is imported once per store and cached on the
instance instead of being looked up on every read.
- Allowlist the test password literals so detect-secrets stops failing
lint-python.
Signed-off-by: Chandler King <[email protected]>
---------
Signed-off-by: Chandler King <[email protected]>
The speedup grows with the number of feature views and is most pronounced in production environments with non-trivial network RTT to Redis.
114
114
115
+
### GLIDE client (`client: glide`)
116
+
117
+
By default the Redis online store uses redis-py, which packs every command and parses every reply in Python. Each socket receive releases and reacquires the GIL, so in a process running other Python threads each reacquire waits on the switch interval. With many entities and feature views per read, that cost dominates.
118
+
119
+
Setting `client: glide` runs the same `HMGET` commands as one non-atomic [GLIDE](https://github.com/valkey-io/valkey-glide) batch. GLIDE has a Rust core, so the batch crosses the FFI boundary once and the fetch runs with the GIL released.
120
+
121
+
```yaml
122
+
online_store:
123
+
type: redis
124
+
connection_string: "localhost:6379"
125
+
client: glide
126
+
```
127
+
128
+
Install the extra to use it:
129
+
130
+
```bash
131
+
pip install 'feast[glide]'
132
+
```
133
+
134
+
Notes:
135
+
136
+
* GLIDE reads the same keys and hash fields Feast already writes, so an existing store needs no migration and no rewrite of data.
137
+
* The default stays `client: redis`. Writes and `get_online_features_async` keep using redis-py, so only the synchronous read path changes.
138
+
* GLIDE supports `host:port`, `db`, `password`, `username`, `ssl`, `socket_timeout`, and `socket_connect_timeout` from `connection_string`. Other parameters are logged and ignored.
139
+
* GLIDE has no Sentinel support, so `client: glide` cannot be combined with `redis_type: redis_sentinel`. Use `client: redis` for Sentinel deployments.
140
+
141
+
If `client: glide` is set without the extra installed, Feast raises an import error naming the extra to install.
142
+
115
143
### Write path: `skip_dedup` for bulk loads
116
144
117
145
By default, `online_write_batch()` checks existing timestamps before writing (to avoid overwriting newer data with older data). This requires two pipeline round trips per batch: one to read existing timestamps, one to write new values.
@@ -144,6 +172,7 @@ The Redis online store implements `online_write_batch_async()` using the async R
144
172
| `key_ttl_seconds` | `null` | Redis `EXPIRE` TTL in seconds applied to the entity hash key after each write. Expires all feature views for that entity together. |
145
173
| `full_scan_for_deletion` | `true` | When `true`, deleting or renaming a feature view scans Redis to remove its hash fields. Set `false` to skip deletion scans (faster `feast apply`, but leaves orphaned data). |
146
174
| `skip_dedup` | `false` | When `true`, skips the existing-timestamp read before each write, halving write round trips. Suitable for initial bulk loads; may cause older values to overwrite newer ones under concurrent writers. |
175
+
| `client` | `redis` | Read client: `redis`for redis-py, or `glide` to run the synchronous batched read as one non-atomic GLIDE batch. Requires `pip install 'feast[glide]'`. |
147
176
148
177
The full set of configuration options is available in [RedisOnlineStoreConfig](https://rtd.feast.dev/en/latest/#feast.infra.online_stores.redis.RedisOnlineStoreConfig).
149
178
@@ -172,5 +201,6 @@ Below is a matrix indicating which functionality is supported by the Redis onlin
0 commit comments