Skip to content

fix key position configuration#7455

Closed
duangg wants to merge 1 commit intoredis:unstablefrom
duangg:unstable
Closed

fix key position configuration#7455
duangg wants to merge 1 commit intoredis:unstablefrom
duangg:unstable

Conversation

@duangg
Copy link

@duangg duangg commented Jul 2, 2020

Fix some configuration in redisCommandTable

@oranagra
Copy link
Member

oranagra commented Jul 2, 2020

@duangg these commands have their own callback to find the argument indexes that refer to keys.
the syntax for these is complicated and is not possible to get the key arguments just by looking at fixed indexes.
As far as i know the changes you made aren't affecting the software at all.
Please correct me if i'm wrong, or give an example of a problem that it fixes.
Thanks.

@duangg
Copy link
Author

duangg commented Jul 2, 2020

@duangg these commands have their own callback to find the argument indexes that refer to keys.
the syntax for these is complicated and is not possible to get the key arguments just by looking at fixed indexes.
As far as i know the changes you made aren't affecting the software at all.
Please correct me if i'm wrong, or give an example of a problem that it fixes.
Thanks.

Recently I use go-redis to scan my redis-cluster for big keys and other infomations. go-redis
will cache infos return by COMMAND, use first key position to find key and calculate slot. Because the position for MEMORY is 0, go-redis will return a random slot

Is there any better solution for this problem? Should I only change info about MEMORY?

Thanks

@oranagra
Copy link
Member

oranagra commented Jul 2, 2020

@duangg i suppose the right thing to do is use COMMAND GETKEYS.
ideally go-redis should be able to tell by the COMMAND info which commands can be resolved by using the first,last,step, and which require a call to COMMAND GETKEYS each time.
I think adding such a tip and improving go-redis is the way to go since the fix you suggested may provide false result.

@duangg
Copy link
Author

duangg commented Jul 2, 2020

@oranagra Thanks for your advice, I will try to fix this bug in go-redis. You can close this PR

@oranagra
Copy link
Member

oranagra commented Jul 2, 2020

@duangg i think that in order to properly fix go-redis, we might have to add some hint in redis.
but that will cause backwards compatibility issues.
please try to figure it out (i haven't looked at go-redis code), and post here or open another PR with your conclusions.

@duangg
Copy link
Author

duangg commented Jul 3, 2020

@oranagra First, go-redis use COMMAND to get all redis command info
ring.go

func (c *Ring) cmdsInfo() (map[string]*CommandInfo, error) {
	shards := c.shards.List()
	var firstErr error
	for _, shard := range shards {
		cmdsInfo, err := shard.Client.Command(context.TODO()).Result()
		if err == nil {
			return cmdsInfo, nil
		}
		if firstErr == nil {
			firstErr = err
		}
	}
	if firstErr == nil {
		return nil, errRingShardsDown
	}
	return nil, firstErr
}

Then it try to find first key position from command info
command.go

func cmdFirstKeyPos(cmd Cmder, info *CommandInfo) int {
	switch cmd.Name() {
	case "eval", "evalsha":
		if cmd.stringArg(2) != "0" {
			return 3
		}

		return 0
	case "publish":
		return 1
	}
	if info == nil {
		return 0
	}
	return int(info.FirstKeyPos)
}

Then use first key postion to calculate key slot
cluster.go

func cmdSlot(cmd Cmder, pos int) int {
	if pos == 0 {
		return hashtag.RandomSlot()
	}
	firstKey := cmd.stringArg(pos)
	return hashtag.Slot(firstKey)
}

Because first key position in MEMORY is 0, it will return a random slot. The command will send to a random shard. I have run my program on different version of redis-cluster. In version 4.0.2, redis-server will return nil when it's in a wrong server, go-redis will throw an error. In version 6.0.5, redis-server will return with a MOVED response, go-redis can handle this situation properly, but I think it's unnecessary if go-redis have correct command info

@oranagra
Copy link
Member

oranagra commented Jul 3, 2020

Again, I think this approach is not enough.
You may argue that since go-redis only cares about the first key (and not others), and since it doesn't care about the key name itself, and it's not a problem to send the command to a random shard, then this PR is enough. I.e. If someone runs another sub-command of MEMORY, it doesn't any key.
Until some day we'll add a new sub-command of MEMORY that uses a key, and not from index 2.

However, what would be the correct "fix" for STRALGO?
You can't really know in advance the index of the key name.

I think we need to change the output of COMMAND to give an indication of which command has a getKeys callback and which one doesn't. Then go-redis has to understand that for these commands it has to call COMMAND GETKEYS each time.
the alternatives are either relying on the MOVED response, or implement more logic per command to know the arguments of each (like it does for EVAL)

@oranagra
Copy link
Member

oranagra commented Jul 5, 2020

ohh, i now see that there already is a movablekeys flag.
so that's the indication that the first/last/step fields should not be trusted, and a GETKEYS command must be executed on every call.

it's even documented: https://redis.io/commands/command

@duangg
Copy link
Author

duangg commented Jul 6, 2020

@oranagra Thanks for your explanation, I understand why I shouldn't change the configuration.

I have reed the document about command and memory, but not familiar enough, my considerations aren't comprehensive enough.

Your advice to implement more logic maybe better for this situation.

Thanks for your help

@oranagra
Copy link
Member

@duangg FYI, i see go-redis has a fix now: redis/go-redis#1400

@duangg
Copy link
Author

duangg commented Jul 20, 2020

@oranagra Thanks, it should resolve my problem

alonre24 added a commit to alonre24/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [redis#7385](RediSearch/RediSearch#7385) Fix high temporary memory consumption when loading multiple search indexes from RDB
* [redis#7430](RediSearch/RediSearch#7430) Fix a potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [redis#7454](RediSearch/RediSearch#7454) Fix a garbage collection performence regression
* [redis#7460](RediSearch/RediSearch#7460) Fix potential double-free in Fork GC error paths
* [redis#7455](RediSearch/RediSearch#7455) Fix internal cursors not being deleted promptly in cluster mode
* [redis#7667](RediSearch/RediSearch#7667) Fix a cursor logical leak upon dropping the index
* [redis#7796](RediSearch/RediSearch#7796) Fix a potential use-after-free when removing connections
* [redis#7792](RediSearch/RediSearch#7792) Fix string comparison for binary data with embedded NULLs in TOLIST reducer in FT.AGGREGATE
* [redis#7823](RediSearch/RediSearch#7823) Update `FT.HYBRID` to accept vector blobs only via parameters
* [redis#7903](RediSearch/RediSearch#7903) Fix a memory leak in Hybrid ASM
* [redis#8052](RediSearch/RediSearch#8052) Fix `FT.HYBRID` behavior when used with `LOAD *`
* [redis#8082](RediSearch/RediSearch#8082) Fix incorrect FULLTEXT field metric counts
* [redis#8089](RediSearch/RediSearch#8089) Fix an edge case in `CLUSTERSET` handling
* [redis#8152](RediSearch/RediSearch#8152) Fix configuration registration issues

**Improvements:**

* [redis#7427](RediSearch/RediSearch#7427) Enhance `FT.PROFILE` with vector search execution details
* [redis#7431](RediSearch/RediSearch#7431) Ensure full `FT.PROFILE` output is returned on timeout with RETURN policy
* [redis#7507](RediSearch/RediSearch#7507) Track timeout warnings and errors in INFO
* [redis#7576](RediSearch/RediSearch#7576) Track OOM warnings and errors in INFO
* [redis#7612](RediSearch/RediSearch#7612) Track `maxprefixexpansions` warnings and errors in INFO
* [redis#7960](RediSearch/RediSearch#7960) Persist query warnings across cursor reads
* [redis#7551](RediSearch/RediSearch#7551), [redis#7616](RediSearch/RediSearch#7616), [redis#7622](RediSearch/RediSearch#7622), [redis#7625](RediSearch/RediSearch#7625) Add runtime thread and pending-jobs metrics
* [redis#7589](RediSearch/RediSearch#7589) Support multiple slot ranges in `search.CLUSTERSET`
* [redis#7707](RediSearch/RediSearch#7707) Add `WITHCOUNT` support to `FT.AGGREGATE`
* [redis#7862](RediSearch/RediSearch#7862) Add support for subquery `COUNT` in `FT.HYBRID`
* [redis#8087](RediSearch/RediSearch#8087) Add warnings when cursor results may be affected by ASM and expose ASM warnings in `FT.PROFILE`
* [redis#8049](RediSearch/RediSearch#8049) Add logging for index-related commands
* [redis#8150](RediSearch/RediSearch#8150) Fix shard total profile time reporting in `FT.PROFILE`
YaacovHazan pushed a commit that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [#7385](RediSearch/RediSearch#7385) Fix high
temporary memory consumption when loading multiple search indexes from
RDB
* [#7430](RediSearch/RediSearch#7430) Fix a
potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [#7454](RediSearch/RediSearch#7454) Fix a
garbage collection performence regression
* [#7460](RediSearch/RediSearch#7460) Fix
potential double-free in Fork GC error paths
* [#7455](RediSearch/RediSearch#7455) Fix
internal cursors not being deleted promptly in cluster mode
* [#7667](RediSearch/RediSearch#7667) Fix a
cursor logical leak upon dropping the index
* [#7796](RediSearch/RediSearch#7796) Fix a
potential use-after-free when removing connections
* [#7792](RediSearch/RediSearch#7792) Fix string
comparison for binary data with embedded NULLs in TOLIST reducer in
FT.AGGREGATE
* [#7704](RediSearch/RediSearch#7704) Use
asynchronous jobs for GC in SVS to accelerate execution
* [#7823](RediSearch/RediSearch#7823) Update
`FT.HYBRID` to accept vector blobs only via parameters
* [#7903](RediSearch/RediSearch#7903) Fix a
memory leak in Hybrid ASM
* [#8052](RediSearch/RediSearch#8052) Fix
`FT.HYBRID` behavior when used with `LOAD *`
* [#8082](RediSearch/RediSearch#8082) Fix
incorrect FULLTEXT field metric counts
* [#8089](RediSearch/RediSearch#8089) Fix an
edge case in `CLUSTERSET` handling
* [#8152](RediSearch/RediSearch#8152) Fix
configuration registration issues

**Improvements:**

* [#7427](RediSearch/RediSearch#7427) Enhance
`FT.PROFILE` with vector search execution details
* [#7431](RediSearch/RediSearch#7431) Ensure
full `FT.PROFILE` output is returned on timeout with RETURN policy
* [#7507](RediSearch/RediSearch#7507) Track
timeout warnings and errors in INFO
* [#7576](RediSearch/RediSearch#7576) Track OOM
warnings and errors in INFO
* [#7612](RediSearch/RediSearch#7612) Track
`maxprefixexpansions` warnings and errors in INFO
* [#7960](RediSearch/RediSearch#7960) Persist
query warnings across cursor reads
* [#7551](RediSearch/RediSearch#7551),
[#7616](RediSearch/RediSearch#7616),
[#7622](RediSearch/RediSearch#7622),
[#7625](RediSearch/RediSearch#7625) Add runtime
thread and pending-jobs metrics
* [#7589](RediSearch/RediSearch#7589) Support
multiple slot ranges in `search.CLUSTERSET`
* [#7707](RediSearch/RediSearch#7707) Add
`WITHCOUNT` support to `FT.AGGREGATE`
* [#7862](RediSearch/RediSearch#7862) Add
support for subquery `COUNT` in `FT.HYBRID`
* [#8087](RediSearch/RediSearch#8087) Add
warnings when cursor results may be affected by ASM and expose ASM
warnings in `FT.PROFILE`
* [#8049](RediSearch/RediSearch#8049) Add
logging for index-related commands
* [#8150](RediSearch/RediSearch#8150) Fix shard
total profile time reporting in `FT.PROFILE`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants