i ran into the similar problem as #8914
i'm using v3.4.13 server and v3.4.12 client.
i can reproduce the error using following steps:
on one terminal , start etcd server and enable auth and shroten the default token ttl:
etcd --log-level warn --logger zap --auth-token-ttl 10 &
etcdctl user add root:root
etcdctl user grant-role root root
etcdctl auth enable
on the other termianal, run a the demo program:
cfz@cfz-desktop:~/ws/etcd-watch-demo$ cat main.go
package main
import (
"context"
"fmt"
"sync"
"time"
"go.etcd.io/etcd/clientv3"
)
var wg sync.WaitGroup
func main() {
cfg := clientv3.Config{
Endpoints: []string{"http://127.0.0.1:2379"},
Username: "root",
Password: "root",
LogConfig: nil,
}
cli, err := clientv3.New(cfg)
if err != nil {
panic(err)
}
watch(cli, 1)
time.Sleep(1 * time.Second)
watch(cli, 2)
time.Sleep(11 * time.Second)
watch(cli, 3)
time.Sleep(1 * time.Second)
watch(cli, 4)
wg.Wait()
}
func watch(cli *clientv3.Client, num int) {
wg.Add(1)
go func() {
defer wg.Done()
for wr := range cli.Watch(context.Background(), "") {
if wr.Err() != nil {
fmt.Printf("%#v\n", wr.Err())
}
}
fmt.Printf("#%v watch existed\n", num)
}()
}
cfz@cfz-desktop:~/ws/etcd-watch-demo$ go run main.go
&errors.errorString{s:"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
#3 watch exited
&errors.errorString{s:"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
#4 watch exited
as shown in the log , the 1st and 2nd watch is ok, while the 3rd and 4th report "permission denied"
meanwile, etcd server will log following:
{"level":"warn","ts":"2020-10-10T20:34:45.010+0800","caller":"auth/store.go:1301","msg":"invalid auth token","token":"JWourXlCoFPTxMHj.8"}
{"level":"warn","ts":"2020-10-10T20:34:46.010+0800","caller":"auth/store.go:1301","msg":"invalid auth token","token":"JWourXlCoFPTxMHj.8"}
i think the causes are:
- the grpc stream's authentication token is provided at the time when the stream is initialized
- the client is trying to reusing the same grpc stream(if avaliable) to setup new watch substream.
- the token attached to the stream would be expired if no other requests to extend its ttl.
i ran into the similar problem as #8914
i'm using v3.4.13 server and v3.4.12 client.
i can reproduce the error using following steps:
on one terminal , start etcd server and enable auth and shroten the default token ttl:
on the other termianal, run a the demo program:
as shown in the log , the 1st and 2nd watch is ok, while the 3rd and 4th report "permission denied"
meanwile, etcd server will log following:
i think the causes are: