1205 03:58:34.355341 1929 instrumented_service.go:129] StartContainer for "6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c" returns successfully
E1205 03:58:39.830445 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E1205 03:58:45.821256 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = Canceled desc = context canceled
I1205 03:59:05.021052 1929 instrumented_service.go:287] ImageFsInfo
I1205 03:59:05.021131 1929 instrumented_service.go:292] ImageFsInfo returns filesystem info [&FilesystemUsage{Timestamp:1512446340907845245,StorageId:&StorageIdentifier{Uuid:27291a55-d3a1-4729-8b41-397876fd302a,},UsedBytes:&UInt64Value{Value:2505028319,},InodesUsed:&UInt64Value{Value:75245,},}]
E1205 03:59:05.021638 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I1205 03:59:05.039675 1929 instrumented_service.go:287] ImageFsInfo
I1205 03:59:05.040096 1929 instrumented_service.go:292] ImageFsInfo returns filesystem info [&FilesystemUsage{Timestamp:1512446340907845245,StorageId:&StorageIdentifier{Uuid:27291a55-d3a1-4729-8b41-397876fd302a,},UsedBytes:&UInt64Value{Value:2505028319,},InodesUsed:&UInt64Value{Value:75245,},}]
E1205 03:59:05.053445 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E1205 04:00:05.033052 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E1205 04:00:05.040684 1929 instrumented_service.go:52] ListPodSandbox failed, error: failed to list sandbox containers: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E1205 04:00:31.804838 1929 sandbox_run.go:216] Failed to delete sandbox container "7c162d9bb254bed8e2c38bcd93340a74e49fd2c08a273b1989e13e4f75bf1554": context deadline exceeded: unknown
E1205 04:00:31.820405 1929 sandbox_run.go:170] Failed to delete containerd container "7c162d9bb254bed8e2c38bcd93340a74e49fd2c08a273b1989e13e4f75bf1554": context deadline exceeded: unknown
ime="2017-12-05T03:56:48Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/delete" type=containerd.events.TaskDelete
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/events" ns=k8s.io topic="/tasks/exit" type=containerd.events.TaskExit
time="2017-12-05T03:58:34Z" level=debug msg="received signal" module=containerd signal=child exited
time="2017-12-05T03:58:34Z" level=info msg="shim reaped" id=2af28178b9da5c29ec147de86732c502d8132a071cd8dd4c5939f48515a7a6c7 module="containerd/tasks"
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/delete" type=containerd.events.TaskDelete
time="2017-12-05T03:58:34Z" level=debug msg="prepare snapshot" key=6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c module="containerd/snapshot" parent="sha256:3e7065e2902c8af4232958c4770df3036568e0b266ccbafefcb8c3c36af4197f"
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/snapshot" ns=k8s.io topic="/snapshot/prepare" type=containerd.events.SnapshotPrepare
time="2017-12-05T03:58:34Z" level=debug msg="get snapshot mounts" key=6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c module="containerd/snapshot"
time="2017-12-05T03:58:34Z" level=debug msg="get snapshot mounts" key=6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c module="containerd/snapshot"
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/containers" ns=k8s.io topic="/containers/create" type=containerd.events.ContainerCreate
time="2017-12-05T03:58:34Z" level=debug msg="get snapshot mounts" key=6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c module="containerd/snapshot"
time="2017-12-05T03:58:34Z" level=info msg="shim /home/cri-containerd/usr/local/bin/containerd-shim started" address="/containerd-shim/k8s.io/6b739d7d56993f63f54b9643563ef6739864653dc377a120cbe876685c9fe42c/shim.sock" debug=false module="containerd/tasks" pid=15055
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/create" type=containerd.events.TaskCreate
time="2017-12-05T03:58:34Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/start" type=containerd.events.TaskStart
time="2017-12-05T04:03:10Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/exec-started" type=containerd.events.TaskExecStarted
time="2017-12-05T04:03:10Z" level=debug msg="event published" module="containerd/events" ns=k8s.io topic="/tasks/exit" type=containerd.events.TaskExit
time="2017-12-05T04:04:10Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/exec-started" type=containerd.events.TaskExecStarted
time="2017-12-05T04:04:10Z" level=debug msg="event published" module="containerd/events" ns=k8s.io topic="/tasks/exit" type=containerd.events.TaskExit
time="2017-12-05T04:05:10Z" level=debug msg="event published" module="containerd/tasks" ns=k8s.io topic="/tasks/exec-started" type=containerd.events.TaskExecStarted
time="2017-12-05T04:05:10Z" level=debug msg="event published" module="containerd/events" ns=k8s.io topic="/tasks/exit" type=containerd.events.TaskExit
I've seen this twice today. Will update more information if I saw it once more.
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-cri-containerd-e2e-ubuntu-gce/653
cri-containerd.log:
containerd.log:
I've seen this twice today. Will update more information if I saw it once more.