Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kubectl 1.10.0 is unable to create deployments on clusters running 1.8.7 #61943

Closed
ihoegen opened this issue Mar 30, 2018 · 12 comments
Closed

Kubectl 1.10.0 is unable to create deployments on clusters running 1.8.7 #61943

ihoegen opened this issue Mar 30, 2018 · 12 comments
Assignees
@liggitt
Labels
area/client-libraries kind/bug Categorizes issue or PR as related to a bug. milestone/incomplete-labels sig/cli Categorizes an issue or PR as relevant to SIG CLI.
Milestone
v1.10

Comments

@ihoegen
Copy link

ihoegen commented Mar 30, 2018

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

/kind bug

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-27T00:13:02Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.7", GitCommit:"b30876a5539f09684ff9fde266fda10b37738c9c", GitTreeState:"clean", BuildDate:"2018-01-16T21:52:38Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:

  • Cloud provider or hardware configuration: AWS
  • OS: macOS 10.13.3
  • Kernel: Darwin 17.4.0
  • Install tools: brew

What happened:

When creating a sample deployment using the command kubectl create -f deployment.yaml, the following output appears:

Error from server (NotAcceptable): unknown

What you expected to happen:

When running the same command again, kubectl create -f deployment.yaml, with kubectl version 1.9.3, the following output appears:

deployment "nginx-deployment" created

A better error message describing incompatibility might be useful.

How to reproduce it (as minimally and precisely as possible):

  1. Use the following config, and save it as deployment.yaml:
apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template: 
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
  1. Run kubectl create -f deployment.yaml using kubectl 1.10.0
@k8s-ci-robot k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. kind/bug Categorizes issue or PR as related to a bug. labels Mar 30, 2018
@ihoegen
Copy link
Author

ihoegen commented Mar 30, 2018

/sig cli

@k8s-ci-robot k8s-ci-robot added sig/cli Categorizes an issue or PR as relevant to SIG CLI. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Mar 30, 2018
@ihoegen ihoegen mentioned this issue Mar 30, 2018
Closed
@jhorwit2
Copy link
Contributor

@ihoegen are you by chance a cluster admin? I can reproduce this when I impersonate a non-admin user but not as an admin. It seems to be RBAC related since /openapi/v2 isn't part of system:discovery. If you run the kubectl command with --v=6 I'm assuming it fails on that request.

@jhorwit2
Copy link
Contributor

Output

I0330 13:35:54.068481   67702 loader.go:357] Config loaded from file /Users/<user>/.kube/config
I0330 13:35:54.192461   67702 round_trippers.go:405] GET https://<url>l/openapi/v2 406 Not Acceptable in 121 milliseconds
I0330 13:35:54.193710   67702 helpers.go:201] server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "unknown",
  "reason": "NotAcceptable",
  "details": {
    "causes": [
      {
        "reason": "UnexpectedServerResponse",
        "message": "unknown"
      }
    ]
  },
  "code": 406
}]
F0330 13:35:54.193740   67702 helpers.go:119] Error from server (NotAcceptable): unknown

@jhorwit2
Copy link
Contributor

/sig auth

@liggitt This response isn't correct for an RBAC deny, right? I see the DENY in the apiserver logs.

@k8s-ci-robot k8s-ci-robot added the sig/auth Categorizes an issue or PR as relevant to SIG Auth. label Mar 30, 2018
@ihoegen
Copy link
Author

ihoegen commented Mar 30, 2018

You're correct, I switched to the admin role, and I was able to create it. I was able to delete it from the developer role though.

The developer rules are:

rules:
- apiGroups: ["*"] 
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "delete", "update", "patch"]

@ihoegen
Copy link
Author

ihoegen commented Mar 30, 2018

I was able to create it with cluster-admin, and my namespace admin roles as well. Namespace admin rules are:

rules:
- apiGroups: ["*"] 
  resources: ["*"]
  verbs: ["*"]

with a clusterrole of

rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["get", "watch", "list"]

The non admins also use this same clusterrole.

@liggitt
Copy link
Member

liggitt commented Mar 30, 2018

looks like a duplicate of #61805

@jhorwit2
Copy link
Contributor

@ihoegen this is a non-resource url so it's not covered by those rules.

@ihoegen
Copy link
Author

ihoegen commented Mar 30, 2018 via email

@liggitt
Copy link
Member

liggitt commented Mar 30, 2018 •
edited
Loading

Why would it work with kubectl 1.9.3 then?

Because kubectl 1.9.x doesn't attempt to fetch that schema as a prereq to creating/updating all objects, in order to maintain compatibility with 1.8.x servers (which are within the supported version skew of +/- one version)

@liggitt liggitt mentioned this issue Mar 30, 2018
Merged
@liggitt liggitt added area/client-libraries and removed sig/auth Categorizes an issue or PR as relevant to SIG Auth. labels Mar 31, 2018
@liggitt liggitt added this to the v1.10 milestone Mar 31, 2018
@liggitt liggitt self-assigned this Mar 31, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Issue Labels Incomplete

@ihoegen @liggitt

Action required: This issue requires label changes. If the required changes are not made within 1 day, the issue will be moved out of the v1.10 milestone.

priority: Must specify exactly one of priority/critical-urgent, priority/important-longterm or priority/important-soon.

Help

k8s-github-robot pushed a commit that referenced this issue Apr 2, 2018
Merge pull request #61949 from liggitt/tolerate-openapi
6610069 
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Tolerate 406 mime-type errors attempting to load new openapi schema

Fixes #61805
Fixes #61943 

```release-note
kubectl: improves compatibility with older servers when creating/updating API objects
```
@liggitt
Copy link
Member

liggitt commented Apr 3, 2018

fixed in master in #61949
fixed in release-1.10 branch in #62024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@liggitt liggitt

Labels
area/client-libraries kind/bug Categorizes issue or PR as related to a bug. milestone/incomplete-labels sig/cli Categorizes an issue or PR as relevant to SIG CLI.
Projects
None yet
Milestone
v1.10
Development

No branches or pull requests
5 participants
@liggitt @jhorwit2 @ihoegen @k8s-github-robot @k8s-ci-robot