Skip to content
This repository was archived by the owner on Oct 10, 2023. It is now read-only.
This repository was archived by the owner on Oct 10, 2023. It is now read-only.

CLUSTER_API_SERVER_PORT not working in classy cluster with kube-vip #3803

@ydp

Description

@ydp

Bug description

Creating classy cluster on vSphere with kube-vip and customized apiserver port failed.

23:14:10  CLUSTER_API_SERVER_PORT: 443
23:14:10  CLUSTER_PLAN: devcc
23:14:10  CNI: antrea
23:14:10  INFRASTRUCTURE_PROVIDER: vsphere
23:14:10  KUBERNETES_VERSION: v1.23.8+vmware.2
23:14:10  OS_ARCH: amd64
23:14:10  OS_NAME: photon
23:14:10  OS_VERSION: '3'
23:14:10  _VSPHERE_CONTROL_PLANE_ENDPOINT: 10.92.108.173
23:14:10  INFO:root:====== 93   CMD: tanzu kubernetes cluster create tkg-vc-antrea-customport -v 9 -f tkg-vc-antrea-customport.yaml
23:14:10  compatibility file (/home/kubo/.config/tanzu/tkg/compatibility/tkg-compatibility.yaml) already exists, skipping download
23:14:10  BOM files inside /home/kubo/.config/tanzu/tkg/bom already exists, skipping download
23:14:10  Using namespace: default
23:14:10  Validating configuration...
23:14:10  waiting for resource pinniped-info of type *v1.ConfigMap to be up and running
23:14:10  configmaps "pinniped-info" not found, retrying
23:14:16  configmaps "pinniped-info" not found, retrying
23:14:21  configmaps "pinniped-info" not found, retrying
23:14:26  configmaps "pinniped-info" not found, retrying
23:14:31  configmaps "pinniped-info" not found, retrying
23:14:35  configmaps "pinniped-info" not found, retrying
23:14:35  configmaps "pinniped-info" not found, retrying
23:14:35  Warning: Pinniped configuration not found; Authentication via Pinniped will not be set up in this cluster. If you wish to set up Pinniped after the cluster is created, please refer to the documentation.
23:14:36  Setting config variable "VSPHERE_DATACENTER" to value "/dc0"
23:14:36  Setting config variable "VSPHERE_NETWORK" to value "/dc0/network/VM Network"
23:14:36  Setting config variable "VSPHERE_RESOURCE_POOL" to value "/dc0/host/cluster0/Resources/rp0"
23:14:36  Setting config variable "VSPHERE_DATASTORE" to value "/dc0/datastore/sharedVmfs-0"
23:14:36  Setting config variable "VSPHERE_FOLDER" to value "/dc0/vm/folder0"
23:14:36  Fetching File="cluster-template-definition-devcc.yaml" Provider="vsphere" Type="InfrastructureProvider" Version="v1.4.1"
23:14:37  
23:14:37  Legacy configuration file detected. The inputs from said file have been converted into the new Cluster configuration as '/home/kubo/.config/tanzu/tkg/clusterconfigs/tkg-vc-antrea-customport.yaml'
23:14:37  
23:14:37  Using this new Cluster configuration '/home/kubo/.config/tanzu/tkg/clusterconfigs/tkg-vc-antrea-customport.yaml' to create the cluster.
23:14:37  creating workload cluster 'tkg-vc-antrea-customport'...
23:14:37  kubectl apply failed, output: secret/tkg-vc-antrea-customport-vsphere-credential created
23:14:37  secret/tkg-vc-antrea-customport-nsxt-credential created
23:14:37  vspherecpiconfig.cpi.tanzu.vmware.com/tkg-vc-antrea-customport created
23:14:37  clusterbootstrap.run.tanzu.vmware.com/tkg-vc-antrea-customport created
23:14:37  secret/tkg-vc-antrea-customport created
23:14:37  Error from server (could not get secret for key: default/tkg-vc-antrea-customport: Secret "tkg-vc-antrea-customport" not found): error when creating "/tmp/kubeapply-2147717331": admission webhook "tkr-vsphere-resolver-webhook.tanzu.vmware.com" denied the request: could not get secret for key: default/tkg-vc-antrea-customport: Secret "tkg-vc-antrea-customport" not found
23:14:37  : exit status 1, retrying
23:14:46  patch cluster object with operation status: 
23:14:46  	{
23:14:46  		"metadata": {
23:14:46  			"annotations": {
23:14:46  				"TKGOperationInfo" : "{\"Operation\":\"Create\",\"OperationStartTimestamp\":\"2022-10-31 15:14:45.75701019 +0000 UTC\",\"OperationTimeout\":1800}",
23:14:46  				"TKGOperationLastObservedTimestamp" : "2022-10-31 15:14:45.75701019 +0000 UTC"
23:14:46  			}
23:14:46  		}
23:14:46  	}
23:14:46  waiting for cluster to be initialized...
23:14:46  cluster state is unchanged 1
23:14:46  [zero or multiple KCP objects found for the given cluster, 0 tkg-vc-antrea-customport default, no MachineDeployment objects found for the given cluster]
23:14:46  [zero or multiple KCP objects found for the given cluster, 0 tkg-vc-antrea-customport default, no MachineDeployment objects found for the given cluster], retrying
23:15:01  cluster state is unchanged 2
...
23:46:47  cluster control plane is still being initialized: WaitingForKubeadmInit, retrying
23:47:02  cluster state is unchanged 120
23:47:02  cluster control plane is still being initialized: WaitingForKubeadmInit, retrying
23:47:17  cluster state is unchanged 121
23:47:17  timed out waiting for cluster creation to complete: cluster control plane is still being initialized: WaitingForKubeadmInit
23:47:17  Error: unable to wait for cluster and get the cluster kubeconfig: error waiting for cluster to be provisioned (this may take a few minutes): timed out waiting for cluster creation to complete: cluster control plane is still being initialized: WaitingForKubeadmInit
23:47:17  INFO:root:====== 93   Errored: 1 Encountered a bad command exit code!

Affected product area (please put an X in all that apply)

  • ( ) APIs
  • ( ) Addons
  • (*) CLI
  • ( ) Docs
  • ( ) IAM
  • ( ) Installation
  • ( ) Plugin
  • ( ) Security
  • ( ) Test and Release
  • ( ) User Experience
  • ( ) Developer Experience

Expected behavior

CLUSTER_API_SERVER_PORT works as in legacy cluster.

Steps to reproduce the bug

Version (include the SHA if the version is not obvious)

Environment where the bug was observed (cloud, OS, etc)

Relevant Debug Output (Logs, manifests, etc)

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugPR/Issue related to a bugneeds-triageIndicates an issue or PR needs to be triaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions