Attaching a vSphere 7.0 with Tanzu supervisor cluster to Tanzu Mission Control and creating new Tanzu Kubernetes clusters

Adding a Tanzu Kubernetes cluster to TMC management as an attached cluster has been possible for quite a while. You didn’t have the same level of control as you would for a cluster created by TMC on AWS but it at least provided some visibility and the policy functionality provided by TMC. As of yesterday, we now have the ability to registry a vSphere 7 with Tanzu supervisor cluster in TMC and then provision Tanzu Kubernetes clusters directly from TMC. This is a huge milestone for both TMC and vSphere 7 with Tanzu and I was incredibly excited to try it out. You can read many more details on this process at Managing the Lifecycle of Tanzu Kubernetes Clusters and How to register/deregister a vSphere with Tanzu supervisor cluster in Tanzu Mission Control (80727) but keep reading below to see my brief walkthrough of the process.

I started out with a simple supervisor cluster and a single namespace named tkg (using vDS networking). You can read more about setting up vSphere 7.0 with Tanzu in my previous post, How to install vSphere with Tanzu with vSphere Networking.

You can see that there are no Tanzu Kubernetes clusters created yet.

Within the TMC UI, there is a new section under Administration called Management clusters.

Be default, you’ll see the the aws-hosted management cluster which is used for provisioning Tanzu Kubernetes clusters on AWS. This was previously hidden from view in earlier versions of TMC. If you drill down into this cluster you can see some details about the accounts in your org that have been used for creating clusters on AWS.

I want to add a supervisor cluster to the recognized management clusters so that I can use TMC to create Tanzu Kubernetes clusters in my vSphere 7 with Tanzu environment. Back on the Management clusters page, click on the Register Management Cluster button.

You only need to provide a name and cluster group here but can provide optional information if desired. Click on the Next button.

At this point, you need to copy the the registration URL as it will be used shortly.

While logged in to the supervisor cluster (via the kubectl vsphere login command), run the kubectl get ns command to get a listing of the available namespaces.

NAME                                        STATUS   AGE
default                                     Active   71d
kube-node-lease                             Active   71d
kube-public                                 Active   71d
kube-system                                 Active   71d
svc-tmc-c8                                  Active   71d
tkg                                         Active   71d
vmware-system-appplatform-operator-system   Active   71d
vmware-system-capw                          Active   71d
vmware-system-cert-manager                  Active   71d
vmware-system-csi                           Active   71d
vmware-system-kubeimage                     Active   71d
vmware-system-lbapi                         Active   71d
vmware-system-license-operator              Active   71d
vmware-system-netop                         Active   71d
vmware-system-registry                      Active   71d
vmware-system-tkg                           Active   71d
vmware-system-ucs                           Active   71d
vmware-system-vmop                          Active   71d

The namespace with which we’ll be working is svc-tmc-c8. This namespace will always start with svc-tmc- and it should be the only one present, making it fairly easy to identify. If you look in this namespace, you’ll see that there isn’t much going on there right now.

kubectl -n svc-tmc-c8 get po

NAME                                   READY   STATUS      RESTARTS   AGE
tmc-agent-installer-1607094720-x9zj7   0/1     Completed   0          46s

A new AgentInstaller object is needed in the supervisor cluster to allow TMC to communicate with it. This is a CRD that will reference the registration URL generated earlier and create all of the resources that are needed for TMC functionality. You can take a closer look at what is being created in this sample.

registration URL YAML

apiVersion: v1
kind: Namespace
metadata:
  labels:
    control-plane: extension-manager
    tmc-extension: "true"
    controller-tools.k8s.io: "1.0"
  name: {{.Namespace}}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: stack-config
  namespace: {{.Namespace}}
  labels:
    tmc.cloud.vmware.com/managed: "true"
data:
  resource_uid: "mc:01ERQ3VX1VDAMNK6ZPA1XHV6WD"
  org_id: "e6f8b4af-faa2-4b55-8403-97d2d6b19341"
  management_cluster_name: "cjl-vsphere7"
  cluster_rid: "rid:mc:e6f8b4af-faa2-4b55-8403-97d2d6b19341:cjl-vsphere7"
  tmc_url: https://epsg.tmc.cloud.vmware.com
  tmc_host: epsg.tmc.cloud.vmware.com
  tls.crt: |+
    -----BEGIN CERTIFICATE-----
    MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/
    MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
    DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow
    PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD
    Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
    AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O
    rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq
    OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b
    xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw
    7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD
    aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV
    HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG
    SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69
    ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr
    AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz
    R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5
    JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo
    Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ
    -----END CERTIFICATE-----

    -----BEGIN CERTIFICATE-----
    MIIDxTCCAq2gAwIBAgIQAqxcJmoLQJuPC3nyrkYldzANBgkqhkiG9w0BAQUFADBs
    MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
    d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
    ZSBFViBSb290IENBMB4XDTA2MTExMDAwMDAwMFoXDTMxMTExMDAwMDAwMFowbDEL
    MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
    LmRpZ2ljZXJ0LmNvbTErMCkGA1UEAxMiRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug
    RVYgUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMbM5XPm
    +9S75S0tMqbf5YE/yc0lSbZxKsPVlDRnogocsF9ppkCxxLeyj9CYpKlBWTrT3JTW
    PNt0OKRKzE0lgvdKpVMSOO7zSW1xkX5jtqumX8OkhPhPYlG++MXs2ziS4wblCJEM
    xChBVfvLWokVfnHoNb9Ncgk9vjo4UFt3MRuNs8ckRZqnrG0AFFoEt7oT61EKmEFB
    Ik5lYYeBQVCmeVyJ3hlKV9Uu5l0cUyx+mM0aBhakaHPQNAQTXKFx01p8VdteZOE3
    hzBWBOURtCmAEvF5OYiiAhF8J2a3iLd48soKqDirCmTCv2ZdlYTBoSUeh10aUAsg
    EsxBu24LUTi4S8sCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQF
    MAMBAf8wHQYDVR0OBBYEFLE+w2kD+L9HAdSYJhoIAu9jZCvDMB8GA1UdIwQYMBaA
    FLE+w2kD+L9HAdSYJhoIAu9jZCvDMA0GCSqGSIb3DQEBBQUAA4IBAQAcGgaX3Nec
    nzyIZgYIVyHbIUf4KmeqvxgydkAQV8GK83rZEWWONfqe/EW1ntlMMUu4kehDLI6z
    eM7b41N5cdblIZQB2lWHmiRk9opmzN6cN82oNLFpmyPInngiK3BD41VHMWEZ71jF
    hS9OMPagMRYjyOfiZRYzy78aG6A9+MpeizGLYAiJLQwGXFK3xPkKmNEVX58Svnw2
    Yzi9RKR/5CYrCsSXaQ3pjOLAEFe4yHYSkVXySGnYvCoCWw9E1CAx2/S6cCZdkGCe
    vEsXCS+0yx5DaMkHJ8HSXPfqIbloEpw8nL+e/IBcm2PN7EeqJSdnoDfzAIJ9VNep
    +OkuE6N36B9K
    -----END CERTIFICATE-----
---
apiVersion: v1
kind: Secret
metadata:
  name: tmc-access-secret
  namespace: {{.Namespace}}
  labels:
    tmc.cloud.vmware.com/managed: "true"
type: Opaque
data:
 access_token_info : "ZXlKaFkyTmxjM05mZEc5clpXNGlPaUoyTVdGblpXNTBMbkJUYTFWeE5WWk5VVkF0VkY4NFkzTnJWSEJ2YzFwMU5tZDFOVTVuVUV3MGQzSlVSbUZ3YkVoRVVVUkJPRFZDY0dSUlZXa3dVVm8wZEdaUFMwYzVjalVpZlE9PQ=="
---
---
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
    tmc.cloud.vmware.com/do-not-delete: "true"
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    controller-tools.k8s.io: "1.0"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: agents.clusters.tmc.cloud.vmware.com
spec:
  additionalPrinterColumns:
  - JSONPath: .status.status
    name: Status
    type: integer
  - JSONPath: .status.health
    name: Health
    type: integer
  group: clusters.tmc.cloud.vmware.com
  names:
    kind: Agent
    listKind: AgentList
    plural: agents
    singular: agent
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      description: Agent is the Schema for the agents API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: AgentSpec defines the desired state of Agent
          properties:
            detach:
              type: boolean
            extensions:
              items:
                type: string
              type: array
            namespace:
              type: string
          type: object
        status:
          description: AgentStatus defines the observed state of Agent
          properties:
            clusterHealth:
              description: AggregatedClusterHealth defines the observed state of the
                cluster.
              properties:
                controllerManagerHealth:
                  description: ComponentHealth defines the health of a component.
                  properties:
                    health:
                      format: int32
                      type: integer
                    message:
                      type: string
                    name:
                      type: string
                  required:
                  - name
                  type: object
                etcdHealth:
                  items:
                    description: ComponentHealth defines the health of a component.
                    properties:
                      health:
                        format: int32
                        type: integer
                      message:
                        type: string
                      name:
                        type: string
                    required:
                    - name
                    type: object
                  type: array
                message:
                  type: string
                schedulerHealth:
                  description: ComponentHealth defines the health of a component.
                  properties:
                    health:
                      format: int32
                      type: integer
                    message:
                      type: string
                    name:
                      type: string
                  required:
                  - name
                  type: object
                timestamp:
                  description: "A Timestamp represents a point in time independent
                    of any time zone or calendar, represented as seconds and fractions
                    of seconds at nanosecond resolution in UTC Epoch time. It is encoded
                    using the Proleptic Gregorian Calendar which extends the Gregorian
                    calendar backwards to year one. It is encoded assuming all minutes
                    are 60 seconds long, i.e. leap seconds are \"smeared\" so that
                    no leap second table is needed for interpretation. Range is from
                    0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting
                    to that range, we ensure that we can convert to and from  RFC
                    3339 date strings. See [https://www.ietf.org/rfc/rfc3339.txt](https://www.ietf.org/rfc/rfc3339.txt).
                    \n # Examples \n Example 1: Compute Timestamp from POSIX `time()`.
                    \n     Timestamp timestamp;     timestamp.set_seconds(time(NULL));
                    \    timestamp.set_nanos(0); \n Example 2: Compute Timestamp from
                    POSIX `gettimeofday()`. \n     struct timeval tv;     gettimeofday(&tv,
                    NULL); \n     Timestamp timestamp;     timestamp.set_seconds(tv.tv_sec);
                    \    timestamp.set_nanos(tv.tv_usec * 1000); \n Example 3: Compute
                    Timestamp from Win32 `GetSystemTimeAsFileTime()`. \n     FILETIME
                    ft;     GetSystemTimeAsFileTime(&ft);     UINT64 ticks = (((UINT64)ft.dwHighDateTime)
                    << 32) | ft.dwLowDateTime; \n     // A Windows tick is 100 nanoseconds.
                    Windows epoch 1601-01-01T00:00:00Z     // is 11644473600 seconds
                    before Unix epoch 1970-01-01T00:00:00Z.     Timestamp timestamp;
                    \    timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL));
                    \    timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));
                    \n Example 4: Compute Timestamp from Java `System.currentTimeMillis()`.
                    \n     long millis = System.currentTimeMillis(); \n     Timestamp
                    timestamp = Timestamp.newBuilder().setSeconds(millis / 1000)         .setNanos((int)
                    ((millis % 1000) * 1000000)).build(); \n Example 5: Compute Timestamp
                    from current time in Python. \n     timestamp = Timestamp()     timestamp.GetCurrentTime()
                    \n # JSON Mapping \n In JSON format, the Timestamp type is encoded
                    as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt)
                    format. That is, the format is \"{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z\"
                    where {year} is always expressed using four digits while {month},
                    {day}, {hour}, {min}, and {sec} are zero-padded to two digits
                    each. The fractional seconds, which can go up to 9 digits (i.e.
                    up to 1 nanosecond resolution), are optional. The \"Z\" suffix
                    indicates the timezone (\"UTC\"); the timezone is required. A
                    proto3 JSON serializer should always use UTC (as indicated by
                    \"Z\") when printing the Timestamp type and a proto3 JSON parser
                    should be able to accept both UTC and other timezones (as indicated
                    by an offset). \n For example, \"2017-01-15T01:30:15.01Z\" encodes
                    15.01 seconds past 01:30 UTC on January 15, 2017. \n In JavaScript,
                    one can convert a Date object to this format using the standard
                    [toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString]
                    method. In Python, a standard `datetime.datetime` object can be
                    converted to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime)
                    with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in
                    Java, one can use the Joda Time's [`ISODateTimeFormat.dateTime()`](
                    http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--
                    ) to obtain a formatter capable of generating timestamps in this
                    format."
                  properties:
                    nanos:
                      description: Non-negative fractions of a second at nanosecond
                        resolution. Negative second values with fractions must still
                        have non-negative nanos values that count forward in time.
                        Must be from 0 to 999,999,999 inclusive.
                      format: int32
                      type: integer
                    seconds:
                      description: Represents seconds of UTC time since Unix epoch
                        1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
                        9999-12-31T23:59:59Z inclusive.
                      format: int64
                      type: integer
                  type: object
              type: object
            deploymentLink:
              type: string
            extensions:
              items:
                type: string
              type: array
            health:
              format: int32
              type: integer
            metadata:
              properties:
                cloudProvider:
                  format: int32
                  type: integer
                clusterCPU:
                  description: ResourceAllocation defines the resource utilisation
                    and availability.
                  properties:
                    allocatable:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    allocatedPercentage:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    requests:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    units:
                      type: string
                  required:
                  - allocatable
                  - allocatedPercentage
                  - requests
                  - units
                  type: object
                clusterMemory:
                  description: ResourceAllocation defines the resource utilisation
                    and availability.
                  properties:
                    allocatable:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    allocatedPercentage:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    requests:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    units:
                      type: string
                  required:
                  - allocatable
                  - allocatedPercentage
                  - requests
                  - units
                  type: object
                kubeServerVersion:
                  type: string
                kubernetesProvider:
                  properties:
                    type:
                      format: int32
                      type: integer
                    version:
                      type: string
                  type: object
                lastUpdate:
                  description: "A Timestamp represents a point in time independent
                    of any time zone or calendar, represented as seconds and fractions
                    of seconds at nanosecond resolution in UTC Epoch time. It is encoded
                    using the Proleptic Gregorian Calendar which extends the Gregorian
                    calendar backwards to year one. It is encoded assuming all minutes
                    are 60 seconds long, i.e. leap seconds are \"smeared\" so that
                    no leap second table is needed for interpretation. Range is from
                    0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting
                    to that range, we ensure that we can convert to and from  RFC
                    3339 date strings. See [https://www.ietf.org/rfc/rfc3339.txt](https://www.ietf.org/rfc/rfc3339.txt).
                    \n # Examples \n Example 1: Compute Timestamp from POSIX `time()`.
                    \n     Timestamp timestamp;     timestamp.set_seconds(time(NULL));
                    \    timestamp.set_nanos(0); \n Example 2: Compute Timestamp from
                    POSIX `gettimeofday()`. \n     struct timeval tv;     gettimeofday(&tv,
                    NULL); \n     Timestamp timestamp;     timestamp.set_seconds(tv.tv_sec);
                    \    timestamp.set_nanos(tv.tv_usec * 1000); \n Example 3: Compute
                    Timestamp from Win32 `GetSystemTimeAsFileTime()`. \n     FILETIME
                    ft;     GetSystemTimeAsFileTime(&ft);     UINT64 ticks = (((UINT64)ft.dwHighDateTime)
                    << 32) | ft.dwLowDateTime; \n     // A Windows tick is 100 nanoseconds.
                    Windows epoch 1601-01-01T00:00:00Z     // is 11644473600 seconds
                    before Unix epoch 1970-01-01T00:00:00Z.     Timestamp timestamp;
                    \    timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL));
                    \    timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));
                    \n Example 4: Compute Timestamp from Java `System.currentTimeMillis()`.
                    \n     long millis = System.currentTimeMillis(); \n     Timestamp
                    timestamp = Timestamp.newBuilder().setSeconds(millis / 1000)         .setNanos((int)
                    ((millis % 1000) * 1000000)).build(); \n Example 5: Compute Timestamp
                    from current time in Python. \n     timestamp = Timestamp()     timestamp.GetCurrentTime()
                    \n # JSON Mapping \n In JSON format, the Timestamp type is encoded
                    as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt)
                    format. That is, the format is \"{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z\"
                    where {year} is always expressed using four digits while {month},
                    {day}, {hour}, {min}, and {sec} are zero-padded to two digits
                    each. The fractional seconds, which can go up to 9 digits (i.e.
                    up to 1 nanosecond resolution), are optional. The \"Z\" suffix
                    indicates the timezone (\"UTC\"); the timezone is required. A
                    proto3 JSON serializer should always use UTC (as indicated by
                    \"Z\") when printing the Timestamp type and a proto3 JSON parser
                    should be able to accept both UTC and other timezones (as indicated
                    by an offset). \n For example, \"2017-01-15T01:30:15.01Z\" encodes
                    15.01 seconds past 01:30 UTC on January 15, 2017. \n In JavaScript,
                    one can convert a Date object to this format using the standard
                    [toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString]
                    method. In Python, a standard `datetime.datetime` object can be
                    converted to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime)
                    with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in
                    Java, one can use the Joda Time's [`ISODateTimeFormat.dateTime()`](
                    http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--
                    ) to obtain a formatter capable of generating timestamps in this
                    format."
                  properties:
                    nanos:
                      description: Non-negative fractions of a second at nanosecond
                        resolution. Negative second values with fractions must still
                        have non-negative nanos values that count forward in time.
                        Must be from 0 to 999,999,999 inclusive.
                      format: int32
                      type: integer
                    seconds:
                      description: Represents seconds of UTC time since Unix epoch
                        1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
                        9999-12-31T23:59:59Z inclusive.
                      format: int64
                      type: integer
                  type: object
                masterNodeCount:
                  format: int64
                  type: integer
                namespacesCount:
                  format: int64
                  type: integer
                physicalMemory:
                  format: int64
                  type: integer
                podCount:
                  format: int64
                  type: integer
                region:
                  type: string
                vcpuCount:
                  format: int64
                  type: integer
                workerNodeCount:
                  format: int64
                  type: integer
              type: object
            status:
              format: int32
              type: integer
          type: object
      type: object
  version: v1alpha1
  versions:
  - name: v1alpha1
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    tmc.cloud.vmware.com/do-not-delete: "true"
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    controller-tools.k8s.io: "1.0"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extensionconfigs.intents.tmc.cloud.vmware.com
spec:
  group: intents.tmc.cloud.vmware.com
  names:
    kind: ExtensionConfig
    plural: extensionconfigs
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            configMaps:
              description: ConfigMaps are the configMaps of the extension
              items:
                type: object
              type: array
          type: object
        status:
          properties:
            state:
              description: State indicates the state of the ExtensionConfig
              type: string
          type: object
  version: v1alpha1
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.2.5
    tmc.cloud.vmware.com/do-not-delete: "true"
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    app: extension-manager
    controller-tools.k8s.io: "1.0"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extensionintegrations.clusters.tmc.cloud.vmware.com
spec:
  group: clusters.tmc.cloud.vmware.com
  names:
    kind: ExtensionIntegration
    listKind: ExtensionIntegrationList
    plural: extensionintegrations
    singular: extensionintegration
  scope: Namespaced
  validation:
    openAPIV3Schema:
      description: ExtensionIntegration is the Schema for the extensionintegrations
        API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: ExtensionIntegrationSpec defines the desired state of ExtensionIntegration
          properties:
            extensionHealth:
              description: Health of the Extension Workload that is managing the underlying
                Application
              format: int32
              type: integer
            extensionState:
              description: Status of the Extension Workload that is managing the underlying
                Application
              format: int32
              type: integer
          type: object
        status:
          description: ExtensionIntegrationStatus defines the observed state of ExtensionIntegration
          properties:
            applicationConditions:
              description: Conditions of the Extension that is integrated with TMC
              items:
                description: Conditions captures readiness and health conditions for
                  an underlying Application mapping to an extension. Collectively
                  they capture the set of conditions that define the state and health
                  of application.
                properties:
                  lastTransitionTime:
                    description: LastTransitionTime is the last time the condition
                      transitioned from one status to another.
                    format: date-time
                    type: string
                  message:
                    description: A human readable message indicating details about
                      the transition.
                    type: string
                  reason:
                    description: The reason for the condition's last transition.
                    type: string
                  severity:
                    description: Severity with which to treat failures of this type
                      of condition. When this is not specified, it defaults to Error.
                    type: string
                  status:
                    description: Status of the condition, one of True, False, Unknown.
                    type: string
                  type:
                    description: Type of condition.
                    type: string
                type: object
              type: array
            applicationVersion:
              description: Version of the WorkLoad Managed by the Operator TMC
              type: string
          type: object
      type: object
  version: v1alpha1
  versions:
  - name: v1alpha1
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
    tmc.cloud.vmware.com/do-not-delete: "true"
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extensionresourceowners.clusters.tmc.cloud.vmware.com
spec:
  group: clusters.tmc.cloud.vmware.com
  names:
    kind: ExtensionResourceOwner
    listKind: ExtensionResourceOwnerList
    plural: extensionresourceowners
    singular: extensionresourceowner
  scope: Cluster
  validation:
    openAPIV3Schema:
      description: ExtensionResourceOwner is the Schema for the extensionresourceowners
        API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: ExtensionResourceOwnerSpec defines the desired state of ExtensionResourceOwner
          type: object
        status:
          description: ExtensionResourceOwnerStatus defines the observed state of
            ExtensionResourceOwner
          type: object
      type: object
  version: v1alpha1
  versions:
  - name: v1alpha1
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
    tmc.cloud.vmware.com/do-not-delete: "true"
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    controller-tools.k8s.io: "1.0"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extensions.clusters.tmc.cloud.vmware.com
spec:
  additionalPrinterColumns:
  - JSONPath: .status.state
    name: State
    type: integer
  - JSONPath: .status.health
    name: Health
    type: integer
  - JSONPath: .status.version
    name: Version
    type: string
  group: clusters.tmc.cloud.vmware.com
  names:
    kind: Extension
    listKind: ExtensionList
    plural: extensions
    singular: extension
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      description: Extension is the Schema for the extensions API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: ExtensionSpec defines the desired state of Extension
          properties:
            deploymentStrategy:
              description: Deployment strategy of an extension.
              properties:
                extensionLifecycleOwner:
                  description: Component Owning Deployment Strategy of the Extension.
                    In case this field is empty it is assumed that Owner of Deployment
                    Strategy is Extension Manager
                  type: string
                overlapTimePeriod:
                  description: Time-Period within which an extension maybe rolled-back
                    to previous version in case the extension becomes unhealthy after
                    the new version is updated successfully. After this time-period
                    elapses, Extensions will not be rolled back to previous versions
                    if they become unhealthy. If the value is zero this field will
                    not be used during Extension Lifecycle Management.
                  format: int64
                  type: integer
                processingTimeout:
                  description: Timeout Value for Processing(Creating/Updating/Deleting/RollingBack)
                    an Extension.
                  format: int64
                  type: integer
                type:
                  description: Type of deployment for extension resource.
                  type: string
              required:
              - type
              type: object
            description:
              type: string
            imageRegistry:
              description: Image registry where the extension images resides.
              type: string
            name:
              type: string
            objects:
              description: Raw JSON/YAML of extension  equivalent to kubernetes 'Unstructured'
                type.
              type: string
            version:
              type: string
          required:
          - deploymentStrategy
          - name
          - objects
          - version
          type: object
        status:
          description: ExtensionStatus defines the observed state of Extension
          properties:
            applicationStatus:
              properties:
                conditions:
                  items:
                    properties:
                      lastTransitionTime:
                        format: date-time
                        type: string
                      message:
                        description: A human readable message indicating details about
                          the transition.
                        type: string
                      reason:
                        description: The reason for the condition's last transition.
                        type: string
                      severity:
                        description: Severity with which to treat failures of this
                          type of condition. When this is not specified, it defaults
                          to Error.
                        type: string
                      status:
                        description: Status of the condition, one of True, False,
                          Unknown.
                        type: string
                      type:
                        description: Type of condition.
                        type: string
                    required:
                    - status
                    - type
                    type: object
                  type: array
                version:
                  type: string
              required:
              - version
              type: object
            health:
              format: int32
              type: integer
            previousVersion:
              type: string
            state:
              format: int32
              type: integer
            status:
              format: int32
              type: integer
            version:
              type: string
          type: object
      type: object
  version: v1alpha1
  versions:
  - name: v1alpha1
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-manager
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extension-manager
  namespace: '{{.Namespace}}'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    app: extension-manager
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extension-manager-role
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    app: extension-manager
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extension-manager-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: extension-manager-role
subjects:
- kind: ServiceAccount
  name: extension-manager
  namespace: '{{.Namespace}}'
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-manager
    control-plane: extension-manager
    controller-tools.k8s.io: "1.0"
    tmc-extension: "true"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extension-manager-service
  namespace: '{{.Namespace}}'
spec:
  ports:
  - port: 443
  selector:
    control-plane: extension-manager
    controller-tools.k8s.io: "1.0"
    tmc-extension: "true"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-manager
    control-plane: extension-manager
    controller-tools.k8s.io: "1.0"
    tmc-extension: "true"
    tmc-extension-name: extension-manager
    tmc.cloud.vmware.com/managed: "true"
  name: extension-manager
  namespace: '{{.Namespace}}'
spec:
  minReadySeconds: 30
  progressDeadlineSeconds: 600
  replicas: 1
  selector:
    matchLabels:
      control-plane: extension-manager
      controller-tools.k8s.io: "1.0"
      tmc-extension: "true"
  strategy:
    rollingUpdate:
      maxSurge: 100%
  template:
    metadata:
      labels:
        control-plane: extension-manager
        controller-tools.k8s.io: "1.0"
        tmc-extension: "true"
        tmc-extension-name: extension-manager
    spec:
      containers:
      - args:
        - --metrics-bind-port=0
        command:
        - /usr/local/bin/manager
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: CLUSTER_TYPE
          value: management
        - name: KUBERNETES_SERVICE_HOST
          value: 127.0.0.1
        - name: KUBERNETES_SERVICE_PORT
          value: "6443"
        image: vmware-docker-olympus-extensions.bintray.io/extension-manager/extension-manager@sha256:4c79eb0028762843399762a5831438f03bc4debffac8726ad3e80ee0618b339e
        imagePullPolicy: Always
        name: extension-manager
        resources:
          limits:
            cpu: 100m
            memory: 256Mi
          requests:
            cpu: 0
            memory: 128Mi
        securityContext:
          runAsGroup: 1000
          runAsUser: 10000
      hostNetwork: true
      nodeSelector:
        node-role.kubernetes.io/master: ""
      serviceAccountName: extension-manager
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/notReady
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/unreachable
        operator: Exists
      - effect: NoSchedule
        key: kubeadmNode
        operator: Equal
        value: master

---
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: extension-updater-serviceaccount
  namespace: '{{.Namespace}}'
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: vmware-system-tmc-agent-permissive
spec:
  fsGroup:
    rule: RunAsAny
  hostNetwork: true
  hostPorts:
  - max: 100
    min: 100
  privileged: false
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: vmware-system-tmc-psp-agent-permissive
rules:
- apiGroups:
  - policy
  resourceNames:
  - vmware-system-tmc-agent-permissive
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: extension-updater-clusterrole
rules:
- apiGroups:
  - security.openshift.io
  resourceNames:
  - nonroot
  resources:
  - securitycontextconstraints
  verbs:
  - use
- apiGroups:
  - clusters.tmc.cloud.vmware.com
  resources:
  - '*'
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - get
  - list
  - watch
  - update
- apiGroups:
  - ""
  resources:
  - nodes
  - pods
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - create
  - get
  - list
  - watch
  - delete
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create
  - get
  - list
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
  - update
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - get
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: vmware-system-tmc-psp-agent-permissive
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vmware-system-tmc-psp-agent-permissive
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:{{.Namespace}}
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: extension-updater-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: extension-updater-clusterrole
subjects:
- kind: ServiceAccount
  name: extension-updater-serviceaccount
  namespace: '{{.Namespace}}'
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: extension-updater
    tmc-extension-name: extension-updater
    tmc.cloud.vmware.com/managed: "true"
  name: extension-updater
  namespace: '{{.Namespace}}'
spec:
  minReadySeconds: 30
  progressDeadlineSeconds: 600
  replicas: 1
  selector:
    matchLabels:
      app: extension-updater
      component: extension-updater
  strategy:
    rollingUpdate:
      maxSurge: 100%
  template:
    metadata:
      labels:
        app: extension-updater
        component: extension-updater
        tmc-extension-name: extension-updater
    spec:
      containers:
      - args:
        - --server=$(TMC_HOST):443
        - --server-name=$(TMC_HOST)
        - --tmc-ca=/etc/tmc/ca.crt
        - --cluster-id=$(CLUSTER_RESOURCE_ID)
        - --management-cluster-name=$(MANAGEMENT_CLUSTER_NAME)
        - --connect-timeout=100s
        - --poll-interval=5m
        - --poll-jitter=0.3
        - --agent-heart-beat-interval=90s
        - --validation-grpc-port=0
        - --metrics-bind-port=0
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: TMC_HOST
          valueFrom:
            configMapKeyRef:
              key: tmc_host
              name: stack-config
        - name: CLUSTER_RESOURCE_ID
          valueFrom:
            configMapKeyRef:
              key: cluster_rid
              name: stack-config
        - name: CLUSTER_TYPE
          value: supervisor
        - name: MANAGEMENT_CLUSTER_NAME
          valueFrom:
            configMapKeyRef:
              key: management_cluster_name
              name: stack-config
        - name: KUBERNETES_SERVICE_HOST
          value: 127.0.0.1
        - name: KUBERNETES_SERVICE_PORT
          value: "6443"
        image: vmware-docker-olympus-extensions.bintray.io/extension-updater/extension-updater@sha256:6d0e950d3e647f2747b74de58297d8678b3161d7e44e08a063d647ce3043bed8
        imagePullPolicy: Always
        name: extension-updater
        resources:
          limits:
            cpu: 100m
            memory: 256Mi
          requests:
            cpu: 0
            memory: 128Mi
        securityContext:
          runAsGroup: 1000
          runAsUser: 10000
        volumeMounts:
        - mountPath: /etc/tmc
          name: tmc-root-ca
          readOnly: true
      hostNetwork: true
      nodeSelector:
        node-role.kubernetes.io/master: ""
      serviceAccountName: extension-updater-serviceaccount
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/notReady
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/unreachable
        operator: Exists
      - effect: NoSchedule
        key: kubeadmNode
        operator: Equal
        value: master
      volumes:
      - configMap:
          items:
          - key: tls.crt
            path: ca.crt
          name: stack-config
        name: tmc-root-ca

---
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: agent-updater
    tmc-extension-name: agent-updater
    tmc.cloud.vmware.com/managed: "true"
  name: agent-updater
  namespace: '{{.Namespace}}'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    app: agent-updater
    tmc-extension-name: agent-updater
    tmc.cloud.vmware.com/managed: "true"
  name: agent-updater-role
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  creationTimestamp: null
  labels:
    app: agent-updater
    tmc-extension-name: agent-updater
    tmc.cloud.vmware.com/managed: "true"
  name: agent-updater-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: agent-updater-role
subjects:
- kind: ServiceAccount
  name: agent-updater
  namespace: '{{.Namespace}}'
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: agent-updater
    component: agent-updater
    tmc-extension: "true"
    tmc-extension-name: agent-updater
    tmc.cloud.vmware.com/managed: "true"
  name: agent-updater
  namespace: '{{.Namespace}}'
spec:
  minReadySeconds: 30
  progressDeadlineSeconds: 600
  replicas: 1
  selector:
    matchLabels:
      component: agent-updater
      tmc-extension: "true"
      tmc-extension-name: agent-updater
  strategy:
    rollingUpdate:
      maxSurge: 100%
  template:
    metadata:
      labels:
        app: agent-updater
        component: agent-updater
        tmc-extension: "true"
        tmc-extension-name: agent-updater
    spec:
      containers:
      - args:
        - --metrics-bind-port=0
        command:
        - /usr/local/bin/manager
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: KUBERNETES_SERVICE_HOST
          value: 127.0.0.1
        - name: KUBERNETES_SERVICE_PORT
          value: "6443"
        image: vmware-docker-olympus-extensions.bintray.io/agent-updater/agent-updater@sha256:db6771f78ed45f1648295571a7c26d3bff1dc90f6ca681fcd1f54791b1a27b41
        imagePullPolicy: Always
        name: agent-updater
        resources:
          limits:
            cpu: 100m
            memory: 150Mi
          requests:
            cpu: 0
            memory: 100Mi
        securityContext:
          runAsGroup: 1000
          runAsUser: 10000
      hostNetwork: true
      nodeSelector:
        node-role.kubernetes.io/master: ""
      serviceAccountName: agent-updater
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/notReady
        operator: Exists
      - effect: NoExecute
        key: node.alpha.kubernetes.io/unreachable
        operator: Exists
      - effect: NoSchedule
        key: kubeadmNode
        operator: Equal
        value: master
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  annotations:
    tmc.cloud.vmware.com/orphan-resource: "true"
  labels:
    app: agent-updater
    component: agentupdater-workload
    tmc-extension: "true"
    tmc-extension-name: agent-updater
    tmc.cloud.vmware.com/managed: "true"
  name: agentupdater-workload
  namespace: '{{.Namespace}}'
spec:
  concurrencyPolicy: Forbid
  jobTemplate:
    metadata:
      labels:
        component: agentupdater-workload
        tmc-extension-name: agent-updater
        tmc.cloud.vmware.com/managed: "true"
    spec:
      template:
        metadata:
          labels:
            tmc-extension-name: agent-updater
            tmc.cloud.vmware.com/managed: "true"
        spec:
          containers:
          - command:
            - /usr/local/bin/agentupdaterworkload
            env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: KUBERNETES_SERVICE_HOST
              value: 127.0.0.1
            - name: KUBERNETES_SERVICE_PORT
              value: "6443"
            image: vmware-docker-olympus-extensions.bintray.io/agent-updater/agentupdater-workload@sha256:82e09f185abdf818a4198c0109d8de5c53ff2ab255b1e4e70fe3efb43901bc40
            imagePullPolicy: IfNotPresent
            name: agentupdater-workload
            resources:
              limits:
                cpu: 100m
                memory: 256Mi
              requests:
                cpu: 0
                memory: 128Mi
            securityContext:
              runAsGroup: 1000
              runAsUser: 10000
          hostNetwork: true
          nodeSelector:
            node-role.kubernetes.io/master: ""
          restartPolicy: Never
          serviceAccountName: agent-updater
          tolerations:
          - effect: NoSchedule
            key: node-role.kubernetes.io/master
            operator: Exists
          - key: CriticalAddonsOnly
            operator: Exists
          - effect: NoExecute
            key: node.alpha.kubernetes.io/notReady
            operator: Exists
          - effect: NoExecute
            key: node.alpha.kubernetes.io/unreachable
            operator: Exists
          - effect: NoSchedule
            key: kubeadmNode
            operator: Equal
            value: master
  schedule: '*/1 * * * *'
  startingDeadlineSeconds: 180

---

As for the AgentInstaller object, the following sample yaml file shows what I created to make use of the generated registration URL.

apiVersion: installers.tmc.cloud.vmware.com/v1alpha1
kind: AgentInstall
metadata:
  name: tmc-agent-installer-config
  namespace: svc-tmc-c8
spec:
  operation: INSTALL
  registrationLink: https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registration

Note that the namespace is svc-tmc-c8, the value noted earlier and the registrationLink is https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registration, which was taken from the TMC UI. Everything else in here would be the same for any supervisor cluster and TMC org.

This can now be installed by running kubectl create -f registration.yaml. You should see the following noting that the process has been started:

agentinstall.installers.tmc.cloud.vmware.com/tmc-agent-installer-config created

You'll be able to tell when the installation is finished by inspecting the status of the AgentInstall object that was created.

kubectl -n svc-tmc-c8 describe agentinstall tmc-agent-installer-config
Name:         tmc-agent-installer-config
Namespace:    svc-tmc-c8
Labels:       <none>
Annotations:  <none>
API Version:  installers.tmc.cloud.vmware.com/v1alpha1
Kind:         AgentInstall
Metadata:
  Creation Timestamp:  2020-12-04T15:14:14Z
  Generation:          1
  Managed Fields:
    API Version:  installers.tmc.cloud.vmware.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:operation:
        f:registrationLink:
    Manager:      kubectl-create
    Operation:    Update
    Time:         2020-12-04T15:14:14Z
    API Version:  installers.tmc.cloud.vmware.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:message:
        f:status:
    Manager:         tmc-agent-installer
    Operation:       Update
    Time:            2020-12-04T15:15:26Z
  Resource Version:  1135301
  Self Link:         /apis/installers.tmc.cloud.vmware.com/v1alpha1/namespaces/svc-tmc-c8/agentinstalls/tmc-agent-installer-config
  UID:               467b0b70-2b2c-4561-b3b3-cd68749fb254
Spec:
  Operation:          INSTALL
  Registration Link:  https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registration
Status:
  Message:  successfully applied the registration link
  Status:   INSTALLED
Events:     <none>

Note that the Status is Installed, indicating a successful installation. If we take a look in the svc-tmc-c8 namespace again, we'll see that there are a few more pods present.

kubectl -n svc-tmc-c8 get po

NAME                                          READY   STATUS      RESTARTS   AGE
agent-updater-7fbb98485f-xsr6x                1/1     Running     0          8m46s
agentupdater-workload-1607095440-92g2l        0/1     Completed   0          10s
cluster-health-extension-59b9dd89f7-hr6x7     1/1     Running     0          5m22s
extension-manager-dd949bc96-4btzn             1/1     Running     0          8m48s
extension-updater-588fff5467-77bk2            1/1     Running     0          8m46s
intent-agent-5c8cd75b57-9c2nl                 1/1     Running     0          5m22s
sync-agent-56559c797b-k8k2t                   1/1     Running     0          5m22s
tmc-agent-installer-1607095380-zvbh8          0/1     Completed   0          68s
tmc-agent-installer-1607095440-z92q7          1/1     Running     0          2s
tmc-auto-attach-6f8df84468-qxkzd              1/1     Running     0          5m22s
vsphere-resource-retriever-7dc6b9546c-92jgg   1/1     Running     3          5m22s

Back in the TMC UI, you can click on the View Management Cluster button to be taken to the details for the now-attached supervisor cluster.

And if you click on the Workload clusters tab, you'll see that there are currently none present.

To finally get to my main goal, adding a Tanzu Kubernetes cluster to a vSphere 7 with Tanzu installation via TMC, we can navigate back to the Clusters page and then click the Create Cluster dropdown (note that this used to be a single button).

We've now got a new option for creating a cluster via the TKG Service included with vSphere 7 with Tanzu. Click on Tanzu Kubernetes Grid Service on vSphere 7.

Choose the recently added supervisor cluster (cjl-vsphere7) and the default provisioner (tkg, the namespace in my supervisor cluster). Click the Next button.

Enter a name for the Tanzu Kubernetes cluster and select the appropriate cluster group. You can enter any optional information that you would like also. Click the Next button.

Much of the information here is dependent on the configuration in the vSphere 7 with Tanzu installation. For example, you can see that the Kubernetes version list corresponds to the images present in the Content Library.

I've chosen the latest Kubernetes version, left the Pod and Service CIDRs at their default values and chosen k8s-policy (my only K8s-based storage policy) for the storage class. Click the Next button.

Choose whether you want a single node cluster or a highly available (multiple nodes) cluster and adjust the size of your nodes as appropriate. I'm going with a single node cluster with best-effort-xsmall size nodes since I'm incredibly starved for resources. Click the Next button.

On the node pool page, you'll need to provide information on how the node pool for this cluster should be configured. I've chosen to go with the same settings as were specified for the control plane nodes (single and best-effort-xsmall). Click the Create Cluster button.

Back in the vSphere Client, you'll see that there is now a cluster (cjl-test-tkc) present and a control plane node is being created.

You can also see from the command line that the Tanzu Kubernetes cluster is being created.

kubectl get tkc

NAME           CONTROL PLANE   WORKER   DISTRIBUTION                     AGE    PHASE
cjl-test-tkc   1               1        v1.18.5+vmware.1-tkg.1.c40d30d   3m5s   creating

Once the control plane node is powered-on and configured, you'll see the worker node being deployed.

kubectl get virtualmachines

NAME                                         AGE
cjl-test-tkc-control-plane-7g4jw             22m
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6   2m1s
kubectl get machines

NAME                                         PROVIDERID                                       PHASE
cjl-test-tkc-control-plane-7g4jw             vsphere://422a7563-65ae-181e-0baa-3f79787010cd   Running
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6                                                    Provisioning

And back in the TMC UI, you'll see much more information about the cluster now that the control plane node is up and running.

When the worker node is done and the cluster is up and functional, there are a few places we can check to validate it.

kubectl get tkc

NAME           CONTROL PLANE   WORKER   DISTRIBUTION                     AGE   PHASE
cjl-test-tkc   1               1        v1.18.5+vmware.1-tkg.1.c40d30d   40m   running

kubectl get machines

NAME                                         PROVIDERID                                       PHASE
cjl-test-tkc-control-plane-7g4jw             vsphere://422a7563-65ae-181e-0baa-3f79787010cd   Running
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6   vsphere://422a545c-4b09-12bd-6633-dcc026fece3c   Running

And if you navigate back to Administration, Management Clusters and drill down into the newly-registered cjl-vsphere7 cluster, you can see that there is a workload cluster present now.

In keeping with the standard means of accessing a Tanzu Kubernetes cluster running in vSphere 7 with Tanzu, you can use a modified kubectl vsphere login command (passing in the namespace and cluster names), similar to the following.

kubectl vsphere login --server=192.168.221.2 -u administrator@vsphere.local --tanzu-kubernetes-cluster-name cjl-test-tkc --tanzu-kubernetes-cluster-namespace tkg

Password:
Logged in successfully.

You have access to the following contexts:
   192.168.221.2
   cjl-test-tkc
   tkg

If the context you wish to use is not in this list, you may need to try
logging in again later, or contact your cluster administrator.

To change context, use `kubectl config use-context <workload name>`

However, you can also make use of the familiar functionality in TMC for getting access to clusters provisioned on AWS. There is an Access this cluster button under the Actions dropdown that you can use to create a kubeconfig file.

After you click the Access this cluster button, you'll be presented with a page containing a link to the kubeconfig file and a sample command for using it.

You need to have the tmc executable installed (the latest version) where you will be using this kubeconfig file and have logged in with it. If you don't have an API token created already, you can do so at the link noted. You'll also need to provide the Management Cluster Name and Provisioner Name as they show up in the TMC UI.

tmc login

i If you don't have an API token, visit the VMware Cloud Services console, select your organization, and create an API token with the TMC service roles:
  https://console.cloud.vmware.com/csp/gateway/portal/#/user/tokens
? API Token ****************************************************************
? Login context name cjl-test-tkc
? Select default log level info
? Management Cluster Name cjl-vsphere7
? Provisioner Name tkg
√ Successfully created context cjl-test-tkc, to manage your contexts run `tmc system context -h`

And finally, we can use the kubeconfig file generated by TMC to access the new cluster.

kubectl --kubeconfig=kubeconfig-cjl-test-tkc.yml get ns

NAME                           STATUS   AGE
default                        Active   84m
kube-node-lease                Active   84m
kube-public                    Active   84m
kube-system                    Active   84m
vmware-system-auth             Active   83m
vmware-system-cloud-provider   Active   82m
vmware-system-csi              Active   82m
vmware-system-tmc              Active   82m

It's worth noting that the first time I tried to use the kubeconfig file, I got the following error...

x could not login: Post https://192.168.221.3:6443/apis/login.concierge.pinniped.dev/v1alpha1/namespaces/vmware-system-tmc/tokencredentialrequests: context deadline exceeded

This only happened the one time and every subsequent kubectl command worked flawlessly....looks like it was just a simple timing issue on that first go at it.

2 thoughts on “Attaching a vSphere 7.0 with Tanzu supervisor cluster to Tanzu Mission Control and creating new Tanzu Kubernetes clusters”

  1. Pingback: Provision and upgrade WCP clusters via TMC – Little Stuff

  2. Pingback: Installing Tanzu Kubernetes Grid 1.3 on vSphere with NSX Advanced Load Balancer – Little Stuff

Leave a Comment

Your email address will not be published. Required fields are marked *