Adding a Tanzu Kubernetes cluster to TMC management as an attached cluster has been possible for quite a while. You didn’t have the same level of control as you would for a cluster created by TMC on AWS but it at least provided some visibility and the policy functionality provided by TMC. As of yesterday, we now have the ability to registry a vSphere 7 with Tanzu supervisor cluster in TMC and then provision Tanzu Kubernetes clusters directly from TMC. This is a huge milestone for both TMC and vSphere 7 with Tanzu and I was incredibly excited to try it out. You can read many more details on this process at Managing the Lifecycle of Tanzu Kubernetes Clusters and How to register/deregister a vSphere with Tanzu supervisor cluster in Tanzu Mission Control (80727) but keep reading below to see my brief walkthrough of the process.
I started out with a simple supervisor cluster and a single namespace named tkg (using vDS networking). You can read more about setting up vSphere 7.0 with Tanzu in my previous post, How to install vSphere with Tanzu with vSphere Networking.
You can see that there are no Tanzu Kubernetes clusters created yet.
Within the TMC UI, there is a new section under Administration called Management clusters.
Be default, you’ll see the the aws-hosted management cluster which is used for provisioning Tanzu Kubernetes clusters on AWS. This was previously hidden from view in earlier versions of TMC. If you drill down into this cluster you can see some details about the accounts in your org that have been used for creating clusters on AWS.
I want to add a supervisor cluster to the recognized management clusters so that I can use TMC to create Tanzu Kubernetes clusters in my vSphere 7 with Tanzu environment. Back on the Management clusters page, click on the Register Management Cluster button.
You only need to provide a name and cluster group here but can provide optional information if desired. Click on the Next button.
At this point, you need to copy the the registration URL as it will be used shortly.
While logged in to the supervisor cluster (via the kubectl vsphere login command), run the kubectl get ns command to get a listing of the available namespaces.
NAME STATUS AGE
default Active 71d
kube-node-lease Active 71d
kube-public Active 71d
kube-system Active 71d
svc-tmc-c8 Active 71d
tkg Active 71d
vmware-system-appplatform-operator-system Active 71d
vmware-system-capw Active 71d
vmware-system-cert-manager Active 71d
vmware-system-csi Active 71d
vmware-system-kubeimage Active 71d
vmware-system-lbapi Active 71d
vmware-system-license-operator Active 71d
vmware-system-netop Active 71d
vmware-system-registry Active 71d
vmware-system-tkg Active 71d
vmware-system-ucs Active 71d
vmware-system-vmop Active 71d
The namespace with which we’ll be working is svc-tmc-c8. This namespace will always start with svc-tmc- and it should be the only one present, making it fairly easy to identify. If you look in this namespace, you’ll see that there isn’t much going on there right now.
kubectl -n svc-tmc-c8 get po
NAME READY STATUS RESTARTS AGE
tmc-agent-installer-1607094720-x9zj7 0/1 Completed 0 46s
A new AgentInstaller object is needed in the supervisor cluster to allow TMC to communicate with it. This is a CRD that will reference the registration URL generated earlier and create all of the resources that are needed for TMC functionality. You can take a closer look at what is being created in this sample.
registration URL YAML
apiVersion: v1
kind: Namespace
metadata:
labels:
control-plane: extension-manager
tmc-extension: "true"
controller-tools.k8s.io: "1.0"
name: {{.Namespace}}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: stack-config
namespace: {{.Namespace}}
labels:
tmc.cloud.vmware.com/managed: "true"
data:
resource_uid: "mc:01ERQ3VX1VDAMNK6ZPA1XHV6WD"
org_id: "e6f8b4af-faa2-4b55-8403-97d2d6b19341"
management_cluster_name: "cjl-vsphere7"
cluster_rid: "rid:mc:e6f8b4af-faa2-4b55-8403-97d2d6b19341:cjl-vsphere7"
tmc_url: https://epsg.tmc.cloud.vmware.com
tmc_host: epsg.tmc.cloud.vmware.com
tls.crt: |+
-----BEGIN CERTIFICATE-----
MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow
PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD
Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O
rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq
OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b
xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw
7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD
aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV
HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG
SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69
ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr
AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz
R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5
JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo
Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDxTCCAq2gAwIBAgIQAqxcJmoLQJuPC3nyrkYldzANBgkqhkiG9w0BAQUFADBs
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
ZSBFViBSb290IENBMB4XDTA2MTExMDAwMDAwMFoXDTMxMTExMDAwMDAwMFowbDEL
MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
LmRpZ2ljZXJ0LmNvbTErMCkGA1UEAxMiRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug
RVYgUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMbM5XPm
+9S75S0tMqbf5YE/yc0lSbZxKsPVlDRnogocsF9ppkCxxLeyj9CYpKlBWTrT3JTW
PNt0OKRKzE0lgvdKpVMSOO7zSW1xkX5jtqumX8OkhPhPYlG++MXs2ziS4wblCJEM
xChBVfvLWokVfnHoNb9Ncgk9vjo4UFt3MRuNs8ckRZqnrG0AFFoEt7oT61EKmEFB
Ik5lYYeBQVCmeVyJ3hlKV9Uu5l0cUyx+mM0aBhakaHPQNAQTXKFx01p8VdteZOE3
hzBWBOURtCmAEvF5OYiiAhF8J2a3iLd48soKqDirCmTCv2ZdlYTBoSUeh10aUAsg
EsxBu24LUTi4S8sCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQF
MAMBAf8wHQYDVR0OBBYEFLE+w2kD+L9HAdSYJhoIAu9jZCvDMB8GA1UdIwQYMBaA
FLE+w2kD+L9HAdSYJhoIAu9jZCvDMA0GCSqGSIb3DQEBBQUAA4IBAQAcGgaX3Nec
nzyIZgYIVyHbIUf4KmeqvxgydkAQV8GK83rZEWWONfqe/EW1ntlMMUu4kehDLI6z
eM7b41N5cdblIZQB2lWHmiRk9opmzN6cN82oNLFpmyPInngiK3BD41VHMWEZ71jF
hS9OMPagMRYjyOfiZRYzy78aG6A9+MpeizGLYAiJLQwGXFK3xPkKmNEVX58Svnw2
Yzi9RKR/5CYrCsSXaQ3pjOLAEFe4yHYSkVXySGnYvCoCWw9E1CAx2/S6cCZdkGCe
vEsXCS+0yx5DaMkHJ8HSXPfqIbloEpw8nL+e/IBcm2PN7EeqJSdnoDfzAIJ9VNep
+OkuE6N36B9K
-----END CERTIFICATE-----
---
apiVersion: v1
kind: Secret
metadata:
name: tmc-access-secret
namespace: {{.Namespace}}
labels:
tmc.cloud.vmware.com/managed: "true"
type: Opaque
data:
access_token_info : "ZXlKaFkyTmxjM05mZEc5clpXNGlPaUoyTVdGblpXNTBMbkJUYTFWeE5WWk5VVkF0VkY4NFkzTnJWSEJ2YzFwMU5tZDFOVTVuVUV3MGQzSlVSbUZ3YkVoRVVVUkJPRFZDY0dSUlZXa3dVVm8wZEdaUFMwYzVjalVpZlE9PQ=="
---
---
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
tmc.cloud.vmware.com/do-not-delete: "true"
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
controller-tools.k8s.io: "1.0"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: agents.clusters.tmc.cloud.vmware.com
spec:
additionalPrinterColumns:
- JSONPath: .status.status
name: Status
type: integer
- JSONPath: .status.health
name: Health
type: integer
group: clusters.tmc.cloud.vmware.com
names:
kind: Agent
listKind: AgentList
plural: agents
singular: agent
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: Agent is the Schema for the agents API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: AgentSpec defines the desired state of Agent
properties:
detach:
type: boolean
extensions:
items:
type: string
type: array
namespace:
type: string
type: object
status:
description: AgentStatus defines the observed state of Agent
properties:
clusterHealth:
description: AggregatedClusterHealth defines the observed state of the
cluster.
properties:
controllerManagerHealth:
description: ComponentHealth defines the health of a component.
properties:
health:
format: int32
type: integer
message:
type: string
name:
type: string
required:
- name
type: object
etcdHealth:
items:
description: ComponentHealth defines the health of a component.
properties:
health:
format: int32
type: integer
message:
type: string
name:
type: string
required:
- name
type: object
type: array
message:
type: string
schedulerHealth:
description: ComponentHealth defines the health of a component.
properties:
health:
format: int32
type: integer
message:
type: string
name:
type: string
required:
- name
type: object
timestamp:
description: "A Timestamp represents a point in time independent
of any time zone or calendar, represented as seconds and fractions
of seconds at nanosecond resolution in UTC Epoch time. It is encoded
using the Proleptic Gregorian Calendar which extends the Gregorian
calendar backwards to year one. It is encoded assuming all minutes
are 60 seconds long, i.e. leap seconds are \"smeared\" so that
no leap second table is needed for interpretation. Range is from
0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting
to that range, we ensure that we can convert to and from RFC
3339 date strings. See [https://www.ietf.org/rfc/rfc3339.txt](https://www.ietf.org/rfc/rfc3339.txt).
\n # Examples \n Example 1: Compute Timestamp from POSIX `time()`.
\n Timestamp timestamp; timestamp.set_seconds(time(NULL));
\ timestamp.set_nanos(0); \n Example 2: Compute Timestamp from
POSIX `gettimeofday()`. \n struct timeval tv; gettimeofday(&tv,
NULL); \n Timestamp timestamp; timestamp.set_seconds(tv.tv_sec);
\ timestamp.set_nanos(tv.tv_usec * 1000); \n Example 3: Compute
Timestamp from Win32 `GetSystemTimeAsFileTime()`. \n FILETIME
ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime)
<< 32) | ft.dwLowDateTime; \n // A Windows tick is 100 nanoseconds.
Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds
before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp;
\ timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL));
\ timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));
\n Example 4: Compute Timestamp from Java `System.currentTimeMillis()`.
\n long millis = System.currentTimeMillis(); \n Timestamp
timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int)
((millis % 1000) * 1000000)).build(); \n Example 5: Compute Timestamp
from current time in Python. \n timestamp = Timestamp() timestamp.GetCurrentTime()
\n # JSON Mapping \n In JSON format, the Timestamp type is encoded
as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt)
format. That is, the format is \"{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z\"
where {year} is always expressed using four digits while {month},
{day}, {hour}, {min}, and {sec} are zero-padded to two digits
each. The fractional seconds, which can go up to 9 digits (i.e.
up to 1 nanosecond resolution), are optional. The \"Z\" suffix
indicates the timezone (\"UTC\"); the timezone is required. A
proto3 JSON serializer should always use UTC (as indicated by
\"Z\") when printing the Timestamp type and a proto3 JSON parser
should be able to accept both UTC and other timezones (as indicated
by an offset). \n For example, \"2017-01-15T01:30:15.01Z\" encodes
15.01 seconds past 01:30 UTC on January 15, 2017. \n In JavaScript,
one can convert a Date object to this format using the standard
[toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString]
method. In Python, a standard `datetime.datetime` object can be
converted to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime)
with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in
Java, one can use the Joda Time's [`ISODateTimeFormat.dateTime()`](
http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--
) to obtain a formatter capable of generating timestamps in this
format."
properties:
nanos:
description: Non-negative fractions of a second at nanosecond
resolution. Negative second values with fractions must still
have non-negative nanos values that count forward in time.
Must be from 0 to 999,999,999 inclusive.
format: int32
type: integer
seconds:
description: Represents seconds of UTC time since Unix epoch
1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
9999-12-31T23:59:59Z inclusive.
format: int64
type: integer
type: object
type: object
deploymentLink:
type: string
extensions:
items:
type: string
type: array
health:
format: int32
type: integer
metadata:
properties:
cloudProvider:
format: int32
type: integer
clusterCPU:
description: ResourceAllocation defines the resource utilisation
and availability.
properties:
allocatable:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
allocatedPercentage:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
requests:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
units:
type: string
required:
- allocatable
- allocatedPercentage
- requests
- units
type: object
clusterMemory:
description: ResourceAllocation defines the resource utilisation
and availability.
properties:
allocatable:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
allocatedPercentage:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
requests:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
units:
type: string
required:
- allocatable
- allocatedPercentage
- requests
- units
type: object
kubeServerVersion:
type: string
kubernetesProvider:
properties:
type:
format: int32
type: integer
version:
type: string
type: object
lastUpdate:
description: "A Timestamp represents a point in time independent
of any time zone or calendar, represented as seconds and fractions
of seconds at nanosecond resolution in UTC Epoch time. It is encoded
using the Proleptic Gregorian Calendar which extends the Gregorian
calendar backwards to year one. It is encoded assuming all minutes
are 60 seconds long, i.e. leap seconds are \"smeared\" so that
no leap second table is needed for interpretation. Range is from
0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting
to that range, we ensure that we can convert to and from RFC
3339 date strings. See [https://www.ietf.org/rfc/rfc3339.txt](https://www.ietf.org/rfc/rfc3339.txt).
\n # Examples \n Example 1: Compute Timestamp from POSIX `time()`.
\n Timestamp timestamp; timestamp.set_seconds(time(NULL));
\ timestamp.set_nanos(0); \n Example 2: Compute Timestamp from
POSIX `gettimeofday()`. \n struct timeval tv; gettimeofday(&tv,
NULL); \n Timestamp timestamp; timestamp.set_seconds(tv.tv_sec);
\ timestamp.set_nanos(tv.tv_usec * 1000); \n Example 3: Compute
Timestamp from Win32 `GetSystemTimeAsFileTime()`. \n FILETIME
ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime)
<< 32) | ft.dwLowDateTime; \n // A Windows tick is 100 nanoseconds.
Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds
before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp;
\ timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL));
\ timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));
\n Example 4: Compute Timestamp from Java `System.currentTimeMillis()`.
\n long millis = System.currentTimeMillis(); \n Timestamp
timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int)
((millis % 1000) * 1000000)).build(); \n Example 5: Compute Timestamp
from current time in Python. \n timestamp = Timestamp() timestamp.GetCurrentTime()
\n # JSON Mapping \n In JSON format, the Timestamp type is encoded
as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt)
format. That is, the format is \"{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z\"
where {year} is always expressed using four digits while {month},
{day}, {hour}, {min}, and {sec} are zero-padded to two digits
each. The fractional seconds, which can go up to 9 digits (i.e.
up to 1 nanosecond resolution), are optional. The \"Z\" suffix
indicates the timezone (\"UTC\"); the timezone is required. A
proto3 JSON serializer should always use UTC (as indicated by
\"Z\") when printing the Timestamp type and a proto3 JSON parser
should be able to accept both UTC and other timezones (as indicated
by an offset). \n For example, \"2017-01-15T01:30:15.01Z\" encodes
15.01 seconds past 01:30 UTC on January 15, 2017. \n In JavaScript,
one can convert a Date object to this format using the standard
[toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString]
method. In Python, a standard `datetime.datetime` object can be
converted to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime)
with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in
Java, one can use the Joda Time's [`ISODateTimeFormat.dateTime()`](
http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--
) to obtain a formatter capable of generating timestamps in this
format."
properties:
nanos:
description: Non-negative fractions of a second at nanosecond
resolution. Negative second values with fractions must still
have non-negative nanos values that count forward in time.
Must be from 0 to 999,999,999 inclusive.
format: int32
type: integer
seconds:
description: Represents seconds of UTC time since Unix epoch
1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
9999-12-31T23:59:59Z inclusive.
format: int64
type: integer
type: object
masterNodeCount:
format: int64
type: integer
namespacesCount:
format: int64
type: integer
physicalMemory:
format: int64
type: integer
podCount:
format: int64
type: integer
region:
type: string
vcpuCount:
format: int64
type: integer
workerNodeCount:
format: int64
type: integer
type: object
status:
format: int32
type: integer
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
tmc.cloud.vmware.com/do-not-delete: "true"
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
controller-tools.k8s.io: "1.0"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extensionconfigs.intents.tmc.cloud.vmware.com
spec:
group: intents.tmc.cloud.vmware.com
names:
kind: ExtensionConfig
plural: extensionconfigs
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
configMaps:
description: ConfigMaps are the configMaps of the extension
items:
type: object
type: array
type: object
status:
properties:
state:
description: State indicates the state of the ExtensionConfig
type: string
type: object
version: v1alpha1
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.2.5
tmc.cloud.vmware.com/do-not-delete: "true"
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
app: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extensionintegrations.clusters.tmc.cloud.vmware.com
spec:
group: clusters.tmc.cloud.vmware.com
names:
kind: ExtensionIntegration
listKind: ExtensionIntegrationList
plural: extensionintegrations
singular: extensionintegration
scope: Namespaced
validation:
openAPIV3Schema:
description: ExtensionIntegration is the Schema for the extensionintegrations
API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ExtensionIntegrationSpec defines the desired state of ExtensionIntegration
properties:
extensionHealth:
description: Health of the Extension Workload that is managing the underlying
Application
format: int32
type: integer
extensionState:
description: Status of the Extension Workload that is managing the underlying
Application
format: int32
type: integer
type: object
status:
description: ExtensionIntegrationStatus defines the observed state of ExtensionIntegration
properties:
applicationConditions:
description: Conditions of the Extension that is integrated with TMC
items:
description: Conditions captures readiness and health conditions for
an underlying Application mapping to an extension. Collectively
they capture the set of conditions that define the state and health
of application.
properties:
lastTransitionTime:
description: LastTransitionTime is the last time the condition
transitioned from one status to another.
format: date-time
type: string
message:
description: A human readable message indicating details about
the transition.
type: string
reason:
description: The reason for the condition's last transition.
type: string
severity:
description: Severity with which to treat failures of this type
of condition. When this is not specified, it defaults to Error.
type: string
status:
description: Status of the condition, one of True, False, Unknown.
type: string
type:
description: Type of condition.
type: string
type: object
type: array
applicationVersion:
description: Version of the WorkLoad Managed by the Operator TMC
type: string
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
tmc.cloud.vmware.com/do-not-delete: "true"
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extensionresourceowners.clusters.tmc.cloud.vmware.com
spec:
group: clusters.tmc.cloud.vmware.com
names:
kind: ExtensionResourceOwner
listKind: ExtensionResourceOwnerList
plural: extensionresourceowners
singular: extensionresourceowner
scope: Cluster
validation:
openAPIV3Schema:
description: ExtensionResourceOwner is the Schema for the extensionresourceowners
API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ExtensionResourceOwnerSpec defines the desired state of ExtensionResourceOwner
type: object
status:
description: ExtensionResourceOwnerStatus defines the observed state of
ExtensionResourceOwner
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.1-0.20200617211605-651903477185
tmc.cloud.vmware.com/do-not-delete: "true"
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
controller-tools.k8s.io: "1.0"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extensions.clusters.tmc.cloud.vmware.com
spec:
additionalPrinterColumns:
- JSONPath: .status.state
name: State
type: integer
- JSONPath: .status.health
name: Health
type: integer
- JSONPath: .status.version
name: Version
type: string
group: clusters.tmc.cloud.vmware.com
names:
kind: Extension
listKind: ExtensionList
plural: extensions
singular: extension
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: Extension is the Schema for the extensions API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ExtensionSpec defines the desired state of Extension
properties:
deploymentStrategy:
description: Deployment strategy of an extension.
properties:
extensionLifecycleOwner:
description: Component Owning Deployment Strategy of the Extension.
In case this field is empty it is assumed that Owner of Deployment
Strategy is Extension Manager
type: string
overlapTimePeriod:
description: Time-Period within which an extension maybe rolled-back
to previous version in case the extension becomes unhealthy after
the new version is updated successfully. After this time-period
elapses, Extensions will not be rolled back to previous versions
if they become unhealthy. If the value is zero this field will
not be used during Extension Lifecycle Management.
format: int64
type: integer
processingTimeout:
description: Timeout Value for Processing(Creating/Updating/Deleting/RollingBack)
an Extension.
format: int64
type: integer
type:
description: Type of deployment for extension resource.
type: string
required:
- type
type: object
description:
type: string
imageRegistry:
description: Image registry where the extension images resides.
type: string
name:
type: string
objects:
description: Raw JSON/YAML of extension equivalent to kubernetes 'Unstructured'
type.
type: string
version:
type: string
required:
- deploymentStrategy
- name
- objects
- version
type: object
status:
description: ExtensionStatus defines the observed state of Extension
properties:
applicationStatus:
properties:
conditions:
items:
properties:
lastTransitionTime:
format: date-time
type: string
message:
description: A human readable message indicating details about
the transition.
type: string
reason:
description: The reason for the condition's last transition.
type: string
severity:
description: Severity with which to treat failures of this
type of condition. When this is not specified, it defaults
to Error.
type: string
status:
description: Status of the condition, one of True, False,
Unknown.
type: string
type:
description: Type of condition.
type: string
required:
- status
- type
type: object
type: array
version:
type: string
required:
- version
type: object
health:
format: int32
type: integer
previousVersion:
type: string
state:
format: int32
type: integer
status:
format: int32
type: integer
version:
type: string
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-manager
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extension-manager
namespace: '{{.Namespace}}'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
app: extension-manager
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extension-manager-role
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
app: extension-manager
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extension-manager-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: extension-manager-role
subjects:
- kind: ServiceAccount
name: extension-manager
namespace: '{{.Namespace}}'
---
apiVersion: v1
kind: Service
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-manager
control-plane: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension: "true"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extension-manager-service
namespace: '{{.Namespace}}'
spec:
ports:
- port: 443
selector:
control-plane: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension: "true"
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-manager
control-plane: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension: "true"
tmc-extension-name: extension-manager
tmc.cloud.vmware.com/managed: "true"
name: extension-manager
namespace: '{{.Namespace}}'
spec:
minReadySeconds: 30
progressDeadlineSeconds: 600
replicas: 1
selector:
matchLabels:
control-plane: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension: "true"
strategy:
rollingUpdate:
maxSurge: 100%
template:
metadata:
labels:
control-plane: extension-manager
controller-tools.k8s.io: "1.0"
tmc-extension: "true"
tmc-extension-name: extension-manager
spec:
containers:
- args:
- --metrics-bind-port=0
command:
- /usr/local/bin/manager
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CLUSTER_TYPE
value: management
- name: KUBERNETES_SERVICE_HOST
value: 127.0.0.1
- name: KUBERNETES_SERVICE_PORT
value: "6443"
image: vmware-docker-olympus-extensions.bintray.io/extension-manager/extension-manager@sha256:4c79eb0028762843399762a5831438f03bc4debffac8726ad3e80ee0618b339e
imagePullPolicy: Always
name: extension-manager
resources:
limits:
cpu: 100m
memory: 256Mi
requests:
cpu: 0
memory: 128Mi
securityContext:
runAsGroup: 1000
runAsUser: 10000
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/master: ""
serviceAccountName: extension-manager
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/notReady
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/unreachable
operator: Exists
- effect: NoSchedule
key: kubeadmNode
operator: Equal
value: master
---
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: extension-updater-serviceaccount
namespace: '{{.Namespace}}'
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: vmware-system-tmc-agent-permissive
spec:
fsGroup:
rule: RunAsAny
hostNetwork: true
hostPorts:
- max: 100
min: 100
privileged: false
runAsUser:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: vmware-system-tmc-psp-agent-permissive
rules:
- apiGroups:
- policy
resourceNames:
- vmware-system-tmc-agent-permissive
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: extension-updater-clusterrole
rules:
- apiGroups:
- security.openshift.io
resourceNames:
- nonroot
resources:
- securitycontextconstraints
verbs:
- use
- apiGroups:
- clusters.tmc.cloud.vmware.com
resources:
- '*'
verbs:
- '*'
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- get
- list
- watch
- update
- apiGroups:
- ""
resources:
- nodes
- pods
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- namespaces
verbs:
- create
- get
- list
- watch
- delete
- apiGroups:
- ""
resources:
- secrets
verbs:
- create
- get
- list
- update
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- batch
resources:
- cronjobs
- jobs
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- deployments
verbs:
- get
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: vmware-system-tmc-psp-agent-permissive
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: vmware-system-tmc-psp-agent-permissive
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts:{{.Namespace}}
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: extension-updater-clusterrolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: extension-updater-clusterrole
subjects:
- kind: ServiceAccount
name: extension-updater-serviceaccount
namespace: '{{.Namespace}}'
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: extension-updater
tmc-extension-name: extension-updater
tmc.cloud.vmware.com/managed: "true"
name: extension-updater
namespace: '{{.Namespace}}'
spec:
minReadySeconds: 30
progressDeadlineSeconds: 600
replicas: 1
selector:
matchLabels:
app: extension-updater
component: extension-updater
strategy:
rollingUpdate:
maxSurge: 100%
template:
metadata:
labels:
app: extension-updater
component: extension-updater
tmc-extension-name: extension-updater
spec:
containers:
- args:
- --server=$(TMC_HOST):443
- --server-name=$(TMC_HOST)
- --tmc-ca=/etc/tmc/ca.crt
- --cluster-id=$(CLUSTER_RESOURCE_ID)
- --management-cluster-name=$(MANAGEMENT_CLUSTER_NAME)
- --connect-timeout=100s
- --poll-interval=5m
- --poll-jitter=0.3
- --agent-heart-beat-interval=90s
- --validation-grpc-port=0
- --metrics-bind-port=0
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: TMC_HOST
valueFrom:
configMapKeyRef:
key: tmc_host
name: stack-config
- name: CLUSTER_RESOURCE_ID
valueFrom:
configMapKeyRef:
key: cluster_rid
name: stack-config
- name: CLUSTER_TYPE
value: supervisor
- name: MANAGEMENT_CLUSTER_NAME
valueFrom:
configMapKeyRef:
key: management_cluster_name
name: stack-config
- name: KUBERNETES_SERVICE_HOST
value: 127.0.0.1
- name: KUBERNETES_SERVICE_PORT
value: "6443"
image: vmware-docker-olympus-extensions.bintray.io/extension-updater/extension-updater@sha256:6d0e950d3e647f2747b74de58297d8678b3161d7e44e08a063d647ce3043bed8
imagePullPolicy: Always
name: extension-updater
resources:
limits:
cpu: 100m
memory: 256Mi
requests:
cpu: 0
memory: 128Mi
securityContext:
runAsGroup: 1000
runAsUser: 10000
volumeMounts:
- mountPath: /etc/tmc
name: tmc-root-ca
readOnly: true
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/master: ""
serviceAccountName: extension-updater-serviceaccount
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/notReady
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/unreachable
operator: Exists
- effect: NoSchedule
key: kubeadmNode
operator: Equal
value: master
volumes:
- configMap:
items:
- key: tls.crt
path: ca.crt
name: stack-config
name: tmc-root-ca
---
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: agent-updater
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
name: agent-updater
namespace: '{{.Namespace}}'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
app: agent-updater
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
name: agent-updater-role
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
creationTimestamp: null
labels:
app: agent-updater
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
name: agent-updater-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: agent-updater-role
subjects:
- kind: ServiceAccount
name: agent-updater
namespace: '{{.Namespace}}'
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: agent-updater
component: agent-updater
tmc-extension: "true"
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
name: agent-updater
namespace: '{{.Namespace}}'
spec:
minReadySeconds: 30
progressDeadlineSeconds: 600
replicas: 1
selector:
matchLabels:
component: agent-updater
tmc-extension: "true"
tmc-extension-name: agent-updater
strategy:
rollingUpdate:
maxSurge: 100%
template:
metadata:
labels:
app: agent-updater
component: agent-updater
tmc-extension: "true"
tmc-extension-name: agent-updater
spec:
containers:
- args:
- --metrics-bind-port=0
command:
- /usr/local/bin/manager
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: KUBERNETES_SERVICE_HOST
value: 127.0.0.1
- name: KUBERNETES_SERVICE_PORT
value: "6443"
image: vmware-docker-olympus-extensions.bintray.io/agent-updater/agent-updater@sha256:db6771f78ed45f1648295571a7c26d3bff1dc90f6ca681fcd1f54791b1a27b41
imagePullPolicy: Always
name: agent-updater
resources:
limits:
cpu: 100m
memory: 150Mi
requests:
cpu: 0
memory: 100Mi
securityContext:
runAsGroup: 1000
runAsUser: 10000
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/master: ""
serviceAccountName: agent-updater
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/notReady
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/unreachable
operator: Exists
- effect: NoSchedule
key: kubeadmNode
operator: Equal
value: master
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
annotations:
tmc.cloud.vmware.com/orphan-resource: "true"
labels:
app: agent-updater
component: agentupdater-workload
tmc-extension: "true"
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
name: agentupdater-workload
namespace: '{{.Namespace}}'
spec:
concurrencyPolicy: Forbid
jobTemplate:
metadata:
labels:
component: agentupdater-workload
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
spec:
template:
metadata:
labels:
tmc-extension-name: agent-updater
tmc.cloud.vmware.com/managed: "true"
spec:
containers:
- command:
- /usr/local/bin/agentupdaterworkload
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: KUBERNETES_SERVICE_HOST
value: 127.0.0.1
- name: KUBERNETES_SERVICE_PORT
value: "6443"
image: vmware-docker-olympus-extensions.bintray.io/agent-updater/agentupdater-workload@sha256:82e09f185abdf818a4198c0109d8de5c53ff2ab255b1e4e70fe3efb43901bc40
imagePullPolicy: IfNotPresent
name: agentupdater-workload
resources:
limits:
cpu: 100m
memory: 256Mi
requests:
cpu: 0
memory: 128Mi
securityContext:
runAsGroup: 1000
runAsUser: 10000
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/master: ""
restartPolicy: Never
serviceAccountName: agent-updater
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/notReady
operator: Exists
- effect: NoExecute
key: node.alpha.kubernetes.io/unreachable
operator: Exists
- effect: NoSchedule
key: kubeadmNode
operator: Equal
value: master
schedule: '*/1 * * * *'
startingDeadlineSeconds: 180
---
As for the AgentInstaller object, the following sample yaml file shows what I created to make use of the generated registration URL.
apiVersion: installers.tmc.cloud.vmware.com/v1alpha1
kind: AgentInstall
metadata:
name: tmc-agent-installer-config
namespace: svc-tmc-c8
spec:
operation: INSTALL
registrationLink: https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registrationNote that the namespace is svc-tmc-c8, the value noted earlier and the registrationLink is https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registration, which was taken from the TMC UI. Everything else in here would be the same for any supervisor cluster and TMC org.
This can now be installed by running kubectl create -f registration.yaml. You should see the following noting that the process has been started:
agentinstall.installers.tmc.cloud.vmware.com/tmc-agent-installer-config created
You'll be able to tell when the installation is finished by inspecting the status of the AgentInstall object that was created.
kubectl -n svc-tmc-c8 describe agentinstall tmc-agent-installer-configName: tmc-agent-installer-config
Namespace: svc-tmc-c8
Labels: <none>
Annotations: <none>
API Version: installers.tmc.cloud.vmware.com/v1alpha1
Kind: AgentInstall
Metadata:
Creation Timestamp: 2020-12-04T15:14:14Z
Generation: 1
Managed Fields:
API Version: installers.tmc.cloud.vmware.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:operation:
f:registrationLink:
Manager: kubectl-create
Operation: Update
Time: 2020-12-04T15:14:14Z
API Version: installers.tmc.cloud.vmware.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:message:
f:status:
Manager: tmc-agent-installer
Operation: Update
Time: 2020-12-04T15:15:26Z
Resource Version: 1135301
Self Link: /apis/installers.tmc.cloud.vmware.com/v1alpha1/namespaces/svc-tmc-c8/agentinstalls/tmc-agent-installer-config
UID: 467b0b70-2b2c-4561-b3b3-cd68749fb254
Spec:
Operation: INSTALL
Registration Link: https://epsg.tmc.cloud.vmware.com/installer?id=b1286efce66d7710c4e56ababed80cd99d18f650fc63a2f8a0ff8ec6daafd903&source=registration
Status:
Message: successfully applied the registration link
Status: INSTALLED
Events: <none>
Note that the Status is Installed, indicating a successful installation. If we take a look in the svc-tmc-c8 namespace again, we'll see that there are a few more pods present.
kubectl -n svc-tmc-c8 get po
NAME READY STATUS RESTARTS AGE
agent-updater-7fbb98485f-xsr6x 1/1 Running 0 8m46s
agentupdater-workload-1607095440-92g2l 0/1 Completed 0 10s
cluster-health-extension-59b9dd89f7-hr6x7 1/1 Running 0 5m22s
extension-manager-dd949bc96-4btzn 1/1 Running 0 8m48s
extension-updater-588fff5467-77bk2 1/1 Running 0 8m46s
intent-agent-5c8cd75b57-9c2nl 1/1 Running 0 5m22s
sync-agent-56559c797b-k8k2t 1/1 Running 0 5m22s
tmc-agent-installer-1607095380-zvbh8 0/1 Completed 0 68s
tmc-agent-installer-1607095440-z92q7 1/1 Running 0 2s
tmc-auto-attach-6f8df84468-qxkzd 1/1 Running 0 5m22s
vsphere-resource-retriever-7dc6b9546c-92jgg 1/1 Running 3 5m22s
Back in the TMC UI, you can click on the View Management Cluster button to be taken to the details for the now-attached supervisor cluster.
And if you click on the Workload clusters tab, you'll see that there are currently none present.
To finally get to my main goal, adding a Tanzu Kubernetes cluster to a vSphere 7 with Tanzu installation via TMC, we can navigate back to the Clusters page and then click the Create Cluster dropdown (note that this used to be a single button).
We've now got a new option for creating a cluster via the TKG Service included with vSphere 7 with Tanzu. Click on Tanzu Kubernetes Grid Service on vSphere 7.
Choose the recently added supervisor cluster (cjl-vsphere7) and the default provisioner (tkg, the namespace in my supervisor cluster). Click the Next button.
Enter a name for the Tanzu Kubernetes cluster and select the appropriate cluster group. You can enter any optional information that you would like also. Click the Next button.
Much of the information here is dependent on the configuration in the vSphere 7 with Tanzu installation. For example, you can see that the Kubernetes version list corresponds to the images present in the Content Library.
I've chosen the latest Kubernetes version, left the Pod and Service CIDRs at their default values and chosen k8s-policy (my only K8s-based storage policy) for the storage class. Click the Next button.
Choose whether you want a single node cluster or a highly available (multiple nodes) cluster and adjust the size of your nodes as appropriate. I'm going with a single node cluster with best-effort-xsmall size nodes since I'm incredibly starved for resources. Click the Next button.
On the node pool page, you'll need to provide information on how the node pool for this cluster should be configured. I've chosen to go with the same settings as were specified for the control plane nodes (single and best-effort-xsmall). Click the Create Cluster button.
Back in the vSphere Client, you'll see that there is now a cluster (cjl-test-tkc) present and a control plane node is being created.
You can also see from the command line that the Tanzu Kubernetes cluster is being created.
kubectl get tkc
NAME CONTROL PLANE WORKER DISTRIBUTION AGE PHASE
cjl-test-tkc 1 1 v1.18.5+vmware.1-tkg.1.c40d30d 3m5s creating
Once the control plane node is powered-on and configured, you'll see the worker node being deployed.
kubectl get virtualmachines
NAME AGE
cjl-test-tkc-control-plane-7g4jw 22m
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6 2m1s
kubectl get machines
NAME PROVIDERID PHASE
cjl-test-tkc-control-plane-7g4jw vsphere://422a7563-65ae-181e-0baa-3f79787010cd Running
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6 Provisioning
And back in the TMC UI, you'll see much more information about the cluster now that the control plane node is up and running.
When the worker node is done and the cluster is up and functional, there are a few places we can check to validate it.
kubectl get tkc
NAME CONTROL PLANE WORKER DISTRIBUTION AGE PHASE
cjl-test-tkc 1 1 v1.18.5+vmware.1-tkg.1.c40d30d 40m running
kubectl get machines
NAME PROVIDERID PHASE
cjl-test-tkc-control-plane-7g4jw vsphere://422a7563-65ae-181e-0baa-3f79787010cd Running
cjl-test-tkc-workers-bqzj9-c65589486-6nrv6 vsphere://422a545c-4b09-12bd-6633-dcc026fece3c Running
And if you navigate back to Administration, Management Clusters and drill down into the newly-registered cjl-vsphere7 cluster, you can see that there is a workload cluster present now.
In keeping with the standard means of accessing a Tanzu Kubernetes cluster running in vSphere 7 with Tanzu, you can use a modified kubectl vsphere login command (passing in the namespace and cluster names), similar to the following.
kubectl vsphere login --server=192.168.221.2 -u administrator@vsphere.local --tanzu-kubernetes-cluster-name cjl-test-tkc --tanzu-kubernetes-cluster-namespace tkg
Password:
Logged in successfully.
You have access to the following contexts:
192.168.221.2
cjl-test-tkc
tkg
If the context you wish to use is not in this list, you may need to try
logging in again later, or contact your cluster administrator.
To change context, use `kubectl config use-context <workload name>`
However, you can also make use of the familiar functionality in TMC for getting access to clusters provisioned on AWS. There is an Access this cluster button under the Actions dropdown that you can use to create a kubeconfig file.
After you click the Access this cluster button, you'll be presented with a page containing a link to the kubeconfig file and a sample command for using it.
You need to have the tmc executable installed (the latest version) where you will be using this kubeconfig file and have logged in with it. If you don't have an API token created already, you can do so at the link noted. You'll also need to provide the Management Cluster Name and Provisioner Name as they show up in the TMC UI.
tmc login
i If you don't have an API token, visit the VMware Cloud Services console, select your organization, and create an API token with the TMC service roles:
https://console.cloud.vmware.com/csp/gateway/portal/#/user/tokens
? API Token ****************************************************************
? Login context name cjl-test-tkc
? Select default log level info
? Management Cluster Name cjl-vsphere7
? Provisioner Name tkg
√ Successfully created context cjl-test-tkc, to manage your contexts run `tmc system context -h`
And finally, we can use the kubeconfig file generated by TMC to access the new cluster.
kubectl --kubeconfig=kubeconfig-cjl-test-tkc.yml get ns
NAME STATUS AGE
default Active 84m
kube-node-lease Active 84m
kube-public Active 84m
kube-system Active 84m
vmware-system-auth Active 83m
vmware-system-cloud-provider Active 82m
vmware-system-csi Active 82m
vmware-system-tmc Active 82m
It's worth noting that the first time I tried to use the kubeconfig file, I got the following error...
x could not login: Post https://192.168.221.3:6443/apis/login.concierge.pinniped.dev/v1alpha1/namespaces/vmware-system-tmc/tokencredentialrequests: context deadline exceeded
This only happened the one time and every subsequent kubectl command worked flawlessly....looks like it was just a simple timing issue on that first go at it.
Pingback: Provision and upgrade WCP clusters via TMC – Little Stuff
Pingback: Installing Tanzu Kubernetes Grid 1.3 on vSphere with NSX Advanced Load Balancer – Little Stuff