I’ve been through several TKGI installs over the last couple of years but not too many upgrades. The last time I went through one was going from 1.7 to 1.8, as documented in my previous post, A Walk-through of Upgrading Tanzu Kubernetes Grid Integrated Edition (Enterprise PKS) from 1.7 to 1.8. Much has changed since then and I’m happy to see that the process is much simpler with the introduction and maturation of the TKGI Management Console (TKGIMC). Since my last TKGI install, version 1.11, was done with the TKGIMC, I was able to take advantage of the much more streamlined process.
There is a brief overview of the upgrade process via the TKGIMC at Upgrade Tanzu Kubernetes Grid Integrated Edition Management Console but I wanted to go into it in much more detail, hence this post.
As with any upgrade, be sure to take a backup of all components and have a tested disaster recovery plan prior to starting.
Deploy the new TKGIMC OVA
After you’ve downloaded the new TKGIMC OVA from VMware, you can proceed with the upgrade process. In my environment, I have a static IP assigned to my TKGIMC appliance and want to keep using the same one. With that in mind, the old appliance will need to be updated with a new, temporary IP address.
Shutdown the old TKGIMC VM and then navigate to Configure, Settings, vApp Options. You should see the old IP address in the Properties section:
Click the Set Value link with this item selected and enter the new, temporary IP address:
Click the OK button and then you can power the old VM back on. The TKGIMC UI should soon be accessible at the new IP address.
Now you can deploy the new TKGIMC OVA while using your desired IP address.
You’ll want to make sure that you’ve got everything you need for the next few screens so that you can get it entered right the first time. You can change just about anything here after deployment but it’s always nice to get it right the first time:
It’s a pretty decent sized OVA (almost 14GB) so the deployment will likely take a little while to complete.
Once the VM is fully deployed, power it on and log in.
Click the Upgrade button.
Enter the temporary IP address for the old TKGIMC instance and the login credentials and then click the Connect button.
You can see the old and new versions in play on this screen. Note that you’ll have to manually update any Windows stemcells you have uploaded. If everything looks good, click Next to proceed.
If any items are red here you will need to drill down into them to correct any issues or enter appropriate information if a new, mandatory field is present. When you’re ready to proceed, click the Generate Upgrade Configuration button.
You can review the upgrade configuration in detail here if you need to before proceeding. The following is an example of what this upgrade configuration might look like:
auth:
auth_type: ldap
cluster_client_access_token_lifetime: 600
cluster_client_refresh_token_lifetime: 21600
ldap_cert: |-
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIQMfZy08muvIVKdZVDz7/rYzANBgkqhkiG9w0BAQsFADBI
MRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3JwMRkw
FwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIwMDgxOTE3MjA0NFoXDTMwMDgx
OTE3MzAzNVowSDEVMBMGCgmSJomT8ixkARkWBXRhbnp1MRQwEgYKCZImiZPyLGQB
GRYEY29ycDEZMBcGA1UEAxMQQ09OVFJPTENFTlRFUi1DQTCCAiIwDQYJKoZIhvcN
AQEBBQADggIPADCCAgoCggIBALKIdX7643PzvtVXlqNIwDuNq+rhcHF0fjR414j+
1IGQUuXrykjhSDthPP+8BGN7mBgHT8AjAS1b95xc8B0S2Fhln3AoRE9z03GtfsBu
FSBRUVwAifX6oXu97WzffhqPtxZfLJXbhOomjlkX6iffAs2TOLUx2Oj4w2vybhzj
lcA70ai+0Sl6axSo3lMZ4KkuZ2WgfEcaDjjj33/pV3/bnFK+7ydPttc2Tek5xsI8
XNMirIVxUiUT4YLy4WLiS200JUfbp1ZnMvnbQ8Jv1QnZl9W7WmBPcgxR4AAub0K4
vZLXu6MXiboTlzkMB/YthCkTNlJcKkhHf60YR/T6Sx1T2nupyBa4deo5UGPzhRiJ
pN37uqqAdK1qMDpCjARjS6U7Lf9JKjfiriLzLeyAjP8kaN4TdHSZd0pcQoZSxexQ
9n+4E4MQm4EJ4DrVZCilsyL2BdETcHXKPc7q+Db4XM7jPKNG5GP1EMV4Xohv58yZ
/rRfmK64gar8AMnOKT2AP681qdZs7lljONcXUALzlX5TqIchYT0DVQmFLYoMBeZz
0l21QjbK0YWnPza6Yi/N4m6rFbEB4WXiqhYSkxzrMvocVUgd4AAP1vfHNnFEsnUR
nSsiglFH/xlyO3cBFrmoZAxbA2091XHWhB4c0mQEI3hOqAB8UoFGBrQpmQ+LesoC
1LZ9AgMBAAGjUTBPMAsGA1UdDwQEAwIBhjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTFAxSvY64Q5adhm8IYecHBAUuobzAQBgkrBgEEAYI3FQEEAwIBADANBgkq
hkiG9w0BAQsFAAOCAgEAjg/v4mIP7gBVCw4pemtGn3PStDh/aB9vbWyjAyxSNaaH
H0nID5q5wow9ueBiDfjTPnhbf3P768HG8oL/+9C+Vm/0liFBd+0/DaayKpANFMLB
BV+s2adWRhQucLQfXPwum8RybWv82wkRkWCCdOBaAvAMuTgk08SwJIyQfVgpk3nY
0OwjFwSAadvevf+LoD/9L8R9NEt/n4WJe+LtEamo9EVb+l+cYqyxyubAVY0Y6BM2
GXqAh3FEW2aQMpwouh/5S7w5oSMYN6miY1ojki8gPm0+4+CILPWh/fr2q0O/bPtb
Tr++nPMmZ8ov9epNGIuqhtk5ja2/JuY+RW46IRc8QpF1EyUae02E6U2Vacs7Gge2
CeSINkoLFFmiKBfIn/HAchlme9aL6DlJ9wAreBDH3E8kH7gRDWbSK2/QD0Hqac+E
geGHwpg/8OtBOHUMnM7eLOXBJFcJosWf0XnEgS4ubgaHgqDEu8p8PE7rpCxtUNur
t+x2xONI/rBWgdbp51lPr7o819zPJCvYZq1Pp1st8fb3RlUSWvbQMPFtGAyaBy+G
0RgZ9WPtyEYgnHAb5/Dq46sne9/QnPwwGpjv1s1oE3ZFQjhvnGis8+dqRxk3YZAk
yiDghW7antzYL9S1CC8sVgVOwFJwfFXpdiir35mQlySG301V4FsRV+Z0cFp4Ni0=
-----END CERTIFICATE-----
ldap_cert_altname: ""
ldap_email_attribute: mail
ldap_email_domains: ""
ldap_external_group_whitelist: '*'
ldap_firstname_attribute: ""
ldap_group_search_base: cn=Users,dc=corp,dc=tanzu
ldap_group_search_filter: member={0}
ldap_lastname_attribute: ""
ldap_password: VMware1!
ldap_referrals: follow
ldap_url: ldaps://controlcenter.corp.tanzu:636
ldap_user_search_base: cn=Users,dc=corp,dc=tanzu
ldap_user_search_filter: userPrincipalName={0}
ldap_username: cn=Administrator,cn=Users,dc=corp,dc=tanzu
oidc_groups_claim: roles
oidc_groups_prefix: 'oidc:'
oidc_username_claim: user_name
oidc_username_prefix: 'oidc:'
pks_cli_access_token_lifetime: 600
pks_cli_refresh_token_lifetime: 21600
saml_default_identity_provider: false
saml_display_name: ""
saml_email_attribute: ""
saml_external_groups_attribute: ""
saml_first_name_attribute: ""
saml_last_name_attribute: ""
saml_name_id_format: ""
saml_pks_cli_autoapprove: false
saml_pks_cluster_client_autoapprove: false
saml_require_signed_assertions: false
saml_sign_auth_requests: false
saml_signature_algorithm: ""
saml_sso_name: ""
saml_sso_url: ""
saml_sso_xml: ""
uaa_as_cluster_oidc_provider: true
availability_zones:
- for_management_only: true
name: TKGI-MGMT-1
resources:
- resgroup-13006
- for_management_only: false
name: TKGI-COMP-1
resources:
- resgroup-13007
bosh_persistent_disk_type: "51200"
bosh_trust_opsman_ca: false
bosh_trusted_root_certs: |-
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIQMfZy08muvIVKdZVDz7/rYzANBgkqhkiG9w0BAQsFADBI
MRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3JwMRkw
FwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIwMDgxOTE3MjA0NFoXDTMwMDgx
OTE3MzAzNVowSDEVMBMGCgmSJomT8ixkARkWBXRhbnp1MRQwEgYKCZImiZPyLGQB
GRYEY29ycDEZMBcGA1UEAxMQQ09OVFJPTENFTlRFUi1DQTCCAiIwDQYJKoZIhvcN
AQEBBQADggIPADCCAgoCggIBALKIdX7643PzvtVXlqNIwDuNq+rhcHF0fjR414j+
1IGQUuXrykjhSDthPP+8BGN7mBgHT8AjAS1b95xc8B0S2Fhln3AoRE9z03GtfsBu
FSBRUVwAifX6oXu97WzffhqPtxZfLJXbhOomjlkX6iffAs2TOLUx2Oj4w2vybhzj
lcA70ai+0Sl6axSo3lMZ4KkuZ2WgfEcaDjjj33/pV3/bnFK+7ydPttc2Tek5xsI8
XNMirIVxUiUT4YLy4WLiS200JUfbp1ZnMvnbQ8Jv1QnZl9W7WmBPcgxR4AAub0K4
vZLXu6MXiboTlzkMB/YthCkTNlJcKkhHf60YR/T6Sx1T2nupyBa4deo5UGPzhRiJ
pN37uqqAdK1qMDpCjARjS6U7Lf9JKjfiriLzLeyAjP8kaN4TdHSZd0pcQoZSxexQ
9n+4E4MQm4EJ4DrVZCilsyL2BdETcHXKPc7q+Db4XM7jPKNG5GP1EMV4Xohv58yZ
/rRfmK64gar8AMnOKT2AP681qdZs7lljONcXUALzlX5TqIchYT0DVQmFLYoMBeZz
0l21QjbK0YWnPza6Yi/N4m6rFbEB4WXiqhYSkxzrMvocVUgd4AAP1vfHNnFEsnUR
nSsiglFH/xlyO3cBFrmoZAxbA2091XHWhB4c0mQEI3hOqAB8UoFGBrQpmQ+LesoC
1LZ9AgMBAAGjUTBPMAsGA1UdDwQEAwIBhjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTFAxSvY64Q5adhm8IYecHBAUuobzAQBgkrBgEEAYI3FQEEAwIBADANBgkq
hkiG9w0BAQsFAAOCAgEAjg/v4mIP7gBVCw4pemtGn3PStDh/aB9vbWyjAyxSNaaH
H0nID5q5wow9ueBiDfjTPnhbf3P768HG8oL/+9C+Vm/0liFBd+0/DaayKpANFMLB
BV+s2adWRhQucLQfXPwum8RybWv82wkRkWCCdOBaAvAMuTgk08SwJIyQfVgpk3nY
0OwjFwSAadvevf+LoD/9L8R9NEt/n4WJe+LtEamo9EVb+l+cYqyxyubAVY0Y6BM2
GXqAh3FEW2aQMpwouh/5S7w5oSMYN6miY1ojki8gPm0+4+CILPWh/fr2q0O/bPtb
Tr++nPMmZ8ov9epNGIuqhtk5ja2/JuY+RW46IRc8QpF1EyUae02E6U2Vacs7Gge2
CeSINkoLFFmiKBfIn/HAchlme9aL6DlJ9wAreBDH3E8kH7gRDWbSK2/QD0Hqac+E
geGHwpg/8OtBOHUMnM7eLOXBJFcJosWf0XnEgS4ubgaHgqDEu8p8PE7rpCxtUNur
t+x2xONI/rBWgdbp51lPr7o819zPJCvYZq1Pp1st8fb3RlUSWvbQMPFtGAyaBy+G
0RgZ9WPtyEYgnHAb5/Dq46sne9/QnPwwGpjv1s1oE3ZFQjhvnGis8+dqRxk3YZAk
yiDghW7antzYL9S1CC8sVgVOwFJwfFXpdiir35mQlySG301V4FsRV+Z0cFp4Ni0=
-----END CERTIFICATE-----
bosh_vm_type: large.disk
dns_servers: ""
enable_telemetry: false
ephemeral_storage:
- datastore-4001
harbor:
address_pool1_cidr: ""
address_pool1_size: ""
address_pool2_cidr: ""
address_pool2_size: ""
address_pool3_cidr: ""
address_pool3_size: ""
admin_password:
api_ca: |-
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIQMfZy08muvIVKdZVDz7/rYzANBgkqhkiG9w0BAQsFADBI
MRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3JwMRkw
FwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIwMDgxOTE3MjA0NFoXDTMwMDgx
OTE3MzAzNVowSDEVMBMGCgmSJomT8ixkARkWBXRhbnp1MRQwEgYKCZImiZPyLGQB
GRYEY29ycDEZMBcGA1UEAxMQQ09OVFJPTENFTlRFUi1DQTCCAiIwDQYJKoZIhvcN
AQEBBQADggIPADCCAgoCggIBALKIdX7643PzvtVXlqNIwDuNq+rhcHF0fjR414j+
1IGQUuXrykjhSDthPP+8BGN7mBgHT8AjAS1b95xc8B0S2Fhln3AoRE9z03GtfsBu
FSBRUVwAifX6oXu97WzffhqPtxZfLJXbhOomjlkX6iffAs2TOLUx2Oj4w2vybhzj
lcA70ai+0Sl6axSo3lMZ4KkuZ2WgfEcaDjjj33/pV3/bnFK+7ydPttc2Tek5xsI8
XNMirIVxUiUT4YLy4WLiS200JUfbp1ZnMvnbQ8Jv1QnZl9W7WmBPcgxR4AAub0K4
vZLXu6MXiboTlzkMB/YthCkTNlJcKkhHf60YR/T6Sx1T2nupyBa4deo5UGPzhRiJ
pN37uqqAdK1qMDpCjARjS6U7Lf9JKjfiriLzLeyAjP8kaN4TdHSZd0pcQoZSxexQ
9n+4E4MQm4EJ4DrVZCilsyL2BdETcHXKPc7q+Db4XM7jPKNG5GP1EMV4Xohv58yZ
/rRfmK64gar8AMnOKT2AP681qdZs7lljONcXUALzlX5TqIchYT0DVQmFLYoMBeZz
0l21QjbK0YWnPza6Yi/N4m6rFbEB4WXiqhYSkxzrMvocVUgd4AAP1vfHNnFEsnUR
nSsiglFH/xlyO3cBFrmoZAxbA2091XHWhB4c0mQEI3hOqAB8UoFGBrQpmQ+LesoC
1LZ9AgMBAAGjUTBPMAsGA1UdDwQEAwIBhjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTFAxSvY64Q5adhm8IYecHBAUuobzAQBgkrBgEEAYI3FQEEAwIBADANBgkq
hkiG9w0BAQsFAAOCAgEAjg/v4mIP7gBVCw4pemtGn3PStDh/aB9vbWyjAyxSNaaH
H0nID5q5wow9ueBiDfjTPnhbf3P768HG8oL/+9C+Vm/0liFBd+0/DaayKpANFMLB
BV+s2adWRhQucLQfXPwum8RybWv82wkRkWCCdOBaAvAMuTgk08SwJIyQfVgpk3nY
0OwjFwSAadvevf+LoD/9L8R9NEt/n4WJe+LtEamo9EVb+l+cYqyxyubAVY0Y6BM2
GXqAh3FEW2aQMpwouh/5S7w5oSMYN6miY1ojki8gPm0+4+CILPWh/fr2q0O/bPtb
Tr++nPMmZ8ov9epNGIuqhtk5ja2/JuY+RW46IRc8QpF1EyUae02E6U2Vacs7Gge2
CeSINkoLFFmiKBfIn/HAchlme9aL6DlJ9wAreBDH3E8kH7gRDWbSK2/QD0Hqac+E
geGHwpg/8OtBOHUMnM7eLOXBJFcJosWf0XnEgS4ubgaHgqDEu8p8PE7rpCxtUNur
t+x2xONI/rBWgdbp51lPr7o819zPJCvYZq1Pp1st8fb3RlUSWvbQMPFtGAyaBy+G
0RgZ9WPtyEYgnHAb5/Dq46sne9/QnPwwGpjv1s1oE3ZFQjhvnGis8+dqRxk3YZAk
yiDghW7antzYL9S1CC8sVgVOwFJwfFXpdiir35mQlySG301V4FsRV+Z0cFp4Ni0=
-----END CERTIFICATE-----
api_cert: |-
-----BEGIN CERTIFICATE-----
MIIHiDCCBXCgAwIBAgITHQAAAAkDm8eswM8dBgAAAAAACTANBgkqhkiG9w0BAQsF
ADBIMRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3Jw
MRkwFwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIxMDIwOTE2MjUyNFoXDTIz
MDIwOTE2MjUyNFowcjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWEx
EjAQBgNVBAcTCVBhbG8gQWx0bzEPMA0GA1UEChMGVk13YXJlMRIwEAYDVQQLEwlU
S0dJIDEuMTAxFTATBgNVBAMMDCouY29ycC50YW56dTCCAiIwDQYJKoZIhvcNAQEB
BQADggIPADCCAgoCggIBAL53OJVD9TUtRpLWHqLr9OpUqF8HZOMG/L8/t8QeDzAE
z2jIGC+ImCS2QUtf+t6NFU+1pDrcjZgfJE4KWoowQlCfwKPQr4YFSyoMN44RcmOD
lSTfYEcjrLWWn+XyU1AUDilhoceTfdZei/1Q3mXxUZLmkrqVOjucjhpOr2gmlD55
FEYeJBplBySsdcg9x0ey1+d/Ly7F2v4IWr91hDyNIJleUBbpF/atjhAazrRM9NLz
H9lp7FE/EEskN1ZzChpQGdcamUEcIlr4ROTw2Jsc9zL9AEw8JoxjYlH7oIEHPVN9
uwa7Ni3Yq9VWWFjZfhNXZQaz8aSQLpUHAgrTFPDkJNcebMFjnNR5exjTcffCV2I1
F0pgh4A5KvGHjn2j13mtxa8W7wGtBssNFmN/q1rKnGLjwwMRI1g78KWS1zuA3Hc8
H67YpoOV4LA31uYsdaTQvc16Qb81DKaiYAvuI4B/+f6OCEAvNIX3C/Ee3XXZB5j8
JAtpTtBasbxAFplntvljfjlcgbsdJ+lKMUInX4xfv0J0dFTOi+xZ2BJhNhXukONV
Po9jWNhPh1JyvhjvOWm8Mn24KcShpmiKdxKWzsEA9S5cN7RVtkMUOvcWrf8zQaKM
+LmZ6/EBId2eLB94OxLB0n1VeFXsxNe42xUEGxieY/4LCG7laTvRdRWi4aCwK/VJ
AgMBAAGjggI/MIICOzAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUH
AwEwFwYDVR0RBBAwDoIMKi5jb3JwLnRhbnp1MB0GA1UdDgQWBBSb5YTxUuhj65vF
ZwneQHFNhGJa+jAfBgNVHSMEGDAWgBTFAxSvY64Q5adhm8IYecHBAUuobzCB0wYD
VR0fBIHLMIHIMIHFoIHCoIG/hoG8bGRhcDovLy9DTj1DT05UUk9MQ0VOVEVSLUNB
LENOPWNvbnRyb2xjZW50ZXIsQ049Q0RQLENOPVB1YmxpYyUyMEtleSUyMFNlcnZp
Y2VzLENOPVNlcnZpY2VzLENOPUNvbmZpZ3VyYXRpb24sREM9Y29ycCxEQz10YW56
dT9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0P2Jhc2U/b2JqZWN0Q2xhc3M9Y1JM
RGlzdHJpYnV0aW9uUG9pbnQwgcEGCCsGAQUFBwEBBIG0MIGxMIGuBggrBgEFBQcw
AoaBoWxkYXA6Ly8vQ049Q09OVFJPTENFTlRFUi1DQSxDTj1BSUEsQ049UHVibGlj
JTIwS2V5JTIwU2VydmljZXMsQ049U2VydmljZXMsQ049Q29uZmlndXJhdGlvbixE
Qz1jb3JwLERDPXRhbnp1P2NBQ2VydGlmaWNhdGU/YmFzZT9vYmplY3RDbGFzcz1j
ZXJ0aWZpY2F0aW9uQXV0aG9yaXR5MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBl
AHIAdgBlAHIwDQYJKoZIhvcNAQELBQADggIBAAJq4Ix7Kd+Nz9ksBsdLbYOITux3
CznnBSALkUAu5aL5PfJM2ww0Z54aOO1PH74jxc/1GQ5MM+xdd12JRklwB76MLXST
8gWrWp229rCDA57qR5NgPY44rRM935WnnMoopQjdJTBveYvzFs8202E6yf4REdsx
RVr7T9fhPz/hkR3tblTdinKeMM1QLN4C2NUjeqXSciham6KpwfPvcB4Ifhfb0PP7
aQ6xbeEyGCc7y2Hj/DP52o64shGvEj4nM72xQhHT/1huXUuX3b1FH1+c8luZsker
s2hrbUwJiMaOP4dY1NhhLsJJMDHr9RZSEgNVl7XHtpMM0Qp4nYL4Xz6W85phqTgF
n8yt+NOeYEt7zuA9kK1/RSTTErXXpNfwTiJWQg3GqYlQ+mfwmjbAaCZ8r802ueNI
hXZjvRtg/uHyl/GYp/WVemygw1XUAUIosUOEY7v+rvvPurN9K0qgcD5zTl/bsV/y
5EFc+Q0KzSIV5CLfejwVJs40QdupWffXHOYqm49zT8ejffEExUBxXH/b4rooumkc
hpsrx5hbo/XJvS7ZbXCH/k8kDq8+9o4QEVjqYyVwA/F3+/Mv2ywGLwKY5B+WvJQt
LrxsDU58LYfVcwKSuryS5Rv9Kh0tZcFH2zpzQJDgMoZqPqZHFxhiV+w4KAD7WQxd
R22CcKK+kduUjv0X
-----END CERTIFICATE-----
api_private_key: |-
-----BEGIN PRIVATE KEY-----
MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQC+dziVQ/U1LUaS
1h6i6/TqVKhfB2TjBvy/P7fEHg8wBM9oyBgviJgktkFLX/rejRVPtaQ63I2YHyRO
ClqKMEJQn8Cj0K+GBUsqDDeOEXJjg5Uk32BHI6y1lp/l8lNQFA4pYaHHk33WXov9
UN5l8VGS5pK6lTo7nI4aTq9oJpQ+eRRGHiQaZQckrHXIPcdHstfnfy8uxdr+CFq/
dYQ8jSCZXlAW6Rf2rY4QGs60TPTS8x/ZaexRPxBLJDdWcwoaUBnXGplBHCJa+ETk
8NibHPcy/QBMPCaMY2JR+6CBBz1TfbsGuzYt2KvVVlhY2X4TV2UGs/GkkC6VBwIK
0xTw5CTXHmzBY5zUeXsY03H3wldiNRdKYIeAOSrxh459o9d5rcWvFu8BrQbLDRZj
f6taypxi48MDESNYO/Clktc7gNx3PB+u2KaDleCwN9bmLHWk0L3NekG/NQymomAL
7iOAf/n+jghALzSF9wvxHt112QeY/CQLaU7QWrG8QBaZZ7b5Y345XIG7HSfpSjFC
J1+MX79CdHRUzovsWdgSYTYV7pDjVT6PY1jYT4dScr4Y7zlpvDJ9uCnEoaZoincS
ls7BAPUuXDe0VbZDFDr3Fq3/M0GijPi5mevxASHdniwfeDsSwdJ9VXhV7MTXuNsV
BBsYnmP+Cwhu5Wk70XUVouGgsCv1SQIDAQABAoICAQCE843Ay943j3Iq/2IFUfX1
OMELDItE2lTFX0H0mRL67vCk8L/JNm0Ve09awRXKEetlZ6LLH7eLD3n1K88FlShF
RS5ga0SKpdlQ8ZQ6DD2v72LFiVOYdPOTEiBtj9jOFiHIiwk12ePGJttLKQ8FVA0g
IOkdaxtqDx82h+RzLDLg5P3c8B89eXYiCGxzKYSYrON/Cc2ytZPnLYfDC9IRvmWa
CTaYt37txzpaTYwqWWmwctuxlPnLwNyrxw0FwGm18mIHP97ojy4AGDtnICPjKrX3
lpmFnZs+9gTku2PPjXEmfaZ2zWnFWPChi5NB+hfCgofXxPYRbD/H8UtgqPV+LZL0
jP6SV1DTbSVl3AIrFtCwWGzpTYLN9kxqRNIlU3hHZorErXQo0GebOqwKEjP81TFQ
oBcBeRoWTkzZAl4oqS2FywXPEuUP21ChjK0jjW/VN267A23cS4IBhR7Cn825eg7/
ZRcW82/i9EfyxFXi1dTRBP98gEj2ls6coLmbPIZHNqG73DMxk9cI3LfjGGu3rbfR
TbqaNzsznYluiHlVWzjk3JKLvCBonGSD6ZhJ5OZkak8+Vl+rzI+9rWBJTBZihlwy
+QeHZGq9htTAQ5F7KO9Tn7D79e7wIwZf2DHMweRGX9dUfkOaHCwwcSjVH/yziXtR
aJ85akugMq/BciUBrffl/QKCAQEA8mmyuAbVecVRpcRVvH6auEAkL0yLp8rfAnm0
ToD/yQE2GOPSB4uucamPBd33wNQA5UDUc0GmqqOAN6ne2fT3ZtzebRobdpvuYUu1
XWlCvribR1zLgsglF2gTlOwOO1exZOlwax8a93xeBN3cQBXTisfXoMPbE80DTEqi
+CPNnMBXKP6JOabDaWD+XvfNEe7PlQcgL7BKiujj98Ui/J/ebnITgpksGDNyaUIG
62RwZeQOka6i6SMuCaD+Da3LjFfg9MAyOC1hsfst2puTkkJE+tJHRHOMSj6bZuW1
R6rwe6SqrnhaBdDdUzZmONOVaJBCrQ9YE3ZiECH3cS3OQvDDDwKCAQEAySQmb+R7
Ouk9IOh1HAZkLMWAPZ6FsZ916iMPnwcv7UADocRCzz3QtQOcI7lv242rrhLhVG/H
fZMZh9aAL0Wm8i1/mrpAIYyMiNQJvWs4aY2bb6rlIL9mR0iThbVZXfyeRLvfyMpy
O6PWYt8WF9dWLE7v3bW5Maqtqc+OND+1TGWTd0eZSSQgnR1VeFVNZSFG88vpmJDR
73POVbMEKpxYfe7heZ0dApcb/IA1a3Zqqz0cZ4uqu1dWehwtb40dYmaqswbY6ke8
3HKGQSBmlxWUF7Nn9Zg79u5YVW2jLOoMgUv3dDGAOIHlC97soA6NtoH/VhzY635+
8+sX3wktvgXiJwKCAQEAhDCzbrr7So4ZegXYoxN/F56SnOBm/7cXaWgotO6PjXMF
pwkFDWxUUlMeVRq38gUp/9ocgEV6t261iqUtizmUeBlVibVE6KcblR8N5cRyy0Is
Gvw1VjoCUANHOlyHXkDx0Y+i6CdsMy00r/60DpZYZ0OXCGoFW4TemYnR2PLdOu+A
GDDFcBTKVvq3e94xi+foduIN4TOHUryxI/nynEQprZyzmvIgI4pah5+j2lVJHacB
ctwCppOylTmfkKIHb560Y4MzX4MP1VidpqpUDNvqdcSZbHB+PjZp0/DLrCtBPIuN
L9sdbDJ7ntb5Y1+uB/kzAuBtLR/PVfDP2H4cDlDwbQKCAQBaRz51REDHLT6BkbRW
csvtiGvJvGfXVHIRN9FgGFK7ktrOdY9jAyS0yjz/j9CT459lzxWR12Xbh/WSkYUR
Mpr+4cr/QI9eP34oP7traD92qNdWJIcYzq9yWTHVdpL461SCFy0XKz5gZGXqFKUO
6FjGJFvm0BSiJTAzInR6IQoXkxPAGsPDH1MAEdV14BuPw4LcE+7xyjZf2kOHFYVO
NsRFKb3L3ufRbM9j4ouXgxvXZeNk2jw0P7wRrKn8AoNo0hnVpsIfTTmIXGLDwm4p
a8b/aEfF5KEtcMb2+PGfTCF2uwkC/uDE/BA45sKgCEg03V4kYWg/MpR6mE8rjSwZ
uPxLAoIBAQCPo0w3rHZMdBmaM9TCWfgCvMNuwhlWfPjQnYgpiPrVqnue1RdMLthY
eETTu+yxL+IiA5u46iJmM60p1nPw2IXWODJ634AbWZgNmZifqWr4Hm8DXCzrYBNp
cwB/NhcyJGX1eTZdooiwzdqukU5TCDRnPxNPv+TVUFcIsdmPHSJlZgkXiKh0JMbN
l8JjE2rjKUQc61kIoll+MNDoW5uCakeb5K0SRuxPHpGC1+6hzW3zQNa+kGd85b5y
zkrdsBEJ8/YXb9g+lId2Qpaj1MO6lgJTwLUkKTsDZ9hBindmBVAlAnVQzRxKeald
I2b/u5gfwdfn/3z+JNpdcdc1A4cX7Qdi
-----END PRIVATE KEY-----
auth_mode: ldap
enabled: true
harbor_http_proxy: ""
harbor_https_proxy: ""
harbor_no_proxy_addresses: ""
hostname: harbor.corp.tanzu
log_insight_enabled: false
log_insight_host: ""
log_insight_port: 514
log_insight_protocol: tcp
manual_ssl_certs: true
notary_enabled: true
post_deploy_enable_smoke_test: true
pre_delete_enable_deregister_uaa: true
storage_aws_access_key: ""
storage_aws_enable_v4auth: true
storage_aws_region: ""
storage_aws_s3_url: ""
storage_aws_secret: ""
storage_aws_secure_mode: true
storage_gcs_service_account_key: ""
storage_local_persistent_disk_type: "20480"
storage_nfs_share: ""
storage_s3_bucket_name: ""
storage_s3_bucket_root_dir: ""
storage_s3_chunksize: 5242880
storage_type: local
trivy_enabled: true
trivy_github_token: ""
trivy_skip_update: false
use_default_network_settings: true
vm_type: medium.disk
identity_manual_ssl_certs: true
identity_pks_api_host: tkgi.corp.tanzu
k8s_pv_storage:
- datastore-4001
log_insight:
ca_cert: ""
enable_ssl: true
enable_ssl_cert_validation: false
enabled: false
host: ""
rate_limiting: 0
logsink:
deploy_log_sink_resources: true
deploy_metric_sink_resources: true
network:
active_t0_edge_node: ""
active_t0_edge_node_ip: ""
additional_dep_reserved_ip_ranges: ""
additional_svc_reserved_ip_ranges: ""
autoprovision_nsx: false
dep_dns: 192.168.110.10
dep_network_cidr: ""
dep_network_gateway: ""
dep_network_moid: network-o17004
dep_network_name: ls-tkgi-mgmt
dep_network_type: OpaqueNetwork
dep_reserved_ip_range_from: 172.31.0.10
dep_reserved_ip_range_to: 172.31.0.10
enable_outbound_traffic: true
enabled_nsx_policy: false
external_portgroup_gateway: ""
external_portgroup_netmask: ""
external_portgroup_subnet: ""
external_vlan_id: 0
flannel_pod_network_cidr: 10.200.0.0/16
flannel_service_network_cidr: 10.100.200.0/24
floating_ips_range: ""
nsx_ca_crt: |-
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIQMfZy08muvIVKdZVDz7/rYzANBgkqhkiG9w0BAQsFADBI
MRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3JwMRkw
FwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIwMDgxOTE3MjA0NFoXDTMwMDgx
OTE3MzAzNVowSDEVMBMGCgmSJomT8ixkARkWBXRhbnp1MRQwEgYKCZImiZPyLGQB
GRYEY29ycDEZMBcGA1UEAxMQQ09OVFJPTENFTlRFUi1DQTCCAiIwDQYJKoZIhvcN
AQEBBQADggIPADCCAgoCggIBALKIdX7643PzvtVXlqNIwDuNq+rhcHF0fjR414j+
1IGQUuXrykjhSDthPP+8BGN7mBgHT8AjAS1b95xc8B0S2Fhln3AoRE9z03GtfsBu
FSBRUVwAifX6oXu97WzffhqPtxZfLJXbhOomjlkX6iffAs2TOLUx2Oj4w2vybhzj
lcA70ai+0Sl6axSo3lMZ4KkuZ2WgfEcaDjjj33/pV3/bnFK+7ydPttc2Tek5xsI8
XNMirIVxUiUT4YLy4WLiS200JUfbp1ZnMvnbQ8Jv1QnZl9W7WmBPcgxR4AAub0K4
vZLXu6MXiboTlzkMB/YthCkTNlJcKkhHf60YR/T6Sx1T2nupyBa4deo5UGPzhRiJ
pN37uqqAdK1qMDpCjARjS6U7Lf9JKjfiriLzLeyAjP8kaN4TdHSZd0pcQoZSxexQ
9n+4E4MQm4EJ4DrVZCilsyL2BdETcHXKPc7q+Db4XM7jPKNG5GP1EMV4Xohv58yZ
/rRfmK64gar8AMnOKT2AP681qdZs7lljONcXUALzlX5TqIchYT0DVQmFLYoMBeZz
0l21QjbK0YWnPza6Yi/N4m6rFbEB4WXiqhYSkxzrMvocVUgd4AAP1vfHNnFEsnUR
nSsiglFH/xlyO3cBFrmoZAxbA2091XHWhB4c0mQEI3hOqAB8UoFGBrQpmQ+LesoC
1LZ9AgMBAAGjUTBPMAsGA1UdDwQEAwIBhjAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBTFAxSvY64Q5adhm8IYecHBAUuobzAQBgkrBgEEAYI3FQEEAwIBADANBgkq
hkiG9w0BAQsFAAOCAgEAjg/v4mIP7gBVCw4pemtGn3PStDh/aB9vbWyjAyxSNaaH
H0nID5q5wow9ueBiDfjTPnhbf3P768HG8oL/+9C+Vm/0liFBd+0/DaayKpANFMLB
BV+s2adWRhQucLQfXPwum8RybWv82wkRkWCCdOBaAvAMuTgk08SwJIyQfVgpk3nY
0OwjFwSAadvevf+LoD/9L8R9NEt/n4WJe+LtEamo9EVb+l+cYqyxyubAVY0Y6BM2
GXqAh3FEW2aQMpwouh/5S7w5oSMYN6miY1ojki8gPm0+4+CILPWh/fr2q0O/bPtb
Tr++nPMmZ8ov9epNGIuqhtk5ja2/JuY+RW46IRc8QpF1EyUae02E6U2Vacs7Gge2
CeSINkoLFFmiKBfIn/HAchlme9aL6DlJ9wAreBDH3E8kH7gRDWbSK2/QD0Hqac+E
geGHwpg/8OtBOHUMnM7eLOXBJFcJosWf0XnEgS4ubgaHgqDEu8p8PE7rpCxtUNur
t+x2xONI/rBWgdbp51lPr7o819zPJCvYZq1Pp1st8fb3RlUSWvbQMPFtGAyaBy+G
0RgZ9WPtyEYgnHAb5/Dq46sne9/QnPwwGpjv1s1oE3ZFQjhvnGis8+dqRxk3YZAk
yiDghW7antzYL9S1CC8sVgVOwFJwfFXpdiir35mQlySG301V4FsRV+Z0cFp4Ni0=
-----END CERTIFICATE-----
nsx_dns: 192.168.110.10
nsx_enable_http_proxy: false
nsx_fip_id: ee88a04b-58a5-45e6-988e-11bd041fba1d
nsx_hybrid_nat: false
nsx_manual_ssl_certs: true
nsx_nat_mode: true
nsx_node_cidr: 10.20.0.0/16
nsx_node_ip_block_id: 1fe8c07b-4f3e-4f14-9c01-533c3bfc8946
nsx_pod_cidr: 10.10.0.0/16
nsx_pod_ip_block_id: ad4a8c09-fff2-4887-81da-9a75ff64f0a1
nsx_service_network_cidr: 10.100.200.0/24
nsx_t0_id: f32981b4-5858-4a0a-b7e1-c7aab619e426
nsx_verify_ssl_certs: true
proxy_http_password: ""
proxy_http_url: ""
proxy_http_user: ""
proxy_https_password: ""
proxy_https_url: ""
proxy_https_user: ""
proxy_no_proxy_addresses: ""
standby_t0_edge_node: ""
standby_t0_edge_node_ip: ""
svc_dns: ""
svc_network_cidr: 172.16.101.0/24
svc_network_gateway: ""
svc_network_name: ""
svc_reserved_ip_range_from: ""
svc_reserved_ip_range_to: ""
t0_edge_node_lb_ip: ""
t0_ha_mode_active_active: false
use_antrea: false
use_nsx: true
nsx_host: nsxmanager.corp.tanzu
nsx_password:
nsx_username: admin
ntp_servers: 192.168.100.1
opsman_fqdn: opsman.corp.tanzu
opsman_private_key: ""
opsman_root_cert: ""
permanent_storage:
- datastore-4001
pks_api_cert: |-
-----BEGIN CERTIFICATE-----
MIIHiDCCBXCgAwIBAgITHQAAAAkDm8eswM8dBgAAAAAACTANBgkqhkiG9w0BAQsF
ADBIMRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3Jw
MRkwFwYDVQQDExBDT05UUk9MQ0VOVEVSLUNBMB4XDTIxMDIwOTE2MjUyNFoXDTIz
MDIwOTE2MjUyNFowcjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWEx
EjAQBgNVBAcTCVBhbG8gQWx0bzEPMA0GA1UEChMGVk13YXJlMRIwEAYDVQQLEwlU
S0dJIDEuMTAxFTATBgNVBAMMDCouY29ycC50YW56dTCCAiIwDQYJKoZIhvcNAQEB
BQADggIPADCCAgoCggIBAL53OJVD9TUtRpLWHqLr9OpUqF8HZOMG/L8/t8QeDzAE
z2jIGC+ImCS2QUtf+t6NFU+1pDrcjZgfJE4KWoowQlCfwKPQr4YFSyoMN44RcmOD
lSTfYEcjrLWWn+XyU1AUDilhoceTfdZei/1Q3mXxUZLmkrqVOjucjhpOr2gmlD55
FEYeJBplBySsdcg9x0ey1+d/Ly7F2v4IWr91hDyNIJleUBbpF/atjhAazrRM9NLz
H9lp7FE/EEskN1ZzChpQGdcamUEcIlr4ROTw2Jsc9zL9AEw8JoxjYlH7oIEHPVN9
uwa7Ni3Yq9VWWFjZfhNXZQaz8aSQLpUHAgrTFPDkJNcebMFjnNR5exjTcffCV2I1
F0pgh4A5KvGHjn2j13mtxa8W7wGtBssNFmN/q1rKnGLjwwMRI1g78KWS1zuA3Hc8
H67YpoOV4LA31uYsdaTQvc16Qb81DKaiYAvuI4B/+f6OCEAvNIX3C/Ee3XXZB5j8
JAtpTtBasbxAFplntvljfjlcgbsdJ+lKMUInX4xfv0J0dFTOi+xZ2BJhNhXukONV
Po9jWNhPh1JyvhjvOWm8Mn24KcShpmiKdxKWzsEA9S5cN7RVtkMUOvcWrf8zQaKM
+LmZ6/EBId2eLB94OxLB0n1VeFXsxNe42xUEGxieY/4LCG7laTvRdRWi4aCwK/VJ
AgMBAAGjggI/MIICOzAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUH
AwEwFwYDVR0RBBAwDoIMKi5jb3JwLnRhbnp1MB0GA1UdDgQWBBSb5YTxUuhj65vF
ZwneQHFNhGJa+jAfBgNVHSMEGDAWgBTFAxSvY64Q5adhm8IYecHBAUuobzCB0wYD
VR0fBIHLMIHIMIHFoIHCoIG/hoG8bGRhcDovLy9DTj1DT05UUk9MQ0VOVEVSLUNB
LENOPWNvbnRyb2xjZW50ZXIsQ049Q0RQLENOPVB1YmxpYyUyMEtleSUyMFNlcnZp
Y2VzLENOPVNlcnZpY2VzLENOPUNvbmZpZ3VyYXRpb24sREM9Y29ycCxEQz10YW56
dT9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0P2Jhc2U/b2JqZWN0Q2xhc3M9Y1JM
RGlzdHJpYnV0aW9uUG9pbnQwgcEGCCsGAQUFBwEBBIG0MIGxMIGuBggrBgEFBQcw
AoaBoWxkYXA6Ly8vQ049Q09OVFJPTENFTlRFUi1DQSxDTj1BSUEsQ049UHVibGlj
JTIwS2V5JTIwU2VydmljZXMsQ049U2VydmljZXMsQ049Q29uZmlndXJhdGlvbixE
Qz1jb3JwLERDPXRhbnp1P2NBQ2VydGlmaWNhdGU/YmFzZT9vYmplY3RDbGFzcz1j
ZXJ0aWZpY2F0aW9uQXV0aG9yaXR5MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBl
AHIAdgBlAHIwDQYJKoZIhvcNAQELBQADggIBAAJq4Ix7Kd+Nz9ksBsdLbYOITux3
CznnBSALkUAu5aL5PfJM2ww0Z54aOO1PH74jxc/1GQ5MM+xdd12JRklwB76MLXST
8gWrWp229rCDA57qR5NgPY44rRM935WnnMoopQjdJTBveYvzFs8202E6yf4REdsx
RVr7T9fhPz/hkR3tblTdinKeMM1QLN4C2NUjeqXSciham6KpwfPvcB4Ifhfb0PP7
aQ6xbeEyGCc7y2Hj/DP52o64shGvEj4nM72xQhHT/1huXUuX3b1FH1+c8luZsker
s2hrbUwJiMaOP4dY1NhhLsJJMDHr9RZSEgNVl7XHtpMM0Qp4nYL4Xz6W85phqTgF
n8yt+NOeYEt7zuA9kK1/RSTTErXXpNfwTiJWQg3GqYlQ+mfwmjbAaCZ8r802ueNI
hXZjvRtg/uHyl/GYp/WVemygw1XUAUIosUOEY7v+rvvPurN9K0qgcD5zTl/bsV/y
5EFc+Q0KzSIV5CLfejwVJs40QdupWffXHOYqm49zT8ejffEExUBxXH/b4rooumkc
hpsrx5hbo/XJvS7ZbXCH/k8kDq8+9o4QEVjqYyVwA/F3+/Mv2ywGLwKY5B+WvJQt
LrxsDU58LYfVcwKSuryS5Rv9Kh0tZcFH2zpzQJDgMoZqPqZHFxhiV+w4KAD7WQxd
R22CcKK+kduUjv0X
-----END CERTIFICATE-----
pks_api_private_key: |-
-----BEGIN PRIVATE KEY-----
MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQC+dziVQ/U1LUaS
1h6i6/TqVKhfB2TjBvy/P7fEHg8wBM9oyBgviJgktkFLX/rejRVPtaQ63I2YHyRO
ClqKMEJQn8Cj0K+GBUsqDDeOEXJjg5Uk32BHI6y1lp/l8lNQFA4pYaHHk33WXov9
UN5l8VGS5pK6lTo7nI4aTq9oJpQ+eRRGHiQaZQckrHXIPcdHstfnfy8uxdr+CFq/
dYQ8jSCZXlAW6Rf2rY4QGs60TPTS8x/ZaexRPxBLJDdWcwoaUBnXGplBHCJa+ETk
8NibHPcy/QBMPCaMY2JR+6CBBz1TfbsGuzYt2KvVVlhY2X4TV2UGs/GkkC6VBwIK
0xTw5CTXHmzBY5zUeXsY03H3wldiNRdKYIeAOSrxh459o9d5rcWvFu8BrQbLDRZj
f6taypxi48MDESNYO/Clktc7gNx3PB+u2KaDleCwN9bmLHWk0L3NekG/NQymomAL
7iOAf/n+jghALzSF9wvxHt112QeY/CQLaU7QWrG8QBaZZ7b5Y345XIG7HSfpSjFC
J1+MX79CdHRUzovsWdgSYTYV7pDjVT6PY1jYT4dScr4Y7zlpvDJ9uCnEoaZoincS
ls7BAPUuXDe0VbZDFDr3Fq3/M0GijPi5mevxASHdniwfeDsSwdJ9VXhV7MTXuNsV
BBsYnmP+Cwhu5Wk70XUVouGgsCv1SQIDAQABAoICAQCE843Ay943j3Iq/2IFUfX1
OMELDItE2lTFX0H0mRL67vCk8L/JNm0Ve09awRXKEetlZ6LLH7eLD3n1K88FlShF
RS5ga0SKpdlQ8ZQ6DD2v72LFiVOYdPOTEiBtj9jOFiHIiwk12ePGJttLKQ8FVA0g
IOkdaxtqDx82h+RzLDLg5P3c8B89eXYiCGxzKYSYrON/Cc2ytZPnLYfDC9IRvmWa
CTaYt37txzpaTYwqWWmwctuxlPnLwNyrxw0FwGm18mIHP97ojy4AGDtnICPjKrX3
lpmFnZs+9gTku2PPjXEmfaZ2zWnFWPChi5NB+hfCgofXxPYRbD/H8UtgqPV+LZL0
jP6SV1DTbSVl3AIrFtCwWGzpTYLN9kxqRNIlU3hHZorErXQo0GebOqwKEjP81TFQ
oBcBeRoWTkzZAl4oqS2FywXPEuUP21ChjK0jjW/VN267A23cS4IBhR7Cn825eg7/
ZRcW82/i9EfyxFXi1dTRBP98gEj2ls6coLmbPIZHNqG73DMxk9cI3LfjGGu3rbfR
TbqaNzsznYluiHlVWzjk3JKLvCBonGSD6ZhJ5OZkak8+Vl+rzI+9rWBJTBZihlwy
+QeHZGq9htTAQ5F7KO9Tn7D79e7wIwZf2DHMweRGX9dUfkOaHCwwcSjVH/yziXtR
aJ85akugMq/BciUBrffl/QKCAQEA8mmyuAbVecVRpcRVvH6auEAkL0yLp8rfAnm0
ToD/yQE2GOPSB4uucamPBd33wNQA5UDUc0GmqqOAN6ne2fT3ZtzebRobdpvuYUu1
XWlCvribR1zLgsglF2gTlOwOO1exZOlwax8a93xeBN3cQBXTisfXoMPbE80DTEqi
+CPNnMBXKP6JOabDaWD+XvfNEe7PlQcgL7BKiujj98Ui/J/ebnITgpksGDNyaUIG
62RwZeQOka6i6SMuCaD+Da3LjFfg9MAyOC1hsfst2puTkkJE+tJHRHOMSj6bZuW1
R6rwe6SqrnhaBdDdUzZmONOVaJBCrQ9YE3ZiECH3cS3OQvDDDwKCAQEAySQmb+R7
Ouk9IOh1HAZkLMWAPZ6FsZ916iMPnwcv7UADocRCzz3QtQOcI7lv242rrhLhVG/H
fZMZh9aAL0Wm8i1/mrpAIYyMiNQJvWs4aY2bb6rlIL9mR0iThbVZXfyeRLvfyMpy
O6PWYt8WF9dWLE7v3bW5Maqtqc+OND+1TGWTd0eZSSQgnR1VeFVNZSFG88vpmJDR
73POVbMEKpxYfe7heZ0dApcb/IA1a3Zqqz0cZ4uqu1dWehwtb40dYmaqswbY6ke8
3HKGQSBmlxWUF7Nn9Zg79u5YVW2jLOoMgUv3dDGAOIHlC97soA6NtoH/VhzY635+
8+sX3wktvgXiJwKCAQEAhDCzbrr7So4ZegXYoxN/F56SnOBm/7cXaWgotO6PjXMF
pwkFDWxUUlMeVRq38gUp/9ocgEV6t261iqUtizmUeBlVibVE6KcblR8N5cRyy0Is
Gvw1VjoCUANHOlyHXkDx0Y+i6CdsMy00r/60DpZYZ0OXCGoFW4TemYnR2PLdOu+A
GDDFcBTKVvq3e94xi+foduIN4TOHUryxI/nynEQprZyzmvIgI4pah5+j2lVJHacB
ctwCppOylTmfkKIHb560Y4MzX4MP1VidpqpUDNvqdcSZbHB+PjZp0/DLrCtBPIuN
L9sdbDJ7ntb5Y1+uB/kzAuBtLR/PVfDP2H4cDlDwbQKCAQBaRz51REDHLT6BkbRW
csvtiGvJvGfXVHIRN9FgGFK7ktrOdY9jAyS0yjz/j9CT459lzxWR12Xbh/WSkYUR
Mpr+4cr/QI9eP34oP7traD92qNdWJIcYzq9yWTHVdpL461SCFy0XKz5gZGXqFKUO
6FjGJFvm0BSiJTAzInR6IQoXkxPAGsPDH1MAEdV14BuPw4LcE+7xyjZf2kOHFYVO
NsRFKb3L3ufRbM9j4ouXgxvXZeNk2jw0P7wRrKn8AoNo0hnVpsIfTTmIXGLDwm4p
a8b/aEfF5KEtcMb2+PGfTCF2uwkC/uDE/BA45sKgCEg03V4kYWg/MpR6mE8rjSwZ
uPxLAoIBAQCPo0w3rHZMdBmaM9TCWfgCvMNuwhlWfPjQnYgpiPrVqnue1RdMLthY
eETTu+yxL+IiA5u46iJmM60p1nPw2IXWODJ634AbWZgNmZifqWr4Hm8DXCzrYBNp
cwB/NhcyJGX1eTZdooiwzdqukU5TCDRnPxNPv+TVUFcIsdmPHSJlZgkXiKh0JMbN
l8JjE2rjKUQc61kIoll+MNDoW5uCakeb5K0SRuxPHpGC1+6hzW3zQNa+kGd85b5y
zkrdsBEJ8/YXb9g+lId2Qpaj1MO6lgJTwLUkKTsDZ9hBindmBVAlAnVQzRxKeald
I2b/u5gfwdfn/3z+JNpdcdc1A4cX7Qdi
-----END PRIVATE KEY-----
pks_db_persistent_disk_type: "10240"
pks_db_vm_type: medium.disk
pks_enable_db_ha: false
pks_enable_quotas: false
pks_enable_vsphere_csi: false
pks_persistent_disk_type: "10240"
pks_run_upgrade_clusters_errand: false
pks_telemetry:
customerNumber: ""
installation_purpose: not_provided
selector: disabled
pks_vm_instance_number: 1
pks_vm_type: medium.disk
plans:
- addons-spec: ""
admission_pod_security_policy: false
admission_security_context_deny: false
allow_privileged_containers: true
apply_addons_azs: []
apply_addons_instances: 1
apply_addons_lifecycle: errand
apply_addons_networks: []
apply_addons_vm_type: medium.disk
cluster_services_linux_ha_enabled: false
description: This plan will configure a lightweight Kubernetes cluster. Not recommended
for production workloads.
kubelet_custimzation_hard_eviction_threshold: ""
kubelet_custimzation_system_reserved: ""
kubelet_drain_delete_local_data: true
kubelet_drain_force: true
kubelet_drain_force_node: false
kubelet_drain_grace_period: 10
kubelet_drain_ignore_daemonsets: true
kubelet_drain_timeout: 0
master_azs:
- enabled: true
name: TKGI-COMP-1
master_instances: 1
master_networks: []
master_persistent_disk_type: "10240"
master_vm_extensions: []
master_vm_type: medium.disk
name: linux-small
plan_id: ""
post_deploy_errands: []
pre_delete_errands: []
windows_worker_os_enabled: false
worker_azs:
- enabled: true
name: TKGI-COMP-1
worker_instances: 1
worker_max_instances: 50
worker_networks: []
worker_persistent_disk_type: "20480"
worker_vm_extensions: []
worker_vm_type: medium.disk
- addons-spec: ""
admission_pod_security_policy: false
admission_security_context_deny: false
allow_privileged_containers: true
apply_addons_azs: []
apply_addons_instances: 1
apply_addons_lifecycle: errand
apply_addons_networks: []
apply_addons_vm_type: medium.disk
cluster_services_linux_ha_enabled: false
description: 'Example: This plan will configure a medium sized Kubernetes cluster,
suitable for more pods.'
kubelet_custimzation_hard_eviction_threshold: ""
kubelet_custimzation_system_reserved: ""
kubelet_drain_delete_local_data: true
kubelet_drain_force: true
kubelet_drain_force_node: false
kubelet_drain_grace_period: 10
kubelet_drain_ignore_daemonsets: true
kubelet_drain_timeout: 0
master_azs:
- enabled: true
name: TKGI-COMP-1
master_instances: 3
master_networks: []
master_persistent_disk_type: "10240"
master_vm_extensions: []
master_vm_type: medium.disk
name: linux-medium
plan_id: ""
post_deploy_errands: []
pre_delete_errands: []
windows_worker_os_enabled: false
worker_azs:
- enabled: true
name: TKGI-COMP-1
worker_instances: 5
worker_max_instances: 50
worker_networks: []
worker_persistent_disk_type: "51200"
worker_vm_extensions: []
worker_vm_type: medium.disk
- addons-spec: ""
admission_pod_security_policy: false
admission_security_context_deny: false
allow_privileged_containers: false
apply_addons_azs: []
apply_addons_instances: 1
apply_addons_lifecycle: errand
apply_addons_networks: []
apply_addons_vm_type: medium.disk
cluster_services_linux_ha_enabled: false
description: small windows plan
kubelet_custimzation_hard_eviction_threshold: ""
kubelet_custimzation_system_reserved: ""
kubelet_drain_delete_local_data: true
kubelet_drain_force: true
kubelet_drain_force_node: false
kubelet_drain_grace_period: 10
kubelet_drain_ignore_daemonsets: true
kubelet_drain_timeout: 0
master_azs:
- enabled: true
name: TKGI-COMP-1
master_instances: 1
master_networks: []
master_persistent_disk_type: "10240"
master_vm_extensions: []
master_vm_type: large.disk
name: windows-small
plan_id: ""
post_deploy_errands: []
pre_delete_errands: []
windows_worker_os_enabled: true
worker_azs:
- enabled: true
name: TKGI-COMP-1
worker_instances: 1
worker_max_instances: 50
worker_networks: []
worker_persistent_disk_type: "20480"
worker_vm_extensions: []
worker_vm_type: xlarge
- addons-spec: ""
admission_pod_security_policy: false
admission_security_context_deny: false
allow_privileged_containers: false
apply_addons_azs: []
apply_addons_instances: 1
apply_addons_lifecycle: errand
apply_addons_networks: []
apply_addons_vm_type: medium.disk
cluster_services_linux_ha_enabled: false
description: medium Windows plan
kubelet_custimzation_hard_eviction_threshold: ""
kubelet_custimzation_system_reserved: ""
kubelet_drain_delete_local_data: true
kubelet_drain_force: true
kubelet_drain_force_node: false
kubelet_drain_grace_period: 10
kubelet_drain_ignore_daemonsets: true
kubelet_drain_timeout: 0
master_azs:
- enabled: true
name: TKGI-COMP-1
master_instances: 3
master_networks: []
master_persistent_disk_type: "10240"
master_vm_extensions: []
master_vm_type: large.disk
name: windows-medium
plan_id: ""
post_deploy_errands: []
pre_delete_errands: []
windows_worker_os_enabled: true
worker_azs:
- enabled: true
name: TKGI-COMP-1
worker_instances: 5
worker_max_instances: 50
worker_networks: []
worker_persistent_disk_type: "51200"
worker_vm_extensions: []
worker_vm_type: xlarge
syslog:
address: ""
enable_tls: false
enabled: false
permitted_peer: ""
port: 0
protocol: tcp
tls_cert: ""
tmc_integration:
enabled: false
tmc_api_token: ""
tmc_api_url: ""
tmc_cluster_name_prefix: tkgi-
tmc_group_name: ""
use_thin_provisioned_disks: true
vcenter_datacenter: datacenter-1001
vcenter_host: vcsa-01a.corp.tanzu
vcenter_password:
vcenter_username: administrator@vsphere.local
vrops:
enabled: false
wavefront:
enabled: false
pks_wavefront_http_proxy: ""
token: ""
url: ""
worker_max_in_flight: 1 When you’re ready to proceed, click the Apply Upgrade Configuration button.
Per the warning, be sure to get any changes that might have been made manually in Opsman synchronized with TKGIMC. You can click the Continue button when you’re ready to proceed.
Upgrade Opsman
And the upgrade is underway. The old opsman VM will be suspended in preparation for the new one to be deployed:
And once that task is finished, a new opsman VM will get deployed:
And you can see the same task in the TKGIMC UI:
The new opsman VM will power on and have the same IP address as the original opsman VM (172.31.0.2 in this case):
The existing NAT rule in NSX-T will allow for the same access to the opsman UI as before (via the 10.40.14.34 address):
The original opsman VM should be deleted automatically.
Upgrade bosh
With the new opsman deployed, you should now be able to login and follow the progress there as well.
You can click on the Show Progress button to see the details on what is happening:
After a few minutes, you should see a new stemcell getting deployed:
This will be used for the new bosh VM. You should see the old bosh VM get deleted and a new one created:
Back in the Opsman UI you can see more details about the current operation:
There will be a lot more activity like this along with numerous tasks in vCenter while the new bosh VM is being configured.
I noticed that by the time the installation of the new bosh VM got to this point, I was able to access it again and run bosh commands:
bosh vms
Using environment '172.31.0.3' as client 'ops_manager'
Task 294
Task 295
Task 293
Task 294 done
Task 295 done
Task 293 done
Deployment 'harbor-container-registry-4cc226eed5c63d07ac2d'
Instance Process State AZ IPs VM CID VM Type Active Stemcell
harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 running TKGI-MGMT-1 172.31.0.6 vm-dcf29dfb-51b7-4e58-8c00-ac50de618222 medium.disk true bosh-vsphere-esxi-ubuntu-xenial-go_agent/621.125
1 vms
Deployment 'pivotal-container-service-5d7c44d55652fb4f3c22'
Instance Process State AZ IPs VM CID VM Type Active Stemcell
pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a running TKGI-MGMT-1 172.31.0.5 vm-b8514659-825b-4a09-bbf9-0f7d45e4c011 medium.disk true bosh-vsphere-esxi-ubuntu-xenial-go_agent/621.125
pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 running TKGI-MGMT-1 172.31.0.4 vm-cd957e99-d010-41bc-861f-23a6a90f4c35 medium.disk true bosh-vsphere-esxi-ubuntu-xenial-go_agent/621.125
2 vms
Deployment 'service-instance_2ed3bec8-1710-48f4-9917-19599e6cdcd0'
Instance Process State AZ IPs VM CID VM Type Active Stemcell
master/c899fd74-5060-461f-bec1-05de2b29566e running TKGI-COMP-1 172.15.0.2 vm-19648717-f394-4d51-afe0-4c9ffd31ce65 medium.disk true bosh-vsphere-esxi-ubuntu-xenial-go_agent/621.125
worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 running TKGI-COMP-1 172.15.0.3 vm-cd711202-0ea9-4813-82e5-107974738b9b medium.disk true bosh-vsphere-esxi-ubuntu-xenial-go_agent/621.125
2 vms
Succeeded
And very shortly after this point, the bosh portion of the upgrade was finished.
The very last thing to happen in this phase is the deletion of the old bosh VM:
You should see that the upgrade process has moved on to the TKGI phase in the TKGIMC UI:
Upgrade TKGI
There’s nothing to really see in vSphere or Opsman while the TKGI tile is being uploaded. Once this step is finished you’ll see activity very similar to what was observed with the bosh VM.
When the process moves on to the deploying phase, you can follow the progress in more detail in the Opsman UI.
You monitor the progress from the command line as well via the bosh task command:
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 327
Task 327 | 13:57:17 | Preparing deployment: Preparing deployment (00:00:11)
Task 327 | 13:57:28 | Preparing deployment: Rendering templates (00:00:08)
Task 327 | 13:57:37 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 327 | 13:57:37 | Compiling packages: golang-1-linux/f064db540b2fa2fe3e640f78d84dd00b8ee4052aec703d50763e9739f134c40e
Task 327 | 13:57:37 | Compiling packages: golang-1-linux/92acc26a97deb3429611aeef37c7f2d2697fd45dc0e8c14d62231d569112dd7b
Task 327 | 13:59:42 | Compiling packages: golang-1-linux/f064db540b2fa2fe3e640f78d84dd00b8ee4052aec703d50763e9739f134c40e (00:02:05)
Task 327 | 13:59:42 | Compiling packages: system-metrics-agent/40d78c7533904cebed45b39b3de555a9af9c71d087e1d2325d66514090806a92
Task 327 | 13:59:47 | Compiling packages: golang-1-linux/92acc26a97deb3429611aeef37c7f2d2697fd45dc0e8c14d62231d569112dd7b (00:02:10)
Task 327 | 13:59:47 | Compiling packages: bosh-dns/5d531fe59a5fa8e74d55db78ddbf0c30418e627046d3cf56549bdfbfaba176e3The process will proceed with deploying a new stemcell for the TKGI VMs (API and DB VMs), cloning the stemcell (multiple times for the canary builds), configuring TKGI and then deleting the original VMs. When the process is complete you’ll be informed in the Opsman UI if you’re following along there.
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 327
Task 327 | 13:57:17 | Preparing deployment: Preparing deployment (00:00:11)
Task 327 | 13:57:28 | Preparing deployment: Rendering templates (00:00:08)
Task 327 | 13:57:37 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 327 | 13:57:37 | Compiling packages: golang-1-linux/f064db540b2fa2fe3e640f78d84dd00b8ee4052aec703d50763e9739f134c40e
Task 327 | 13:57:37 | Compiling packages: golang-1-linux/92acc26a97deb3429611aeef37c7f2d2697fd45dc0e8c14d62231d569112dd7b
Task 327 | 13:59:42 | Compiling packages: golang-1-linux/f064db540b2fa2fe3e640f78d84dd00b8ee4052aec703d50763e9739f134c40e (00:02:05)
Task 327 | 13:59:42 | Compiling packages: system-metrics-agent/40d78c7533904cebed45b39b3de555a9af9c71d087e1d2325d66514090806a92
Task 327 | 13:59:47 | Compiling packages: golang-1-linux/92acc26a97deb3429611aeef37c7f2d2697fd45dc0e8c14d62231d569112dd7b (00:02:10)
Task 327 | 13:59:47 | Compiling packages: bosh-dns/5d531fe59a5fa8e74d55db78ddbf0c30418e627046d3cf56549bdfbfaba176e3
Task 327 | 14:00:15 | Compiling packages: system-metrics-agent/40d78c7533904cebed45b39b3de555a9af9c71d087e1d2325d66514090806a92 (00:00:33)
Task 327 | 14:00:37 | Compiling packages: bosh-dns/5d531fe59a5fa8e74d55db78ddbf0c30418e627046d3cf56549bdfbfaba176e3 (00:00:50)
Task 327 | 14:00:59 | Updating instance pks-db: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:00:59 | L executing pre-stop: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:01:00 | L executing drain: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:01:18 | L stopping jobs: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:01:26 | L executing post-stop: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:04:10 | L installing packages: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:04:36 | L configuring jobs: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:04:36 | L executing pre-start: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:05:13 | L starting jobs: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary)
Task 327 | 14:05:45 | L executing post-start: pks-db/c4761a65-e2c3-4244-bc8d-cf4b59451a64 (0) (canary) (00:06:16)
Task 327 | 14:07:15 | Updating instance pivotal-container-service: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:07:16 | L executing pre-stop: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:07:17 | L executing drain: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:07:18 | L stopping jobs: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:07:28 | L executing post-stop: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:10:04 | L installing packages: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:11:11 | L configuring jobs: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:11:11 | L executing pre-start: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:13:37 | L starting jobs: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary)
Task 327 | 14:14:08 | L executing post-start: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a (0) (canary) (00:07:31)
Task 327 Started Thu Sep 23 13:57:17 UTC 2021
Task 327 Finished Thu Sep 23 14:14:46 UTC 2021
Task 327 Duration 00:17:29
Task 327 done
Succeeded
Upgrade Harbor
As with TKGI, while the Harbor tile is uploading you’re not going to see much activity. This is a fairly small tile so it should move on fairly quickly.
And just like the previous components, you can follow along in the Opsman UI or at the command line to get more details:
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 334
Task 334 | 14:20:20 | Preparing deployment: Preparing deployment (00:00:04)
Task 334 | 14:20:24 | Preparing deployment: Rendering templates (00:00:02)
Task 334 | 14:20:26 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 334 | 14:20:26 | Compiling packages: busybox/8a4c96b84928f6e9540e11eadaaa2d5a0afd5b90166c4e43ec0997a380eb51f9
Task 334 | 14:20:26 | Compiling packages: nfs-common/28863548db503f8be6a588950676eb13a536dd077567edb8f38f7d9de6403e8e
Task 334 | 14:20:26 | Compiling packages: smoke-test/249e495341e27147c0305e562d8ccce1b2a3b9fb818211cf9cbcfe3769e3e043
Task 334 | 14:20:26 | Compiling packages: python/2634ba233f5c4a1602f15712d146dd88451bd0661d757e8f1950045ee7022052
Once the Harbor upgrade is finished you should see notification of the completion wherever you are watching:
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 334
Task 334 | 14:20:20 | Preparing deployment: Preparing deployment (00:00:04)
Task 334 | 14:20:24 | Preparing deployment: Rendering templates (00:00:02)
Task 334 | 14:20:26 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 334 | 14:20:26 | Compiling packages: busybox/8a4c96b84928f6e9540e11eadaaa2d5a0afd5b90166c4e43ec0997a380eb51f9
Task 334 | 14:20:26 | Compiling packages: nfs-common/28863548db503f8be6a588950676eb13a536dd077567edb8f38f7d9de6403e8e
Task 334 | 14:20:26 | Compiling packages: smoke-test/249e495341e27147c0305e562d8ccce1b2a3b9fb818211cf9cbcfe3769e3e043
Task 334 | 14:20:26 | Compiling packages: python/2634ba233f5c4a1602f15712d146dd88451bd0661d757e8f1950045ee7022052
Task 334 | 14:22:02 | Compiling packages: smoke-test/249e495341e27147c0305e562d8ccce1b2a3b9fb818211cf9cbcfe3769e3e043 (00:01:36)
Task 334 | 14:22:02 | Compiling packages: uaa/d443ffe1547c43bd37d9f2973f12f877c93a0083d5b0da5c1617a623d09ed289 (00:00:18)
Task 334 | 14:22:20 | Compiling packages: wavefront/0d38ef4dfec0f35aeaac6f292e154b0bd9948eb3aa7682116820f5814a161efc
Task 334 | 14:22:32 | Compiling packages: nfs-common/28863548db503f8be6a588950676eb13a536dd077567edb8f38f7d9de6403e8e (00:02:06)
Task 334 | 14:22:32 | Compiling packages: docker-compose/a316e53e5ac4f708a82e4317c4e674c7fac92621061cac7bc253e41157462d3f
Task 334 | 14:22:34 | Compiling packages: busybox/8a4c96b84928f6e9540e11eadaaa2d5a0afd5b90166c4e43ec0997a380eb51f9 (00:02:08)
Task 334 | 14:22:34 | Compiling packages: common/1ee9dd03e6d818e2158fd430c06e81ca519f9fde706dcb8359fc22db1d2f0f50
Task 334 | 14:22:50 | Compiling packages: docker-compose/a316e53e5ac4f708a82e4317c4e674c7fac92621061cac7bc253e41157462d3f (00:00:18)
Task 334 | 14:22:52 | Compiling packages: common/1ee9dd03e6d818e2158fd430c06e81ca519f9fde706dcb8359fc22db1d2f0f50 (00:00:18)
Task 334 | 14:22:52 | Compiling packages: docker/3b37ad894d154483514c95232240e0a2529384cd22975f3738eaaea307a1690f
Task 334 | 14:22:52 | Compiling packages: harbor-common/c7fdde453d1de2011ac8de8bc593cb998fdbab565dab2ac130ce35fa9ee4bd77 (00:00:15)
Task 334 | 14:23:07 | Compiling packages: harbor-app/df55cfe8c9763b068684a7ed6e58434e16739a7a9646b4d95e0e7058348f9c18
Task 334 | 14:23:38 | Compiling packages: docker/3b37ad894d154483514c95232240e0a2529384cd22975f3738eaaea307a1690f (00:00:46)
Task 334 | 14:24:09 | Compiling packages: wavefront/0d38ef4dfec0f35aeaac6f292e154b0bd9948eb3aa7682116820f5814a161efc (00:01:49)
Task 334 | 14:25:09 | Compiling packages: python/2634ba233f5c4a1602f15712d146dd88451bd0661d757e8f1950045ee7022052 (00:04:43)
Task 334 | 14:26:11 | Compiling packages: harbor-app/df55cfe8c9763b068684a7ed6e58434e16739a7a9646b4d95e0e7058348f9c18 (00:03:04)
Task 334 | 14:26:43 | Updating instance harbor-app: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:26:43 | L executing pre-stop: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:26:44 | L executing drain: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:26:45 | L stopping jobs: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:27:36 | L executing post-stop: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:30:07 | L installing packages: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:31:05 | L configuring jobs: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:31:05 | L executing pre-start: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:35:45 | L starting jobs: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary)
Task 334 | 14:36:16 | L executing post-start: harbor-app/cc2959d0-b4d5-4f83-b592-80630b856c61 (0) (canary) (00:09:34)
Task 334 Started Thu Sep 23 14:20:20 UTC 2021
Task 334 Finished Thu Sep 23 14:36:17 UTC 2021
Task 334 Duration 00:15:57
Task 334 done
SucceededI didn’t actually get to see the final stage show “Completed” in the TKGIMC UI since my login timed out but this was the last page I saw:
But after logging back in…
The upgrade was complete with all components showing the new versions.
Upgrade clusters
The upgrade process does not automatically upgrade any deployed clusters so this still remains to be done. I had a single linux cluster deployed and it still showed up as being a TKGI 1.11 cluster:
tkgi clusters
Upgrade is available to PKS Version: 1.12.0-build.42
PKS Version Name k8s Version Plan Name UUID Status Action
1.11.0-build.46 linux-cluster 1.20.6 linux-small 2ed3bec8-1710-48f4-9917-19599e6cdcd0 succeeded CREATE
tkgi cluster linux-cluster
Upgrade is available to PKS Version: 1.12.0-build.42
PKS Version: 1.11.0-build.46
Name: linux-cluster
K8s Version: 1.20.6
Plan Name: linux-small
UUID: 2ed3bec8-1710-48f4-9917-19599e6cdcd0
Last Action: CREATE
Last Action State: succeeded
Last Action Description: Instance provisioning completed
Kubernetes Master Host: linux-cluster.corp.tanzu
Kubernetes Master Port: 8443
Worker Nodes: 1
Kubernetes Master IP(s): 10.40.14.42
Network Profile Name:
Kubernetes Profile Name:
Compute Profile Name:
Tags:
You can see similar information from the TKGIMC UI:
Having the TKGIMC in use makes upgrading clusters incredibly easy.
Just select the appropriate cluster and click the Upgrade button.
The best way to keep an eye on the upgrade is with the bosh and tkgi commands:
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 353
Task 353 | 14:56:35 | Deprecation: Global 'properties' are deprecated. Please define 'properties' at the job level.
Task 353 | 14:56:37 | Preparing deployment: Preparing deployment
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:50 | Preparing deployment: Preparing deployment (00:00:13)
Task 353 | 14:56:50 | Preparing deployment: Rendering templates (00:00:07)
Task 353 | 14:56:58 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 353 | 14:56:58 | Compiling packages: nsx-cni/3e9b647679f5ca57037c7eaf3d3292ca75b07fb5
Task 353 | 14:56:58 | Compiling packages: jq/c6a6daa7f64fc4775d11c0d4441d9fcf49506746
Task 353 | 14:56:58 | Compiling packages: nsx-python27/75df9f63298d0d2644c6030b160b9b7486a9c195
Task 353 | 14:56:58 | Compiling packages: nsx-cni-common/864a3d57e17b9bd533a7a9ac17af0fe1935b5926
tkgi tasks
ID Type Status StartTime EndTime Clusters
bf37a147-0ae9-4441-90f9-2cbdcbe662d9 UPGRADE in progress Thu, 23 Sep 2021 08:56:34 MDT --- linux-cluster
tkgi task bf37a147-0ae9-4441-90f9-2cbdcbe662d9
Your UPGRADE task is: in progress
Name Status Start time End time isCanary
linux-cluster in progress Thu, 23 Sep 2021 08:56:34 MDT --- false
Just like with the other TKGI components, you should see a lot of activity in vSphere as the old nodes are deleted and the new nodes are created:
You can see that the upgrade is finished in multiple places:
bosh task
Using environment '172.31.0.3' as client 'ops_manager'
Task 353
Task 353 | 14:56:35 | Deprecation: Global 'properties' are deprecated. Please define 'properties' at the job level.
Task 353 | 14:56:37 | Preparing deployment: Preparing deployment
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:39 | Warning: DNS address not available for the link provider instance: pivotal-container-service/fc1121b1-accc-48f0-9f30-ef72aec6959a
Task 353 | 14:56:50 | Preparing deployment: Preparing deployment (00:00:13)
Task 353 | 14:56:50 | Preparing deployment: Rendering templates (00:00:07)
Task 353 | 14:56:58 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 353 | 14:56:58 | Compiling packages: nsx-cni/3e9b647679f5ca57037c7eaf3d3292ca75b07fb5
Task 353 | 14:56:58 | Compiling packages: jq/c6a6daa7f64fc4775d11c0d4441d9fcf49506746
Task 353 | 14:56:58 | Compiling packages: nsx-python27/75df9f63298d0d2644c6030b160b9b7486a9c195
Task 353 | 14:56:58 | Compiling packages: nsx-cni-common/864a3d57e17b9bd533a7a9ac17af0fe1935b5926
Task 353 | 14:59:01 | Compiling packages: jq/c6a6daa7f64fc4775d11c0d4441d9fcf49506746 (00:02:03)
Task 353 | 14:59:01 | Compiling packages: ncp_rootfs/4fad3aa0fb55f62a11e4dc1fcd2d718ed5e109d1
Task 353 | 14:59:09 | Compiling packages: nsx-cni/3e9b647679f5ca57037c7eaf3d3292ca75b07fb5 (00:02:11)
Task 353 | 14:59:12 | Compiling packages: nsx-cni-common/864a3d57e17b9bd533a7a9ac17af0fe1935b5926 (00:02:14)
Task 353 | 15:00:19 | Compiling packages: ncp_rootfs/4fad3aa0fb55f62a11e4dc1fcd2d718ed5e109d1 (00:01:18)
Task 353 | 15:01:44 | Compiling packages: nsx-python27/75df9f63298d0d2644c6030b160b9b7486a9c195 (00:04:46)
Task 353 | 15:01:44 | Compiling packages: openvswitch/e350e39dbe0019e57494e4410d7f532ebfab177c (00:03:47)
Task 353 | 15:06:02 | Updating instance master: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:06:02 | L executing pre-stop: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:06:03 | L executing drain: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:06:05 | L stopping jobs: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:06:29 | L executing post-stop: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:09:00 | L installing packages: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:09:47 | L configuring jobs: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:09:47 | L executing pre-start: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:10:28 | L starting jobs: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary)
Task 353 | 15:10:53 | L executing post-start: master/c899fd74-5060-461f-bec1-05de2b29566e (0) (canary) (00:04:59)
Task 353 | 15:11:01 | Updating instance worker: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:11:02 | L executing pre-stop: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:11:41 | L executing drain: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:11:45 | L stopping jobs: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:12:03 | L executing post-stop: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:14:38 | L installing packages: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:15:50 | L configuring jobs: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:15:50 | L executing pre-start: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:16:17 | L starting jobs: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary)
Task 353 | 15:16:43 | L executing post-start: worker/745b1ef9-c38b-4da6-9dee-6a8a25c021e2 (0) (canary) (00:07:09)
Task 353 Started Thu Sep 23 14:56:35 UTC 2021
Task 353 Finished Thu Sep 23 15:18:10 UTC 2021
Task 353 Duration 00:21:35
Task 353 done
Succeeded
tkgi task bf37a147-0ae9-4441-90f9-2cbdcbe662d9
Your UPGRADE task is: done
Name Status Start time End time isCanary
linux-cluster succeeded Thu, 23 Sep 2021 08:56:34 MDT Thu, 23 Sep 2021 09:21:23 MDT false
You can see here that the Kubernetes version has been upgraded to 1.21.3. From the command line, you can see the same and that the TKGI version has been updated:
tkgi clusters
PKS Version Name k8s Version Plan Name UUID Status Action
1.12.0-build.42 linux-cluster 1.21.3 linux-small 2ed3bec8-1710-48f4-9917-19599e6cdcd0 succeeded UPGRADE
tkgi cluster linux-cluster
PKS Version: 1.12.0-build.42
Name: linux-cluster
K8s Version: 1.21.3
Plan Name: linux-small
UUID: 2ed3bec8-1710-48f4-9917-19599e6cdcd0
Last Action: UPGRADE
Last Action State: succeeded
Last Action Description: Instance upgrade completed
Kubernetes Master Host: linux-cluster.corp.tanzu
Kubernetes Master Port: 8443
Worker Nodes: 1
Kubernetes Master IP(s): 10.40.14.42
Network Profile Name:
Kubernetes Profile Name:
Compute Profile Name:
Tags:
What happened to my Windows plans?
I don’t have any Windows clusters deployed but I did create plans for them and upload a Windows stemcell (you can see more about this in my earlier post, TGKI 1.9 with Windows workers). However, when I went to deploy a Windows cluster in my upgraded TKGI environment, I quickly noticed that I had no Windows plans any longer.
tkgi plans
Name ID Description
linux-small 8A0E21A8-8072-4D80-B365-D1F502085560 This plan will configure a lightweight Kubernetes cluster. Not recommended for production workloads.
linux-medium 58375a45-17f7-4291-acf1-455bfdc8e371 Example: This plan will configure a medium sized Kubernetes cluster, suitable for more pods.
Windows plans need to in spots 11 through 13 and sure enough, these were inactive in Opsman on the TKGI tile (I was only using Plan 11 and Plan 12):
I started to dig further and saw that my Windows stemcell did not appear to be present:
Oddly enough, checking for the stemcell from the command line gave conflicting results:
bosh stemcells
Using environment '172.31.0.3' as client 'ops_manager'
Name Version OS CPI CID
bosh-vsphere-esxi-ubuntu-xenial-go_agent 621.141* ubuntu-xenial e5fb7b6ede9a1242b8eb sc-eed15e83-ea38-4eb3-99c4-8c34fafde669
~ 621.125 ubuntu-xenial e5fb7b6ede9a1242b8eb sc-3ab644c7-6337-4090-8da1-99aa13a5a003
bosh-vsphere-esxi-windows2019-go_agent 2019.36 windows2019 e5fb7b6ede9a1242b8eb sc-dfb4061a-429a-4fb8-828f-2bb52c755f56
(*) Currently deployed
3 stemcells
So bosh knows that there is a Windows stemcell present but Opsman does not.
Since this is all running in a vCD vApp it’s not terribly difficult to go back and double-check the configuration prior to starting the upgrade:
tkgi plans
Name ID Description
linux-small 8A0E21A8-8072-4D80-B365-D1F502085560 This plan will configure a lightweight Kubernetes cluster. Not recommended for production workloads.
linux-medium 58375a45-17f7-4291-acf1-455bfdc8e371 Example: This plan will configure a medium sized Kubernetes cluster, suitable for more pods.
windows-small 228fe8fa-8e98-44f1-aaf3-5f931b995840 small windows plan
windows-medium 612cc499-ff58-45be-acf9-2f9f72bfa236 medium Windows plan
bosh stemcells
Using environment '172.31.0.3' as client 'ops_manager'
Name Version OS CPI CID
bosh-vsphere-esxi-ubuntu-xenial-go_agent 621.125* ubuntu-xenial e5fb7b6ede9a1242b8eb sc-3ab644c7-6337-4090-8da1-99aa13a5a003
bosh-vsphere-esxi-windows2019-go_agent 2019.36 windows2019 e5fb7b6ede9a1242b8eb sc-dfb4061a-429a-4fb8-828f-2bb52c755f56
(*) Currently deployed
2 stemcells
It turns out that this is not entirely unexpected. If you deploy TKGI via the TKGIMC, there is no ability to automate the configuration of Windows plans and upload Windows stemcells. This is largely due to the fact that the Windows stemcells are not managed/delivered by VMware as they must be manually created. To have Windows plans available via the TKGIMC, you must manually create them in Opsman, upload the appropriate Windows stemcell, Apply Changes and then synchronize the configuration in TKGIMC. I had done this in my 1.11 TKGIMC installation but didn’t realize that I would need to do it again after upgrading to 1.12. The Windows stemcell is no longer present in Opsman as this is a fresh deployment with just the settings from the old Opsman imported. The Windows plans cannot be created since there is no stemcell for them to use. If any Windows clusters had been deployed, the tkgi clusters command would actually error out as the Windows plans would no longer be available.
In the Opsman UI, navigate to the TKGI tile and then click on Plan 11. Fill out the information for the plan as appropriate:
Name: windows-small
Type: Windows
Description: small windows plan
Master/etcd node instances: 1
Master/etcd availability zones: tkg-comp-1
Master persistent disk size: 10 GB
Master/etcd VM type: medium.disk (cpu: 2, ram: 4 GB, disk: 32 GB)
Worker node instances: 1
Worker persistent disk size: 20 GB
Worker VM Type: automatic xlarge (cpu: 4, ram: 16GB, disk: 32GB)
Worker Availability Zone: tkg-comp-1
Errand VM type: medium.disk (cpu: 2, ram: 4 GB, disk: 32 GB)
Repeat for Plans 12 and 13 if you’re using them as well.
Navigate to Installation Dashboard and click on the Missing Stemcell link on the TKGI tile.
Click the Import Stemcell button. Select an appropriate Windows stemcell file. Once the file is imported, you can associate it with TKGI (these are typically much larger than the Linux stemcells so the import process may take a while):
Click the Save button here and then navigate to Installation Dashboard.
Click the Review Pending Changes button
You can see that the Windows stemcell will be uploaded for use with TKGI. Click the Apply Changes button.
When this is done, you should see the Windows plans and stemcell everywhere but the TKGIMC UI.
bosh stemcells
Using environment '172.31.0.3' as client 'ops_manager'
Name Version OS CPI CID
bosh-vsphere-esxi-ubuntu-xenial-go_agent 621.141* ubuntu-xenial e5fb7b6ede9a1242b8eb sc-eed15e83-ea38-4eb3-99c4-8c34fafde669
~ 621.125 ubuntu-xenial e5fb7b6ede9a1242b8eb sc-3ab644c7-6337-4090-8da1-99aa13a5a003
bosh-vsphere-esxi-windows2019-go_agent 2019.40 windows2019 e5fb7b6ede9a1242b8eb sc-150d5b7a-8dcb-48a6-b0ca-2c9af615be89
~ 2019.36 windows2019 e5fb7b6ede9a1242b8eb sc-dfb4061a-429a-4fb8-828f-2bb52c755f56
(*) Currently deployed
4 stemcellstkgi plans
Name ID Description
linux-small 8A0E21A8-8072-4D80-B365-D1F502085560 This plan will configure a lightweight Kubernetes cluster. Not recommended for production workloads.
linux-medium 58375a45-17f7-4291-acf1-455bfdc8e371 Example: This plan will configure a medium sized Kubernetes cluster, suitable for more pods.
windows-small 228fe8fa-8e98-44f1-aaf3-5f931b995840 small windows plan
windows-medium 612cc499-ff58-45be-acf9-2f9f72bfa236 medium Windows plan
We still need to synchronize the changes made in Opsman to TKGIMC. This is done by navigating to the TKGI Configuration page and clicking on the Generate Configuration button.
Click on the Apply Configuration button.
The process will look very similar to what was observed during the upgrade, albeit much faster since very little is actually being done.
When this is finished, you should see the Windows plans as available options when creating a cluster in the TKGIMC UI:
Update the tkgi/pks/kubectl binaries
This is pretty minor but you don’t want to forget it. You can download the TKGi 1.12 pks/tkgi/kubectl binaries from Pivotal.
Once you have the files download, you can use the following steps to install them.
install kubectl-linux-amd64-1.21.3 /usr/local/bin/kubectl
kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.6+vmware.1", GitCommit:"088f01db2ffab397a290be443902918b59ee032c", GitTreeState:"clean", BuildDate:"2021-04-17T01:01:00Z", GoVersion:"go1.15.10", Compiler:"gc", Platform:"linux/amd64"}
install tkgi-linux-amd64-1.12.0-build.312 /usr/local/bin/tkgi
tkgi --version
PKS CLI version: 1.12.0-build.312
install pks-linux-amd64-1.12.0-build.312 /usr/local/bin/pks
pks --version
PKS CLI version: 1.12.0-build.312Cleaning up
You can delete the old TKGIMC VM if you no longer need it.
If you have any unwanted Windows stemcells in bosh, you can delete them with the bosh delete-stemcell command.
bosh delete-stemcell bosh-vsphere-esxi-windows2019-go_agent/2019.36
Using environment '172.31.0.3' as client 'ops_manager'
Continue? [yN]: y
Task 438. Done
Succeeded