In my previous post, Enable and use Data Protection in TMC, I walked through the process of using the Data Protection feature in TMC to backup and restore a workload. While this works great, it has a limitation in that you must use an Amazon S3 bucket for the backup storage location. While this might work in many scenarios, it won’t help if you have an on-prem location you’d like to use for storing your backups (or just something that is not an Amazon S3 bucket). TMC recently introduced a new feature, Target Locations, that lest you specify a custom backup destination to be used with Data Protection.
In my labs, I’ve been using MinIO deployed via a TrueNAS VM for storing my Velero backups. Since Data Protection is using Velero behind the scenes, MinIO should work well for this exercise.
The first step is to make sure you have a bucket created. I already had one bucket created (velero) but decided to create a new one just for use with Data Protection. In MinIO this is very simple as you just click the giant red Plus sign at the bottom right of the page.
Click on the Create bucket icon and give the new bucket a name.
It’s not exactly intuitive but you only have to click Enter to finish the creation of the bucket at this point. And you should see the new bucket visible in the MinIO UI.
With that out of the way, you can start the process of creating the custom Target Location. You must first create a new credential that will be associated with the custom Target Location though. From the Administration Menu, select the Account tab and then click on the Create Account Credential button. There is a new option entitled Customer Provisioned Storage Credential S3 that you will need to select.
You only need to provide a name for the credential and username/password (Access key id and Secret access key). These are the same as the login to MinIO (minio/minio123).
After you click the Create button you should see the new credential (cjl-minio in this example) present on the Accounts page.
The new tab on the Administration page is named Target locations and this is where you can configure the custom backup location. If you previously configured a data protection credential for AWS S3, you should see a target location with the same name as that credential. To create a custom location, click the Create Target Location button and then select Customer provisioned S3-compatible storage.
Select the previously created credential and click the Next button.
Specify the MinIO URL, Bucket and Region. The concept of Region doesn’t really work with MinIO and is typically associated with an AWS region but it is required for Velero to work so you can simply specify minio as the region.
Select an appropriate Cluster Group whose member clusters can make use of this backup location. You need to click the Add button after selecting the Cluster Group.
Click the Next button once the appropriate cluster group is selected.
Give the Target Location a name and click the Create button.
You should now see the new Target Location present on the Target Locations tab.
My lab consists of a vSphere 7 with Tanzu (7.0 U2) supervisor cluster with a single Tanzu Kubernetes cluster named tkg-cluster. tkg-cluster is managed by TMC and you can see that Data Protection has not been enabled for it yet.
Enabling Data Protection is as simple as clicking the Enable Data Protection link and then clicking the Enable button.
While Data Protection is being enabled, the TMC UI will show a spinning circle to let you know that the operation is still in progress.
Within the cluster, you should see a velero namespace get created.
kubectl get ns
NAME STATUS AGE
default Active 66d
kube-node-lease Active 66d
kube-public Active 66d
kube-system Active 66d
velero Active 2m39s
vmware-system-auth Active 66d
vmware-system-cloud-provider Active 66d
vmware-system-csi Active 66d
vmware-system-tmc Active 16hAnd once the installation is complete, you should see a single velero pod and a restic pod for each worker node in the velero namespace.
kubectl -n velero get po
NAME READY STATUS RESTARTS AGE
restic-6xflm 1/1 Running 0 3m
restic-wsdpc 1/1 Running 0 3m
velero-5866578fdf-hjm4j 1/1 Running 0 3m37sBack in the TMC UI, you can go to the Data Protection tab and see that there are no backups present.
You can click the Create Backup link to get into the process of backing up workloads in your cluster. I’m going to create a backup of the default namespace which has a WordPress application running in it.
Since the cjl-minio Target Location is associated with the clittle Cluster Group, and the tkg-cluster cluster is in the clittle Cluster Group, the available Target Location in the backup wizard is cjl-minio. With this select, you can click the Next button.
Since I am backing up now and not concerned with modifying the retention time for the backup, I was able to just click Next on the following two screens.
The last step is to give the backup a name. With that done, click the Create button to start the backup.
You’ll see the backup in a Pending state for short while in the TMC UI.
Within the cluster, we can see that the cjl-minio Target Location has been configured as a backupstoragelocation (a CRD created when Velero is installed).
kubectl -n velero get backupstoragelocations
NAME AGE
cjl-minio 14mWe can get a little more information about this backupstoragelocation to confirm that it matches the configuration specified in the TMC UI.
kubectl -n velero get backupstoragelocations cjl-minio -o json | jq '.spec'
{
"config": {
"bucket": "cjl-v7tanzu-bucket",
"profile": "cjl-minio",
"publicUrl": "http://192.168.110.60:9000/",
"region": "minio",
"s3ForcePathStyle": "true",
"s3Url": "http://192.168.110.60:9000/"
},
"objectStorage": {
"bucket": "cjl-v7tanzu-bucket",
"prefix": "01F6G1T00GBBQTHBWM0M9TGWSP/"
},
"provider": "aws"
}The credentials for accessing the new backupstoragelocation are in a secret named cloud-credentials in the velero namespace.
kubectl -n velero get secret cloud-credentials -o 'go-template={{ index .data "cloud" }}' | base64 -d
[cjl-minio]
aws_access_key_id=minio
aws_secret_access_key=minio123Another Velero CRD, backups, should exist and show an instance for the backup that was just kicked off.
kubectl -n velero get backups
NAME AGE
cjl-defaultns-backup 28sIn the MinIO UI, you can see that there is a folder created under the new bucket.
And drilling down into the folder shows several files related to the backup (most named with a prefix that matches the name the backup was given in TMC, cjl-defaultns-backup).
When the backup is finished, the TMC UI will show the status for the backup as Completed.
And at the command line, you can pull some information out of the backup object to see it’s status.
kubectl -n velero get backups cjl-defaultns-backup -o=custom-columns="NAME:.metadata.name,NS:.spec.includedNamespaces,S3:.spec.storageLocation,STATUS:.status.phase"
NAME NS S3 STATUS
cjl-defaultns-backup [default] cjl-minio CompletedYou can read more about this new feature at <doc link>