Upgrading/Downgrading Trident¶
Upgrading Trident¶
This section walks you through the upgrade process to move to the latest release of Trident.
Initiate the upgrade¶
Note
Before upgrading Trident, ensure that the required feature gates are enabled.
Warning
Trident only supports the beta feature release of Volume Snapshots. When upgrading Trident, all previous alpha snapshot CRs and CRDs (Volume Snapshot Classes, Volume Snapshots and Volume Snapshot Contents) must be removed before the upgrade is performed. Refer to this blog to understand the steps involved in migrating alpha snapshots to the beta spec.
The best way to upgrade to the latest version of Trident is to download the latest installer bundle and run:
./tridentctl uninstall -n <namespace>
./tridentctl install -n <namespace>
When upgrading to the latest release of Trident, there are a few items to consider:
- Previous releases of Trident may have supported alpha Volume Snapshots. The 20.01 release moves forward to only support beta Volume Snapshots. Kubernetes admins must take care to safely backup or convert the alpha snapshot objects to the beta feature set if they would like to retain the legacy alpha snapshots.
- The beta feature release of Volume Snapshots introduces a modified set of CRDs and a snapshot controller, both of which should be set up before installing Trident.
This blog discusses the steps involved in migrating alpha Volume Snapshots to the beta format. It is purely meant to be a guiding document for users to understand the steps involved should they choose to retain alpha Volume Snapshots.
Upgrading Trident on Kubernetes 1.13¶
On Kubernetes 1.13, there are a couple of options when upgrading Trident:
- Install Trident in the desired namespace by executing the
tridentctl installcommand with the--csiflag. This configures Trident to function as an enhanced CSI provisioner. This will require enabling some feature gates. - If for some reason the feature gates required by Trident
cannot be enabled, you can install Trident without the
--csiflag. This will configure Trident to work in its traditional format without using the CSI specification.
Upgrading Trident on Kubernetes 1.14 and above¶
For Kubernetes 1.14 and greater, simply perform an uninstall followed by
a reinstall to upgrade to the latest version of Trident.
What happens when you upgrade¶
By default the uninstall command will leave all of Trident’s state intact by not deleting the PVC and PV used by the Trident deployment, allowing an uninstall followed by an install to act as an upgrade.
The 19.07 release of Trident will move away from etcd and use CRDs to maintain
state. Upgrading Trident from any release prior to 19.07 by running a tridentctl install will:
- initiate a one-time process that copies the metadata stored in the Trident PV into CRD objects.
- provide periodic updates throughout the upgrade with respect to the migration of Trident’s metadata.
Warning
When upgrading Trident, do not interrupt the upgrade process. Ensure that the installer runs to completion.
PVs that have already been provisioned will remain available while Trident is offline, and Trident will provision volumes for any PVCs that are created in the interim once it is back online.
Next steps¶
When upgrading on Kubernetes versions 1.14 and above, the
CSI provisioner will be used by default. For legacy PVs, all features made available
by the previous Trident version will be supported.
To make use of the rich set of features that will be provided in newer
Trident releases (such as On-Demand Volume Snapshots),
volumes can be upgraded using the tridentctl upgrade
command.
The Upgrading legacy volumes to CSI volumes section explains how legacy non-CSI volumes can be upgraded to the CSI type.
If you encounter any issues, visit the troubleshooting guide for more advice.
Upgrading legacy volumes to CSI volumes¶
When upgrading Trident, there is a possibility of having legacy volumes that need to be ported to the CSI specification to make use of the complete set of features made available by Trident. A legacy PV that has been provisioned by Trident will still support the traditional set of features. For all additional features that Trident will provide (such as On-Demand Volume Snapshots), the Trident volume must be upgraded from a NFS/iSCSI type to the CSI type.
Before proceeding, you must determine if your Trident deployment is capable
of upgrading legacy volumes.
The version of Trident installed should be at least 19.07 and Trident should be
configured as a CSI Provisioner. This can be confirmed by doing a kubectl get pods -n <trident-namespace>.
The presence of a trident-csi-<generated-id> indicates that it is running as a CSI
provisioner and supports upgrading legacy volumes.
$ tridentctl version
+----------------+----------------+
| SERVER VERSION | CLIENT VERSION |
+----------------+----------------+
| 19.10.0 | 19.10.0 |
+----------------+----------------+
$ kubectl get pods -n <trident-namespace>
NAME READY STATUS RESTARTS AGE
trident-csi-426nx 2/2 Running 0 20m
trident-csi-b5cf8fd7c-fnq24 4/4 Running 0 20m
Things to consider when upgrading volumes¶
When deciding to upgrade volumes to the CSI type, make sure to consider the following:
- It may not be required to upgrade all existing volumes. Previously created volumes will still continue to be accessible and function normally.
- A PV can be mounted as part of a Deployment/StatefulSet when upgrading. It is not required to bring down the Deployment/StatefulSet.
- A PV cannot be attached to a standalone pod when upgrading. You will have to shut down the pod before upgrading the volume.
- To upgrade a volume, it must be bound to a PVC. Volumes that are not bound to PVCs will need to be removed and imported before upgrading.
Example volume upgrade¶
Here is an example that shows how a volume upgrade is performed.
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
default-pvc-1-a8475 1073741824 RWO Delete Bound default/pvc-1 standard 19h
default-pvc-2-a8486 1073741824 RWO Delete Bound default/pvc-2 standard 19h
default-pvc-3-a849e 1073741824 RWO Delete Bound default/pvc-3 standard 19h
default-pvc-4-a84de 1073741824 RWO Delete Bound default/pvc-4 standard 19h
trident 2Gi RWO Retain Bound trident/trident 19h
There are currently 4 PVs that have been created by Trident 19.04, using the
netapp.io/trident provisioner.
$ kubectl describe pv default-pvc-2-a8486
Name: default-pvc-2-a8486
Labels: <none>
Annotations: pv.kubernetes.io/provisioned-by: netapp.io/trident
volume.beta.kubernetes.io/storage-class: standard
Finalizers: [kubernetes.io/pv-protection]
StorageClass: standard
Status: Bound
Claim: default/pvc-2
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 1073741824
Node Affinity: <none>
Message:
Source:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 10.xx.xx.xx
Path: /trid_1907_alpha_default_pvc_2_a8486
ReadOnly: false
The PV was created using the netapp.io/trident provisioner and is of type NFS.
To support all new features provided by Trident, this PV will need to be upgraded
to the CSI type.
To upgrade a legacy Trident volume to the CSI spec, you must execute the
tridenctl upgrade volume <name-of-trident-volume> command.
$ ./tridentctl get volumes -n trident
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
| NAME | SIZE | STORAGE CLASS | PROTOCOL | BACKEND UUID | STATE | MANAGED |
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
| default-pvc-2-a8486 | 1.0 GiB | standard | file | c5a6f6a4-b052-423b-80d4-8fb491a14a22 | online | true |
| default-pvc-3-a849e | 1.0 GiB | standard | file | c5a6f6a4-b052-423b-80d4-8fb491a14a22 | online | true |
| default-pvc-1-a8475 | 1.0 GiB | standard | file | c5a6f6a4-b052-423b-80d4-8fb491a14a22 | online | true |
| default-pvc-4-a84de | 1.0 GiB | standard | file | c5a6f6a4-b052-423b-80d4-8fb491a14a22 | online | true |
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
$ ./tridentctl upgrade volume default-pvc-2-a8486 -n trident
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
| NAME | SIZE | STORAGE CLASS | PROTOCOL | BACKEND UUID | STATE | MANAGED |
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
| default-pvc-2-a8486 | 1.0 GiB | standard | file | c5a6f6a4-b052-423b-80d4-8fb491a14a22 | online | true |
+---------------------+---------+---------------+----------+--------------------------------------+--------+---------+
After upgrading the PV, performing a kubectl describe pv will show
that the volume is a CSI volume.
$ kubectl describe pv default-pvc-2-a8486
Name: default-pvc-2-a8486
Labels: <none>
Annotations: pv.kubernetes.io/provisioned-by: csi.trident.netapp.io
volume.beta.kubernetes.io/storage-class: standard
Finalizers: [kubernetes.io/pv-protection]
StorageClass: standard
Status: Bound
Claim: default/pvc-2
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 1073741824
Node Affinity: <none>
Message:
Source:
Type: CSI (a Container Storage Interface (CSI) volume source)
Driver: csi.trident.netapp.io
VolumeHandle: default-pvc-2-a8486
ReadOnly: false
VolumeAttributes: backendUUID=c5a6f6a4-b052-423b-80d4-8fb491a14a22
internalName=trid_1907_alpha_default_pvc_2_a8486
name=default-pvc-2-a8486
protocol=file
Events: <none>
In this manner, volumes of the NFS/iSCSI type that were created by Trident can be upgraded to the CSI type, on a per-volume basis.
Downgrading Trident¶
This section examines the various factors involved when attempting to move back to an earlier release of Trident and its implications. There could be certain reasons that require considering a downgrade path:
- Contingency planning
- Immediate fix for bugs observed as a result of an upgrade
- Dependency issues, unsuccessful and incomplete upgrades
With the 19.07 release, Trident introduced
Custom Resource Definitions(CRDs)
to store Trident’s metadata. This marked a move from previous versions,
which used a dedicated etcd instance to write the metadata to a PV created
and managed by Trident for this purpose (the trident PV). A
greenfield installation of Trident versions 19.07 and above will create these CRDs
and use the CRDs to maintain its state. Upgrading from any version of Trident that
used the legacy etcd [versions 19.04 and below] to versions 19.07 and above
will involve
migrating the etcd contents to create CRD objects.
The trident volume still lives on the storage cluster.
When to downgrade¶
Only consider a downgrade when moving to a Trident release that uses CRDs.
Since Trident now uses CRDs for maintaining state, all
storage entities created (backends, storage classes, PVs and Volume Snapshots) have
associated CRD objects instead of data written into the trident PV [used by the
earlier installed version of Trident]. Newly created PVs, backends and Storage
Classes are all maintained as CRD objects. If you need to downgrade, this should
only be attempted for a version of Trident that runs using CRDs (19.07 and above).
This way, all operations performed on the current Trident release are visible once
the downgrade occurs.
When not to downgrade¶
You must not downgrade to a release of Trident that uses etcd to maintain state (19.04 and below). All operations performed with the current Trident release will not be reflected after the downgrade. Newly created PVs will not be usable when moving back to an earlier version. Changes made to objects such as backends, PVs, storage classes and Volume Snapshots (created/updated/deleted) will not be visible to Trident when downgraded moving back to an earlier version. Going back to an earlier version will not disrupt access for PVs that were already created using the older release, unless they have been upgraded.
How to downgrade¶
After understanding when to downgrade/not downgrade, these
are the steps involved in moving down to an earlier release. This sequence walks you
through the downgrade process to move from Trident 19.10 to 19.07
1. Before beginning the downgrade, it is recommended to take a snapshot of your
Kubernetes cluster’s etcd. This allows you to backup the current state of Trident’s
CRDs.
2. Uninstall Trident with the existing tridentctl binary. In this case, you will
uninstall with the 19.10 binary.
$ tridentctl version -n trident
+----------------+----------------+
| SERVER VERSION | CLIENT VERSION |
+----------------+----------------+
| 19.10.0 | 19.10.0 |
+----------------+----------------+
$ tridentctl uninstall -n trident
INFO Deleted Trident deployment.
INFO Deleted Trident daemonset.
INFO Deleted Trident service.
INFO Deleted Trident secret.
INFO Deleted cluster role binding.
INFO Deleted cluster role.
INFO Deleted service account.
INFO Deleted pod security policy. podSecurityPolicy=tridentpods
INFO The uninstaller did not delete Trident's namespace in case it is going to be reused.
INFO Trident uninstallation succeeded.
3. Obtain the Trident binary for the desired version [19.07]
and use it to install Trident.
Generate custom yamls for a customized installation
if needed.
$ cd 19.07/trident-installer/
$ ./tridentctl install -n trident-ns
INFO Created installer service account. serviceaccount=trident-installer
INFO Created installer cluster role. clusterrole=trident-installer
INFO Created installer cluster role binding. clusterrolebinding=trident-installer
INFO Created installer configmap. configmap=trident-installer
...
...
INFO Deleted installer cluster role binding.
INFO Deleted installer cluster role.
INFO Deleted installer service account.