On-Demand Volume Snapshots¶
Beginning with the 20.01 release of Trident, it is now possible to use the
Volume Snapshot feature to create snapshots of PVs at the Kubernetes
layer. These snapshots can be used to maintain point-in-time copies of
volumes that have been created by Trident and can also be used to schedule
the creation of additional volumes (clones). This feature is available from
1.17 [Beta] and is GA from
Kubernetes Volume Snapshot is GA from Kubernetes
1.20. To understand the
changes involved in moving from beta to GA, take a look at the release blog.
With the graduation to GA, the
v1 API version is introduced and is backward compatible
Creating volume snapshots requires an external snapshot controller to be created, as well as some Custom Resource Definitions (CRDs). This is the responsibility of the Kubernetes orchestrator that is being used (E.g., Kubeadm, GKE, OpenShift). Check with your Kubernetes orchestrator to confirm these requirements are met. You can create an external snapshot-controller and snapshot CRDs using the code snippet below.
$ cat snapshot-setup.sh #!/bin/bash # Create volume snapshot CRDs kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml # Create the snapshot-controller in the desired namespace. Edit the YAML manifests below to modify namespace. kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml
CSI Snapshotter provides a
to help users validate existing
v1beta1 snapshots and confirm they are valid
resource objects. The validating webhook will automatically label invalid snapshot
objects and prevent the creation of future invalid objects. As with volume snapshot
CRDs and the snapshot controller, the validating webhook is deployed by the
Kubernetes orchestrator. Instructions to deploy the validating webhook manually
can be found here.
Examples of invalid snapshot manifests can be obtained in the
directory of the external-snapshotter.
Volume snapshot is supported by the
azure-netapp-files drivers. This feature requires Kubernetes
1.17 and above.
Trident handles the creation of VolumeSnapshots for its drivers as explained below:
- For the
azure-netapp-filesdrivers, each PV maps to a FlexVol. As a result, VolumeSnapshots are created as NetApp Snapshots. NetApp’s Snapshot technology delivers more stability, scalability, recoverability, and performance than competing snapshot technologies. These Snapshot copies are extremely efficient both in the time needed to create them and in storage space.
- For the
ontap-san-economydriver, PVs map to LUNs created on shared FlexVols. VolumeSnapshots of PVs are achieved by performing FlexClones of the associated LUN. ONTAP’s FlexClone technology makes it possible to create copies of even the largest datasets almost instantaneously. Copies share data blocks with their parents, consuming no storage except what is required for metadata.
- For the
solidfire-sandriver, each PV maps to a LUN created on the Element/HCI cluster. VolumeSnapshots are represented by Element snapshots of the underlying LUN. These snapshots are point-in-time copies and only take up a small amount of system resources and space.
When working with the
ontap-san drivers, ONTAP snapshots are
point-in-time copies of the FlexVol and consume space on the FlexVol itself. This
can result in the amount of writable space in the volume to reduce with time as
snapshots are created/scheduled. One simple way of addressing this is to grow
the volume by resizing through Kubernetes. Another
option is to delete snapshots that are no longer required. When a VolumeSnapshot
created through Kubernetes is deleted, Trident will delete the associated ONTAP
snapshot. ONTAP snapshots that were not created through Kubernetes can also be
With Trident, you can use VolumeSnapshots to create new PVs from them. Creating
PVs from these snapshots is performed by using the FlexClone technology for
supported ONTAP & CVS backends.
When creating a PV from a snapshot, the backing volume is a FlexClone of the
snapshot’s parent volume. The
solidfire-san driver uses ElementOS volume
clones to create PVs from snapshots. Here it creates a clone from the Element
The example detailed below explains the constructs required for working with snapshots and shows how snapshots can be created and used.
Before creating a Volume Snapshot, a VolumeSnapshotClass must be set up.
$ cat snap-sc.yaml #Use apiVersion v1 for Kubernetes 1.20 and above. For Kubernetes 1.17 - 1.19, use apiVersion v1beta1. apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-snapclass driver: csi.trident.netapp.io deletionPolicy: Delete
driver points to Trident’s CSI driver. The
deletionPolicy can be set
Retain. When set to
Retain, the underlying physical snapshot
on the storage cluster is retained even when the
VolumeSnapshot object is deleted.
Create a VolumeSnapshot¶
We can now create a snapshot of an existing PVC.
$ cat snap.yaml #Use apiVersion v1 for Kubernetes 1.20 and above. For Kubernetes 1.17 - 1.19, use apiVersion v1beta1. apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: name: pvc1-snap spec: volumeSnapshotClassName: csi-snapclass source: persistentVolumeClaimName: pvc1
The snapshot is being created for a PVC named
pvc1, and the
name of the snapshot is set to
$ kubectl create -f snap.yaml volumesnapshot.snapshot.storage.k8s.io/pvc1-snap created $ kubectl get volumesnapshots NAME AGE pvc1-snap 50s
It is possible to identify the VolumeSnapshotContent object for the
pvc1-snap VolumeSnapshot by describing it.
$ kubectl describe volumesnapshots pvc1-snap Name: pvc1-snap Namespace: default . . . Spec: Snapshot Class Name: pvc1-snap Snapshot Content Name: snapcontent-e8d8a0ca-9826-11e9-9807-525400f3f660 Source: API Group: Kind: PersistentVolumeClaim Name: pvc1 Status: Creation Time: 2019-06-26T15:27:29Z Ready To Use: true Restore Size: 3Gi . .
Snapshot Content Name identifies the VolumeSnapshotContent
object which serves this snapshot. The
Ready To Use parameter indicates
that the Snapshot can be used to create a new PVC.
Create PVCs from VolumeSnapshots¶
A PVC can be created using the snapshot as shown in the example below:
$ cat pvc-from-snap.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-from-snap spec: accessModes: - ReadWriteOnce storageClassName: golden resources: requests: storage: 3Gi dataSource: name: pvc1-snap kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io
dataSource shows that the PVC must be created using a VolumeSnapshot
pvc1-snap as the source of the data. This instructs Trident
to create a PVC from the snapshot. Once the PVC is created, it can be attached
to a pod and used just like any other PVC.
When deleting a Persistent Volume with associated snapshots, the corresponding Trident volume is updated to a “Deleting state”. For the Trident volume to be deleted, the snapshots of the volume must be removed.