💡 Rook-Ceph Distributed Storage Cluster Helm — Executive Summary
Technical architectural analysis, practical configuration manifests, and deployment instructions for Rook-Ceph Distributed Storage Cluster Helm.
References and Guidelines (NOTE)
Important note:
- You must customize the ‘CephCluster’ resource in the sample manifest for your cluster.
- Each CephCluster must be deployed in its own namespace, the sample uses ‘rook-ceph’ for the namespace.
- The sample manifest assumes that you have also installed the rook-ceph operator in the ‘rook-ceph’ namespace.
- The Helm chart includes all the RBAC required to create a CephCluster CRD in the same namespace.
- All disk devices you add to the cluster in ‘CephCluster’ must be empty (no file systems and partitions).
rook-ceph-operator
bash
helm repo add rook https://charts.rook.io/release
helm pull rook/rook-ceph --untar
helm install -n rook-ceph rook-ceph . --create-namespacerook-ceph-cluster
Prerequisites
- Proceed after installing the operator
bash
helm pull rook/rook-ceph-cluster --untar
helm install -n rook-ceph rook-ceph-cluster .Confirm installation complete
bash
kubectl get all -n rook-ceph
>>>
NAME READY STATUS RESTARTS AGE
pod/rook-ceph-operator-fd9bc499d-zr8vc 1/1 Running 0 27m
pod/csi-cephfsplugin-4vg2g 2/2 Running 0 7m38s
pod/csi-rbdplugin-q7hxn 2/2 Running 0 7m38s
pod/csi-rbdplugin-provisioner-7879747877-cthpr 5/5 Running 0 7m38s
pod/csi-cephfsplugin-provisioner-674fb56756-h99rw 5/5 Running 0 7m38s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/csi-cephfsplugin 1 1 1 1 1 <none> 7m38s
daemonset.apps/csi-rbdplugin 1 1 1 1 1 <none> 7m38s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rook-ceph-operator 1/1 1 1 27m
deployment.apps/csi-rbdplugin-provisioner 1/1 1 1 7m39s
deployment.apps/csi-cephfsplugin-provisioner 1/1 1 1 7m39s
NAME DESIRED CURRENT READY AGE
replicaset.apps/rook-ceph-operator-fd9bc499d 1 1 1 27m
replicaset.apps/csi-rbdplugin-provisioner-7879747877 1 1 1 7m39s
replicaset.apps/csi-cephfsplugin-provisioner-674fb56756 1 1 1 7m39s
<<<
kubectl get sc -n rook-ceph
>>>
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ceph-bucket rook-ceph.ceph.rook.io/bucket Delete Immediate false 8m31s
ceph-filesystem rook-ceph.cephfs.csi.ceph.com Delete Immediate true 8m31s
ceph-block (default) rook-ceph.rbd.csi.ceph.com Delete Immediate true 8m31s
<<<CRD lookup
bash
kubectl get \
cephclients.ceph.rook.io,\
objectbucketclaims.objectbucket.io,\
objectbuckets.objectbucket.io,\
cephobjectzonegroups.ceph.rook.io,\
cephobjectrealms.ceph.rook.io,\
cephbucketnotifications.ceph.rook.io,\
cephblockpools.ceph.rook.io,\
cephfilesystemmirrors.ceph.rook.io,\
cephbuckettopics.ceph.rook.io,\
cephcosidrivers.ceph.rook.io,\
cephblockpoolradosnamespaces.ceph.rook.io,\
cephfilesystemsubvolumegroups.ceph.rook.io,\
cephobjectstoreusers.ceph.rook.io,\
cephobjectstores.ceph.rook.io,\
cephrbdmirrors.ceph.rook.io,\
cephfilesystems.ceph.rook.io,\
cephobjectzones.ceph.rook.io,\
cephnfses.ceph.rook.io,\
cephclusters.ceph.rook.io \
-n rook-cephRemove
bash
helm uninstall -n rook-ceph rook-ceph-cluster
helm uninstall -n rook-ceph rook-ceph
#crd finalizer update
for CRD in $(kubectl get crd -n rook-ceph | awk '/ceph.rook.io/ {print $1}'); do
kubectl get -n rook-ceph "$CRD" -o name | \
xargs -I {} kubectl patch -n rook-ceph {} --type merge -p '{"metadata":{"finalizers": []}}'
done
sudo rm -rf /var/lib/rookIf the namespace is still in terminating state, check which resources are pending deletion and remove their finalizers as well.
bash
kubectl api-resources --verbs=list --namespaced -o name \
| xargs -n 1 kubectl get --show-kind --ignore-not-found -n rook-ceph
Leave a Reply