Computers and servers come with internal storage devices accessible to only one machine and not any other. In personal computers, these could be Hard Disk Drives (HDDs) or Solid State Drives (SSDs). In Servers, such local drives for storage are known as Direct Attached Storage (DAS).
DAS can also refer to systems including drives that are directly attached to a server. In this case DAS can actually use often routed protocols however it would not be using switches or routers in the path. For example, you can connect to devices and to directly attached systems through:
Again, DAS storage non-networked. External drives and enclosures that can be connected through USB, SATA, or other interfaces can also be part of DAS. A key aspect of DAS is that all storage devices are controlled by a single machine to which it is attached. Any device that intends to access data from the DAS storage has to be authenticated and authorized by the machine attached to the DAS.
There are various types of devices available to expose DAS storage to servers and workstations. These include:
A Hard Disk Drive (HDD) is an electro-magnetic data storage device that stores and retrieves information from spinning magnetic cylinders. Hard Disks are affordable, high-capacity storage devices that allow enterprises to keep large amounts of data for little money. These drives are, however, prone to mechanical damage and should only be used in areas with little movement. Since they are made of moving, rotating platters, HDDs are prone to wear, and typically last 5-10 years.
These drives use persistent electronic memory to store data. Electronic memory is characterized by super fast read/write times and is only limited by the interfaces used by the DAS devices to access data. There are two types of SSD devices:
HDDs and SSDs can be grouped together to form a Redundant Arrays of Inexpensive Disks (RAID). A RAID can be thought of as a single disk composed of many smaller disks. This type of storage virtualization improves capacity, performance and can also be used to provide data redundancy to recover from the loss of a single disk.
RAID can be enabled for both SSDs and HDDs:
This configuration offers the least expensive method of having a large storage capacity. RAID can help address the relative fragility of HDDs by ensuring that systems are resilient despite single or even multiple disk failures. . As disk drives have become larger without increasing substantially their average speed - rebuild times have become a major concern. Rebuild times of days can be possible for multi TB disks, resulting in the increased use of RAID approaches such as RAID-6 that have two parity disks for resiliency in order to reduce or eliminate the risk of data loss due to two disk failures.
SSD based RAID can be much faster, reducing the risk of a second disk loss during a rebuild, with some configurations capable of achieving speeds of 3000 MBps. Since SSD memory drives cannot suffer mechanical damage, these RAIDs are rugged. They can support any form of content, from light files to large, high-definition multimedia editing. These RAIDs are, however, more expensive to set up and offer a limited storage capacity when compared to HDD RAIDs.
The CNCF project OpenEBS is amongst the most popular means of deploying many different types of Local Persistent Volumes to create direct-storage that is accessible from a single node of a cluster. For PODs to use the Local Volume for data storage, they are scheduled on the node into which the volume has been provisioned.
To get started in creating DAS storage for a Kubernetes node using OpenEBS Local Persistent Volume, OpenEBS should be installed on the node using the command:
$ kubectl apply -f https://openebs.github.io/charts/openebs-operator.yamlThis command installs OpenEBS in default mode, which is enough for an experimental cluster.
To install OpenEBS in custom mode, additional details on advanced configuration options can be found here.
OpenEBS uses a Dynamic Provisioner to create Persistent Volumes for Kubernetes Cluster Node. While doing so, a default approach is to save persistent data using OpenEBS Local hostPath Volumes.
To learn more about the different flavors of LocalPV available from OpenEBS and how users like Flipkart and ByteDance (TikTok) utilize them, please read this blog from MayaData co-founder and OpenEBS project lead Kiran Mova: https://openebs.io/blog/how-are-tiktok-flipkart-kubesphere-and-others-using-openebs-for-local-volumes/
To proceed with the default installation mode, hostPath volumes are created under the /var/openebs/local directory.
For custom installation, a StorageClass named local-hostpath-sc.yaml with a custom BasePath is created with values similar to:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-hostpath
annotations:
openebs.io/cas-type: local
cas.openebs.io/config: |
- name: StorageType
value: hostpath
- name: BasePath
value: /var/local-hostpath
provisioner: openebs.io/local
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumerNote to edit the name and BasePath to match the production environment.
This StorageClass is then created using the command:
$ kubectl apply -f local-hostpath-sc.yamlA PersistentVolumeClaim local-hostpath-pvc.yaml that the PODs will use to request the hostPath from the provisioner is created with specification similar to:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: local-hostpath-pvc
spec:
storageClassName: openebs-hostpath
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5GThe PVC is then created using the command:
$ kubectl apply -f local-hostpath-pvc.yamlA POD is then created to access data from OpenEBS storage using the hostPath. The POD’s manifest file will contain specification similar to:
apiVersion: v1
kind: Pod
metadata:
name: hello-local-hostpath-pod
spec:
volumes:
- name: local-storage
persistentVolumeClaim:
claimName: local-hostpath-pvc
containers:
- name: hello-container
image: busybox
command:
- sh
- -c
- 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
volumeMounts:
- mountPath: /mnt/store
name: local-storageUsing the specification above, a POD is created using the command:
$ kubectl apply -f local-hostpath-pod.yamlThese steps collectively when run in order, create a Persistent Local Volume on the Node that the POD local-hostpath-pod.yaml can read and write data onto.
OpenEBS for Provisioning Direct Storage
DAS offers an efficient storage solution that can provide higher throughput and reduced latency. Additionally DAS can reduce the complexity and the blast radius of managing stateful workloads.In Kubernetes, Local PVs represent a directly-attached local disk or cloud volume that is attached to a single Kubernetes Node.
OpenEBS extends the agility and affordability of DAS to Kubernetes through the Persistent Volume Dynamic Provisioner. Through the dynamic provisioner, OpenEBS initiates volume provisioning to application PODS and implements the Kubernetes specification for PVs. This solution persists data on a node so that PODs scheduled to it can consume data stored in the volume.
While using OpenEBS as the preferred storage solution for Kubernetes stateful workloads, users have earlier reported several benefits, including:
OpenEBS also offers replicated storage and other capabilities for those uses that would like more out of their Container Attached Storage than DAS or LocalPV can offer and yet do not want to use traditional shared storage.
References:
https://mayadata.io/assets/pdf/WP-OpenEBS-0_7.pdf
https://thenewstack.io/how-kubernetes-provides-networking-and-storage-to-applications/
https://docs.openebs.io/docs/next/cas.html