How to monitor OpenEBS cStor pools?

In the previous blog, we discussed how to monitor OpenEBS volumes. In that post, I primarily talked about monitoring volume controllers of the storage engines jiva and cstor. In this post, I want to cover how to monitor cstor-pools using Prometheus. From this, we can pull metrics from maya-exporter sidecar of cstor-pool pods and Grafana to visualize the metrics.

Monitoring cstor pools

It’s been almost a year since we released cstor, and we feel there is a need for a continuous monitoring tool to monitor cstor-pool containers to obtain details about various metrics such as health of cstor-pool container, number of reads/writes, number of io’s, rebuilding status, throughput, latency etc. So, we introduced maya-exporter in the 0.9.0 release as a sidecar of cstor-pool pod as well, which collects metrics from cstor-pool container.

Prerequisites

• Openebs version > 0.9.0
• cstor pool must be up and running
• PVC (*optional)

PVC is optional because the pool should be available before creating pvc’s.

Steps:

1. Run kubectl create -f openebs-monitoring-pg.yaml
2. Run kubectl get pods -o wide to verify and determine where Prometheus and Grafana instances are running.
3. Run kubectl get nodes -o wide to obtain the node’s IP.
4. Run kubectl get svc and note the port allocated to Prometheus and Grafana services.
5. Open your browser, then open <nodeip:nodeport>(32514 for Prometheus and 32515 for Grafana). Here, you will be able to see the Prometheus’s expression browser and Grafana’s UI on your browser.
6. Enter your login information, then add the data source name as prometheus. Select datasource type Prometheus and pass the <nodeip:nodeport> of Prometheus in the url field. Then click on run and test.
7. Go to the next tab (Dashboard) and import the desired dashboard. You should be able to get the preloaded dashboard of prometheus.
8. To create a pool dashboard, go to the import button and paste the json.
9. You should now be able to obtain the graph of various metrics that provide info from OpenEBS pools and volumes in the UI by clicking the desired items from the dropdown menu.

Note: I am using NodePort here only for demonstration purposes. Please use service names or ingress to access if you wish to expose the Grafana and Prometheus.

If you have already read my previous blog, many of the steps are very similar. The only difference is that the dashboard json for pool metrics is different and we have added configurations to pull metrics from OpenEBS pools.

I hope you have enjoyed this post and your pool dashboard is up and running. If you encounter any issues, don’t hesitate to reach out to us on slack or comment. There are many other important metrics that are planned for future releases, which will help in visualizing OpenEBS from various perspectives.

That’s all folks! 😃