Designating OpenEBS pods to run on specific K8S Nodes

OpenEBS does not have a separate scheduler used to manage scheduling pods. Instead, it uses the Kubernetes scheduler for managing the scheduling needs of an administrator. Kubernetes provides the following methods for controlling the scheduling pods on cluster nodes:

For more details and information about these features, you can refer to the Kubernetes documentation.

Designating OpenEBS Pods to run on specific K8s nodes

In this post, I would like to cover the different aspects of how to restrict/control the OpenEBS pods scheduling to a set of specific nodes in the Kubernetes cluster.

OpenEBS deals with many types of Kubernetes pods throughout its life cycle. These can be broadly categorized into two types, control plane pods and data plane pods. Control plane pods are installed as part of installation of the following OpenEBS components:

  • OpeEBS API Server
  • OpenEBS Provisioner
  • OpenEBS Snapshot Controller

Data plane pods are installed as part of volume provisioning:

  • Target Pod
  • Replica Pods

For details on the exact steps of scheduling, see the configuration section here.

Use case: Let’s consider a scenario in which you have 20 nodes named Node1, Node2 ... Node20. You may want to designate Node1, Node2, Node3 as storage nodes so that all storage pods are scheduled only on these nodes.

Solution: You can use Kubernetes scheduling methods to achieve this. Below are some of the possible options and their effect on scheduling pods to respective nodes.

image1 (2)

You may select and use any of the above options based on your unique requirements.

Option 2 does not necessarily guarantee storage pod scheduling on Node1, Node2 and Node3.

Option 1, Option 3, and Option 4 will limit the scheduled OpenEBS pods to Node1, Node2 and Node3. Option 3 is my preferred choice, for the following reasons:

  • Other application pods will not be scheduled on my storage nodes, whereas Option 1 does present the possibility of other application pods being scheduled on Node1, Node2, and Node3.
  • While scaling the cluster for application deployments, I do not have to worry about changing the policy for storage. If I use option4, I must taint the new nodes with respective applications.
  • In this scenario, I am worried only about storage nodes, as these have local disks attached to them. I am not restricted to schedule an application pod deployment on any nodes other than storage nodes.

This article was first published on Jul 31, 2018 on OpenEBS's Medium Account

Prateek Pandey
Software Engineer at MayaData
Murat Karslioglu
VP @OpenEBS & @MayaData_Inc. Murat Karslioglu is a serial entrepreneur, technologist, and startup advisor with over 15 years of experience in storage, distributed systems, and enterprise hardware development. Prior to joining MayaData, Murat worked at Hewlett Packard Enterprise / 3PAR Storage in various advanced development projects including storage file stack performance optimization and the storage management stack for HPE’s Hyper-converged solution. Before joining HPE, Murat led virtualization and OpenStack integration projects within the Nexenta CTO Office. Murat holds a Bachelor’s Degree in Industrial Engineering from the Sakarya University, Turkey, as well as a number of IT certifications. When he is not in his lab, he loves to travel, advise startups, and spend time with his family. Lives to innovate! Opinions my own!
Evan Powell
Founding CEO of a few companies including StackStorm (BRCD) and Nexenta — and CEO & Chairman of OpenEBS/MayaData. ML and DevOps and Python, oh my!