Designating OpenEBS pods to run on specific K8S Nodes

OpenEBS does not have a separate scheduler used to manage scheduling pods. Instead, it uses the Kubernetes scheduler for managing the scheduling needs of an administrator. Kubernetes provides the following methods for controlling the scheduling pods on cluster nodes:

For more details and information about these features, you can refer to the Kubernetes documentation.

In this post, I would like to cover the different aspects of how to restrict/control the OpenEBS pods scheduling to a set of specific nodes in the Kubernetes cluster.

OpenEBS deals with many types of Kubernetes pods throughout its life cycle. These can be broadly categorized into two types, control plane pods and data plane pods. Control plane pods are installed as part of installation of the following OpenEBS components:

  • OpeEBS API Server
  • OpenEBS Provisioner
  • OpenEBS Snapshot Controller

Data plane pods are installed as part of volume provisioning:

  • Target Pod
  • Replica Pods

For details on the exact steps of scheduling, see the configuration section here.

Use case: Let’s consider a scenario in which you have 20 nodes named Node1, Node2 ... Node20. You may want to designate Node1, Node2, Node3 as storage nodes so that all storage pods are scheduled only on these nodes.

Solution: You can use Kubernetes scheduling methods to achieve this. Below are some of the possible options and their effect on scheduling pods to respective nodes.

Stoarge pods

You may select and use any of the above options based on your unique requirements.

Option 2 does not necessarily guarantee storage pod scheduling on Node1, Node2 and Node3.

Option 1, Option 3, and Option 4 will limit the scheduled OpenEBS pods to Node1, Node2 and Node3. Option 3 is my preferred choice, for the following reasons:

  • Other application pods will not be scheduled on my storage nodes, whereas Option 1 does present the possibility of other application pods being scheduled on Node1, Node2, and Node3.
  • While scaling the cluster for application deployments, I do not have to worry about changing the policy for storage. If I use option4, I must taint the new nodes with respective applications.
  • In this scenario, I am worried only about storage nodes, as these have local disks attached to them. I am not restricted to schedule an application pod deployment on any nodes other than storage nodes.

This article was first published on Jul 31, 2018 on OpenEBS's Medium Account

Kiran Mova
Kiran Mova is a Passionate Technologist with 17 years of extensive experience working for product companies like Cisco, Lucent, Novell. Kiran has led efforts in performance engineering, simplifying the products in terms of usability, operation and deployment complexities, introducing multi-tenancy and enabling SAAS services in domains like IP/Optical Networks, Storage and Access Management. At MayaData, Kiran leads overall architecture and is responsible for architecting, solution design and customer adoption of OpenEBS and related software. Kiran evangelizes open culture and open source execution models and is a lead maintainer and contributor to the OpenEBS project Passionate about Storage Orchestration. Contributor and Maintainer OpenEBS projects. Chief Architect MayaData Inc. Open Source Dreamer
Akhil Mohan
Building Mayadata one commit at a time.
Uma Mukkara
Umasankar Mukkara (Uma) has over 20 years of experience as a hands-on developer, architecting scalable products and building innovative teams. Uma led product development in the early days of MayaData (CloudByte). He has led multiple innovations in multi-tenant storage and has contributed to more than 10 patents. Prior to CloudByte, he has contributed significantly to the development of Access Management solutions and has a sound understanding of cloud storage and security architecture. Uma also spends significant time in building chaos engineering practices in OpenEBS development eco-system, community engagement, and partner eco-system development. Uma holds a Masters degree in Telecommunications and software engineering from Illinois Institute of Technology, Chicago and a bachelor’s degree in Communications from S.V.University, Tirupati, India. Contributor at openebs.io, Co-founder& COO@MayaData