Chaos Engineering of stateful applications

In Kubecon Copenhagen 2018; we formally launched Litmus as a framework to introduce chaos engineering into stateful applications on Kubernetes. Since then, we have added more features to Litmus to make it usable natively on Kubernetes, we added more chaos tests, added Litmus books using KOPS to setup K8S clusters on AWS and GCP, and some more features.

Untitled design (3)

Litmus integration with MayaOnline for chaos engineering

For example, DevOps operators managing a Gitlab CI pipeline on Kubernetes can now build the chaos stage easily and quickly using these approaches:

kubectl apply -f induce-replica-chaos-mongodb.yaml

Making Litmus easy to use for developers in Kubernetes

Today MayaData announces support for Litmus integration into MayaOnline. Now cloud-native application developers requiring persistent storage can use MayaOnline not only to get great visibility of their application and simplified management of OpenEBS deployments but also to introduce chaos tests quickly to validate and harden the application for usage with either OpenEBS or Kubernetes local PV.

Quick recap — what is Litmus?

Before discussing on how Litmus books are invoked on MayaOnline or on a CI pipeline, let us quickly recap what Litmus is.

Litmus is a framework or set of tools put together to help manage the lifecycle of chaos engineering of applications on Kubernetes. Currently, the focus of Litmus is on adding chaos tests to OpenEBS software itself and to well-known stateful applications such as MongoDB or Percona.

Litmus books are broadly categorized into four types.

  1. K8S infrastructure books (github.com/openebs/litmus/k8s)
  2. Stateful applications deployment books (github.com/openebs/litmus/apps/percona/deployers/)
  3. Stateful applications chaos books (github.com/openebs/litmus/apps/percona/chaos/)
  4. Deployers for providers such as OpenEBS. Other storage providers helper functionality will go here

Developers and operators clone the litmus repository and change the configuration in the litmus books to suit their Kubernetes cluster and the particular application. Finally, litmus books are invoked by the usual kubectl command.

Each Litmus runs as a Kubernetes job and returns the result as a Kubernetes custom resource object. If you know how to use and operate and manage Kubernetes jobs and resources, using litmus is no different. It’s easy.

Using Litmus books, operators can easily add new chaos stage and developers can use MayaOnline to run Litmus books to harden their stateful applications.

 

Reference users of Litmus

The fact that Litmus was started from the chaos principles followed in the OpenEBS project and its development pipelines, We expect Litmus books to be liberally used by developers in their CI pipelines written to run on Kubernetes. OpenEBS CI and E2E systems are now restructured to use Litmus books, and you can see them in action at https://openebs.ci. This CI and E2E project is open sourced and is accessible on the GitHub at https://github.com/openebs/e2e-infrastructure

Litmus is available as a hosted version on MayaOnline. Developers can make use of Litmus books if their DevOps operators have integrated Litmus into the CI pipelines. There is also a simpler way to run chaos tests during development.

⁃ Connect the Kubernetes cluster to MayaOnline

⁃ Login into MayaOnline and setup Litmus repository for the cluster with your github credentials

⁃ Choose your application, choose your chaos tests, and MayaOnline will run them on the cluster and report the results

Yes, now chaos engineering for stateful applications is that simple

Contributions to Litmus project, our e2e project are welcome and encouraged. Tell us a specific stateful application chaos scenario you would like see a Litmus book for. Or better yet, send a PR.

This article was first published on Aug 29, 2018 on MayaData's Medium Account

 

 

Kiran Mova
Kiran Mova is a Passionate Technologist with 17 years of extensive experience working for product companies like Cisco, Lucent, Novell. Kiran has led efforts in performance engineering, simplifying the products in terms of usability, operation and deployment complexities, introducing multi-tenancy and enabling SAAS services in domains like IP/Optical Networks, Storage and Access Management. At MayaData, Kiran leads overall architecture and is responsible for architecting, solution design and customer adoption of OpenEBS and related software. Kiran evangelizes open culture and open source execution models and is a lead maintainer and contributor to the OpenEBS project Passionate about Storage Orchestration. Contributor and Maintainer OpenEBS projects. Chief Architect MayaData Inc. Open Source Dreamer
Akhil Mohan
Building Mayadata one commit at a time.
Uma Mukkara
Umasankar Mukkara (Uma) has over 20 years of experience as a hands-on developer, architecting scalable products and building innovative teams. Uma led product development in the early days of MayaData (CloudByte). He has led multiple innovations in multi-tenant storage and has contributed to more than 10 patents. Prior to CloudByte, he has contributed significantly to the development of Access Management solutions and has a sound understanding of cloud storage and security architecture. Uma also spends significant time in building chaos engineering practices in OpenEBS development eco-system, community engagement, and partner eco-system development. Uma holds a Masters degree in Telecommunications and software engineering from Illinois Institute of Technology, Chicago and a bachelor’s degree in Communications from S.V.University, Tirupati, India. Contributor at openebs.io, Co-founder& COO@MayaData