KUBEMOVE: Avoiding the Tragedy of the Commons

Extending Kubernetes to Cross-Cluster Data Mobility

To the benefit of several thousands of users, Kubernetes has become almost ubiquitous and has matured immensely in the management of stateful workloads.

One reason for the success of Kubernetes is that it is open and provides a common set of APIs that provide an abstraction layer for cloud and on-premise operations of cloud native environments.

There are some storm clouds on the horizon, however. Uncertainty related to the direction of Kubernetes exists in management across clusters. This higher level of control and abstraction is being advanced in Kubernetes itself in cluster federation work and is discussed in other projects. However, commercial providers of Kubernetes-based solutions are not waiting. They are moving ahead with additional functionality that extends Kubernetes for common use cases such as cross-platform data migration.

As an example, at MayaData we have extended OpenEBS and the OpenEBS Enterprise Edition to enable data migration as a service, or DMaaS. Unlike functions such as snapshots and clones that are triggered by Kubernetes API calls — the movement of data from one location to another, or the back-up of data, retrieval, or similar functions — these are governed by our own APIs and control plane.

Similarly, Google recently launched services as a part of their multi-cloud Kubernetes that they call “Anthos.” It is governed by proprietary APIs including MOVE and other similar commands.

We would like to suggest an alternate path that forestalls the potential confusion and potential deceleration related to transitioning to the cloud that a possible tragedy of the commons could engender.

Specifically, we would like to assist in uniting a coalition that invests more heavily in the components of Kubernetes that need to progress more quickly in order to avoid the confusion and additional lock-in that, contrary to the spirit of Kubernetes, will likely result from the addition of these proprietary APIs. Anyone and everyone is welcome. All you need is an interest in preserving the promise of Kubernetes as a universal control plane that frees users from lock-in.

With that in mind, I’m happy to announce what we are calling the KUBEMOVE project. We even have a sample logo for the project!

KubeMove: An Operator and API set for Kubernetes data mobility.

KubeMove: An Operator and API set for Kubernetes data mobility.

Perhaps more importantly, we are publishing an initial proposal for an operator and API set to manage and tackle data mobility.

This project is dependent on contributions from application developers and cloud providers for the design of the APIs as well as implementation of the reference example for a stateful application using Restik, which we call DDM (Dynamic Data Mover). The project can be found on GitHub at github.com/kubemove.

Contributors can show their love and support through the GitHub issue https://github.com/kubemove/kubemove/issues/14

We always welcome your feedback. In fact, we are counting on it! Please take a second and engage with the community via GitHub or by responding to this blog.

Many thanks for taking a look and getting engaged. Feel free to reach out with any comments or questions.

This article was first published on May 19, 2019 on MayaData's Medium Account


Utkarsh Mani Tripathi
Utkarsh is a maintainer of jiva project and has contributed in building both control and data plane of OpenEBS. He loves to learn about file-system, distributed systems and networking. Currently, he is mainly focusing on enhancing jiva and maya-exporter In his free time, he loves to write poems and make lip smacking dishes
Chuck Piercey
Chuck Piercey is a Silicon Valley product manager with experience shipping more than 15 products in several different market segments representing a total of $2.5Bn revenue under both commercial and open source business models. Most recently he has been working for MayaData, Inc. focused on software-defined storage, network, and compute for Kubernetes environments. Chuck occasionally writes articles about the technology industry.
Sagar Kumar
Sagar is a software engineer at Mayadata who loves coding and solving real-world problems. He has been playing with Kubernetes for the last couple of years. Currently, he is focused on building OpenEBS Director as the go-to solution for OpenEBS users. In his free time, he loves playing cricket and traveling.