Extending Kubernetes to Cross-Cluster Data Mobility
To the benefit of several thousands of users, Kubernetes has become almost ubiquitous and has matured immensely in the management of stateful workloads.
One reason for the success of Kubernetes is that it is open and provides a common set of APIs that provide an abstraction layer for cloud and on-premise operations of cloud native environments.
There are some storm clouds on the horizon, however. Uncertainty related to the direction of Kubernetes exists in management across clusters. This higher level of control and abstraction is being advanced in Kubernetes itself in cluster federation work and is discussed in other projects. However, commercial providers of Kubernetes-based solutions are not waiting. They are moving ahead with additional functionality that extends Kubernetes for common use cases such as cross-platform data migration.
As an example, at MayaData we have extended OpenEBS and the OpenEBS Enterprise Edition to enable data migration as a service, or DMaaS. Unlike functions such as snapshots and clones that are triggered by Kubernetes API calls — the movement of data from one location to another, or the back-up of data, retrieval, or similar functions — these are governed by our own APIs and control plane.
Similarly, Google recently launched services as a part of their multi-cloud Kubernetes that they call “Anthos.” It is governed by proprietary APIs including MOVE and other similar commands.
We would like to suggest an alternate path that forestalls the potential confusion and potential deceleration related to transitioning to the cloud that a possible tragedy of the commons could engender.
Specifically, we would like to assist in uniting a coalition that invests more heavily in the components of Kubernetes that need to progress more quickly in order to avoid the confusion and additional lock-in that, contrary to the spirit of Kubernetes, will likely result from the addition of these proprietary APIs. Anyone and everyone is welcome. All you need is an interest in preserving the promise of Kubernetes as a universal control plane that frees users from lock-in.
With that in mind, I’m happy to announce what we are calling the KUBEMOVE project. We even have a sample logo for the project!
KubeMove: An Operator and API set for Kubernetes data mobility.
Perhaps more importantly, we are publishing an initial proposal for an operator and API set to manage and tackle data mobility.
This project is dependent on contributions from application developers and cloud providers for the design of the APIs as well as implementation of the reference example for a stateful application using Restik, which we call DDM (Dynamic Data Mover). The project can be found on GitHub at github.com/kubemove.
Contributors can show their love and support through the GitHub issue https://github.com/kubemove/kubemove/issues/14
We always welcome your feedback. In fact, we are counting on it! Please take a second and engage with the community via GitHub or by responding to this blog.
Many thanks for taking a look and getting engaged. Feel free to reach out with any comments or questions.
This article was first published on May 19, 2019 on MayaData's Medium Account