APIs, Competition & Kubernetes

In the early 2000s, the internet was dominated by a central marketplace for all manner of things.

It was a time of desktop computing, and somehow, it seemed that a natural monopoly had emerged almost overnight. And Craigslist was our source for places to live, used goods, gigs, entertainment, and much more.

While there were commercial alternatives, like Renters.com, Craigslist seemed to have an insurmountable lead. Everything was available. And it just kept innovating with new categories, pulling us farther and farther into its comfortable clutches.

And then…The iPhone and the AppStore enabled the rise of best of breed competitors that were singularly focused on particular markets. Countless unicorns sprung from the primordial soup of the desktop world and Craigslist.

Today history seems to be rhyming. Another simple white background GUI that keeps innovating is dominating the world - this time, the world of cloud computing.

And it seems truly unstoppable—almost a natural monopoly. The more services it launches, the more eyeballs/developers it attracts, and the more developers it attracts, the more critical mass it has to launch the next set of services. It’s Craigslist - this time for developers. And then…

What can this rhyming history tell us of what is to come?

Will we see increased freedom for consumers? Will they soon have an equal to or superior user experience without always relying on the ever-broader central service? If the parallel holds, we will see AWS decline in relative importance, though continuing to grow, as best of breed competitors emerge.

These best of breed alternatives started their rise well before Kubernetes, of course, leveraging common APIs that AWS already needed to support. Examples include IAM alternatives such as OKTA and S3 alternatives that are API compliant such as Cloudian and even Minio for secure and cloud independent object storage. APM and monitoring are already not dominated by AWS CloudWatch. CloudWatch is widely seen as an ok, but not great, solution for operations, and it already seems to have missed the mark with developers. DataDog, AppDynamics, and others in the broader monitoring and logging space are flourishing.

Kubernetes and the necessary embrace of it by Amazon appears to be creating an opening for other best of class providers to emerge as well.

Examples include, of course, Kubernetes itself - for orchestration - and service meshes refactoring and replacing traditional service meshes from AWS, such as cloud-native leaders Istio and Linkerd.

We see all manner of databases flourishing on Kubernetes, often smaller databases that can be spun up and destroyed efficiently and then tied together via a pub/sub architecture supported by technologies such as Kafka. The benefits from running your own database apparently often already outweigh the pain of operating them, even though DB operators and underlying data management systems such as OpenEBS are only now maturing. And if you do want to buy DB as a service, the growth of microservice architectures, facilitated by Kubernetes though not strictly dependent upon Kubernetes, is one reason that Atlas from MongoDB and other similar offerings are doing so well.

This is true at all levels of the stack. From the lowest level, data engines speak to NVMe drives and cloud volumes such as OpenEBS and the related NDM for device management. At the highest levels, entire code and data pipelines from the likes of GitLab and Pachyderm proliferate. The rise of Kubernetes and related trends such as microservices architectures facilitated by open APIs is leading to an unprecedented explosion in systems that enable software developers to achieve real agility in building and operating software at scale. This is the next step of how software is eating the world.

It seems increasingly likely that we will look back on the “everything to everyone” business models of AWS and similar clouds with nostalgia, much as many of us remember fondly our first forays buying services and goods over the internet via Craigslist and eBay.

Our early team at MayaData predicted this disaggregation trend some years ago and started extending Kubernetes itself to the data layer. Today users from Bloomberg to Orange, Comcast, and Arista and many thousands of others are utilizing the OpenEBS data management project, which is now housed within the CNCF. Please take a look and provide us with your feedback and your wish list. The only way we get better is with your help.