How to install IBM Cloud Private (ICP)

It’s been some time since I wrote about IBM Cloud Private 2.1, although I’ve been heavily using it in my lab.

Improvements from version 1.2 to 2.1 were massive. We have also noticed the changes in the OpenEBS user community, according to our surveys usage of ICP increased dramatically. The Community Edition of the ICP 3.1 came out three weeks ago. CE container images were released with a slight delay of two weeks after the enterprise version announced. And I believe that it deserves an updated blog to talk about the steadily advancing developer experience (and my favorite new features).

In this blog, I will provide step-by-step instructions on how to configure a Kubernetes-based managed private cloud using ICP. I will also highlight the top improvements since the last version that makes me excited.


Quick note: Developers who use only public cloud ask me this question frequently, “Why would you need a private cloud, and maintain it, everything on the public cloud is much easier?”. Well, you are probably not the one paying the cloud bill, right? For various tests, (some running IO intensive workloads) and learning experiments, I maintain 4 clusters (IBM Cloud Private, Rancher, Red Hat OpenShift and one installed with kubeadm) each 3–5 nodes. That’s 16–20 nodes, exactly $1,069.05/month I learned it all the hard way. True that education is expensive, but running the same on a few cheap servers adds only ~$135/month to my electric bill soon will be almost free thanks to the solar panels ;)

When it’s not enough I also use StackPointCloud to deploy temporary clusters on AWS or GKE, etc, but I never keep them overnight unless I have to.

On-premise turnkey solutions that allow me to create Kubernetes clusters on my internal, secure, cloud network with only a few commands like IBM Cloud Private makes my live really easy.

Now, let’s take a look at the requirements.


Minimum requirements for a multi-node community edition cluster:


  • Boot node: 1x 1+ core(s) >= 2.4 GHz CPU, 4GB RAM, >=100 GB disk space
  • Master node: 1x 8+ cores >= 2.4 GHz CPU, 16+GB RAM, >=200 GB disk space
  • Proxy node: 1 or more 2+ cores >= 2.4 GHz CPU, 4GB RAM, >=150 GB disk space
  • Worker node: 1 or more 1+ cores >= 2.4 GHz CPU, 4GB RAM, >=150 GB disk space

Since I’m using the Community Edition this time, I will be using single master and multiple workers configuration.


How to install IBM Cloud Private-CE 3.1

If you are already using the older version of the ICP you can skip the cluster preparation steps and jump to the “Install IBM Cloud Private-CE 3.1” section after uninstalling the older version as described under “Uninstalling an older version of the IBM Cloud Private” section.

We need a few things installed before we get up and running with ICP 3.1. First, I’ll configure my Ubuntu servers and share SSH keys, so the master node can access all my other nodes. Then I’ll install Docker and after that ICP. From there, ICP will take care of my Kubernetes cluster installation.

Install the base O/S — Ubuntu (30–45mins)

Download your preferred version of Ubuntu. I use Ubuntu Server 16.04.3 LTS.

Install Ubuntu on all servers with default options. I used user/nopassword as username/password for simplicity.

Log in to your Ubuntu host via terminal.

Edit the /etc/network/interfaces file, assign a static IP and set a hostname.

Edit the /etc/hosts file, add your nodes to the list, and make sure you can ping them by the hostname:
cat /etc/hosts
For my setup, this is how hosts file looks like:

$ cat /etc/hosts localhost
# icp3101# The following lines are desirable for IPv6 capable hosts
#::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters icp3101 icp3102 icp3103 icp3104 icp3105

Ping nodes by hostname to make sure all are accessible:

$fping icp21032 icp21033 icp21034 icp21035
icp21032 is alive
icp21033 is alive
icp21034 is alive
icp21035 is alive

On your Ubuntu host, install the SSH server:

sudo apt-get install openssh-server

Now, you should be able to access your servers using SSH. Check the status by running:

$ sudo service ssh status
● ssh.service — OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2018–10–08 07:16:37 PDT; 8h ago
Main PID: 1160 (sshd)
Tasks: 1
Memory: 6.8M
CPU: 45.454s
CGroup: /system.slice/ssh.service
└─1160 /usr/sbin/sshd -D

Install open-iscsi and curl if it’s not already installed:

sudo apt install curl open-iscsi

Repeat above steps on all servers.

Now, you need to share SSH keys among all nodes:

Log in to your first node, which will be the boot node (ubuntu36), as root.

Generate an SSH key:

ssh-keygen -b 4096 -t rsa -f ~/.ssh/master.id_rsa -N “”

Add the SSH key to the list of authorized keys:

cat ~/.ssh/ | sudo tee -a ~/.ssh/authorized_keys

From the boot node, add the SSH public key to other nodes in the cluster:

ssh-copy-id -i ~/.ssh/ root;

Repeat for all nodes.

Log in to the other nodes and restart the SSH service:

sudo systemctl restart sshd

Now the boot node can connect through SSH to all other nodes without the password.

Install Docker CE (5mins)

To get the latest supported version of Docker, install it from the official Docker repository.

On your Ubuntu nodes, update the apt package index:

sudo apt-get update

Make sure below packages are installed:

sudo apt-get install \
apt-transport-https \
ca-certificates \

Add Docker’s GPG key:

curl -fsSL | sudo apt-key add -

Add the repository:

sudo add-apt-repository \
“deb [arch=amd64] \
$(lsb_release -cs) \

Update the apt package index:

sudo apt-get update

Check the available versions:

apt-cache madison docker-ce

Install Docker CE

sudo apt-get install docker-ce=18.03.1~ce-0~ubuntu

Make sure it’s up and running after installation is complete:

$ sudo systemctl status docker
● docker.service — Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2018–10–08 07:17:42 PDT; 8h ago
Main PID: 1165 (dockerd)
Tasks: 1442
Memory: 7.2G
CPU: 55min 10.946s
CGroup: /system.slice/docker.service

Install IBM Cloud Private-CE 3.1 (60–75 mins)

Download the IBM Cloud Private-CE container images:

sudo docker pull ibmcom/icp-inception:3.1.0

Create an installation folder for configuration files and extract the sample config file:

mkdir /opt/ibm-cloud-private-ce-3.1.0; \
cd /opt/ibm-cloud-private-ce-3.1.0
sudo docker run -e LICENSE=accept \
-v “$(pwd)”:/data ibmcom/icp-inception:3.1.0 cp -r cluster /data

Above command creates the cluster directory under /opt/ibm-cloud-private-ce-3.1.0 with the following files: config.yaml, hosts, and ssh_key. Before deploying ICP, these files need to be modified.

Replace the ssh_key file with the private SSH key you have created earlier.

sudo cp ~/.ssh/master.id_rsa /opt/cluster/ssh_key

Add the IP address of all your nodes to the hosts file in the /opt/ibm-cloud-private-ce-3.1.0/cluster directory. If you plan to run serious workloads, I recommend separating master and worker Kubernetes nodes. Since Community Edition supports single master node only, my config file looks like this:

$ cat /opt/ibm-cloud-private-ce-3.1.0/cluster/hosts

Finally, deploy the environment. Change directory to the cluster folder with the config.yaml file and deploy your ICP environment:

sudo docker run — net=host -t -e LICENSE=accept \
-v “$(pwd)”:/installer/cluster ibmcom/icp-inception:3.1.0 install

Note: I have tried to deploy 3.1 on my old VMs (where ICP v2.1.0.2 used to run) and my first attempt failed due to increased resource requirements compared to the previous version. If your deployment times-out while waiting for cloudant and you an error similar to below:

TASK [addon : Waiting for cloudant to start] ******************************************
Waiting for cloudant to start (13 retries left).
Waiting for cloudant to start

Double check your h/w resources and run the installer with verbose options to see more details:

docker run -e LICENSE=accept — net=host \
-t -v “$(pwd)”:/installer/cluster \
ibmcom/icp-inception:3.1.0 install -vvv | tee -a install_log.txt

I ended up increased memory from 8GB to 16GB and disk capacity to 200GB (from 150GB) and all worked well after that.

After a successful install you should see a message similar to the below:

 PLAY RECAP ********************************************************************* : ok=173 changed=94 unreachable=0 failed=0 : ok=113 changed=55 unreachable=0 failed=0 : ok=108 changed=51 unreachable=0 failed=0 : ok=108 changed=50 unreachable=0 failed=0 : ok=108 changed=49 unreachable=0 failed=0
localhost : ok=265 changed=161 unreachable=0 failed=0POST DEPLOY MESSAGE ************************************************************The Dashboard URL:, default username/password is admin/admin
Playbook run took 0 days, 0 hours, 53 minutes, 54 seconds

One thing I have noticed is that installation completed much faster than the previous version. ICP got smarter and only pulling images that are required for the roles, also seems like taking advantage of the local image registries. My 5 node installation time dropped from 95 minutes to 53 minutes.

If your deployment is successful, you should be able to access your ICP login screen by visiting https://MASTERNODEIP:8443 (Default username/password is admin/admin).


IBM Cloud Private Login Screen



IBM Cloud Private Dashboard



IBM Cloud Private Catalog

Uninstalling an older version of the IBM Cloud Private

IBM documentation is very clear with upgrade steps described here. It is unfortunate that upgrade is only supported from the release since I had problems installing that version I kept my cluster at and later clean install was the online way to go for me.

I have uninstalled the existing version with the comment below:

sudo docker run -e LICENSE=accept — net=host -t -v “$(pwd)”:/installer/cluster ibmcom/icp-inception: uninstall

Also removed all stopped containers:

docker system prune -af

What’s next?

I will go over the configuration of other optional features in my next blog post as I get more familiar with the new platform.

  • Introduction to IBM Cloud Private #2 — What’s new in 3.1 — quick comparison vs 2.1
  • Introduction to IBM Cloud Private #3 — Catalog Applications
  • Introduction to IBM Cloud Private #4 — How to deploy workloads on OpenEBS
  • Introduction to IBM Cloud Private #5 — Monitoring IBM Cloud Private with Prometheus
  • Introduction to IBM Cloud Private #6 — Use of metrics for monitoring utilization
  • Introduction to IBM Cloud Private #7 — Contributing to the Community Charts
  • Introduction to IBM Cloud Private #8 — Chaos Engineering with Litmus
  • Introduction to IBM Cloud Private #9 — Backup your ICP cluster with OpenEBS & Heptio Ark

Above are some of the content I believe that it would be useful. If you like to see anything not covered here feel free to comment on my blog or contact me via Twitter @muratkarslioglu.

To be continued…


Originally published at Containerized Me.

Kumar Indrajeet
Kumar has 10 years' experience in the design, implementation, management of enterprise distributed systems, and highly available clustered solutions. Expert in Architecting Cloud infrastructure, migration to containers, cost optimization, security enhancements, and predictive monitoring. Certified Professional with expertise in AWS, Openstack, Linux, and DevOps concepts. Persuasive communicator with good analytical, logical, and multitasking skills. Adroit at analyzing and understanding organizational requirements.
Niladri is a CSAT Solutions / Support Engineer at MayaData. He works with Kubernetes, Docker, and the rest of the stack down below.
Kiran Mova
Kiran Mova is a Passionate Technologist with 17 years of extensive experience working for product companies like Cisco, Lucent, Novell. Kiran has led efforts in performance engineering, simplifying the products in terms of usability, operation and deployment complexities, introducing multi-tenancy and enabling SAAS services in domains like IP/Optical Networks, Storage and Access Management. At MayaData, Kiran leads overall architecture and is responsible for architecting, solution design and customer adoption of OpenEBS and related software. Kiran evangelizes open culture and open source execution models and is a lead maintainer and contributor to the OpenEBS project Passionate about Storage Orchestration. Contributor and Maintainer OpenEBS projects. Chief Architect MayaData Inc. Open Source Dreamer