openshift origin multi-master manually deployment part-1

i have deployed an openshift origin muliti-master cluster successfully on centos and suse enterprise server.
Since the deployment is done manually, i will describe it in detail as possibly as i can.

first, take a whole view of architecture,
we have three master nodes, all of them are in active state. the number of nodes is according to your cluster size. we do not need a lb node, so no haproxy is needed. we setup a VIP floating in three master nodes. for this architecture, no loadbanlance muchinasm is provided, but it is okay for high availibilty. If you realy need a loadbanlance, it is easy to deploy one after the whole cluster is up.

next, for deployment environment.
three master nodes, based on centos minimal installation, with static ip address.
hostname: ip address

i will break this deployment into four parts.
part1: deploy named, keepalived, and docker
part2: deploy master service
part3: deploy etcd cluster with ssl enabled
part4: deploy node service
part5: deploy router and docker registry service

okay, let’s start.

1, disable default firewalld service on all nodes

systemctl stop firewalld
systemctl disable firewalld 

2, we need a dns service, i use named instead of dnsmasq, and install it on all three master node.

yum install -y bind

configure bind for all three master nodes.

vi /etc/named.conf, modify the following entries:

listen-on port 53 { any; };
allow-query     {; };
dnssec-enable no;
dnssec-validation no;

vi /etc/named.rfc1912.zones, add the following zones.

zone "" IN {
    type master;
    file "";
    allow-update { none; };

zone "cluster.local" IN {
    type forward;
    forward only;
    forwarders { port 8053; };
zone "kubernetes.default" IN {
    type forward;
    forward only;
    forwarders { port 8053; };
zone "openshift.local" IN {
    type forward;
    forward only;
    forwarders { port 8053; };

NOTE1: for openshift internal serivce dnslookup, we just forword it to openshift skydns service.
NOTE2: is mine, change it to yours.

create file
vi /var/named/

@       IN SOA rname.invalid. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
@       IN NS @
master1 A
master2 A
master3 A
node1   A
*.route A

enable and start named service

systemctl enable named
systemctl start named

3, install keepalived on all three master nodes.

yum install -y keepalived

backup old configuration

mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.bak

create new configuration file with contents below.
vi /etc/keepalived/keepalived.conf

global_defs {
  router_id master1

vrrp_instance 60 {
  virtual_router_id 60
  advert_int 1
  priority 101
  state BACKUP
  interface enp0s3
  virtual_ipaddress { dev enp0s3
  unicast_peer {


router_id should be updated for each master node, master1 for master node 1,
master2 for master node 2 .etc.
enp0s3 should be updated to your eth0 interface name.
we configure keepalived to use unicast to communicate with each other, so, the
src ip and peer ip should be updated on different master node. for example, on master 2,
src ip is, and peer is 206,208.

enable and start keepalived.

systemctl enable keepalived
systemctl start keepalived

to verify it is okay, run “ip a” on every master node, you can see that vip is
on one of your master node.

last, add in /etc/resolv.conf on every node.
vi /etc/resolv.conf

4, install and setup docker, add parameters to docker daemon, on all nodes

yum install -y docker

vim /etc/sysconfig/docker

OPTIONS=' --selinux-enabled --log-driver=json-file --log-opt max-size=50m'

do not start docker service at this time.


  1. Please provide password to me, secondly have you tried manually installation of openshift 3.11(OKD), as they did lot of changes

Leave a Reply

Your email address will not be published. Required fields are marked *