i have deployed an openshift origin muliti-master cluster successfully on centos and suse enterprise server.
Since the deployment is done manually, i will describe it in detail as possibly as i can.
first, take a whole view of architecture,
we have three master nodes, all of them are in active state. the number of nodes is according to your cluster size. we do not need a lb node, so no haproxy is needed. we setup a VIP floating in three master nodes. for this architecture, no loadbanlance muchinasm is provided, but it is okay for high availibilty. If you realy need a loadbanlance, it is easy to deploy one after the whole cluster is up.
next, for deployment environment.
three master nodes, based on centos minimal installation, with static ip address.
hostname: ip address
master1.openshift.qyos.com 192.168.2.206
master2.openshift.qyos.com 192.168.2.207
master3.openshift.qyos.com 192.168.2.208
node1.openshift.qyos.com 192.168.2.209
VIP 192.168.2.205
i will break this deployment into four parts.
part1: deploy named, keepalived, and docker
part2: deploy master service
part3: deploy etcd cluster with ssl enabled
part4: deploy node service
part5: deploy router and docker registry service
okay, let’s start.
1, disable default firewalld service on all nodes
systemctl stop firewalld systemctl disable firewalld
2, we need a dns service, i use named instead of dnsmasq, and install it on all three master node.
yum install -y bind
configure bind for all three master nodes.
vi /etc/named.conf, modify the following entries:
listen-on port 53 { any; }; allow-query { 0.0.0.0/0; }; dnssec-enable no; dnssec-validation no;
vi /etc/named.rfc1912.zones, add the following zones.
zone "openshift.qyos.com" IN { type master; file "named.openshift.qyos.com"; allow-update { none; }; }; zone "cluster.local" IN { type forward; forward only; forwarders { 192.168.2.205 port 8053; }; }; zone "kubernetes.default" IN { type forward; forward only; forwarders { 192.168.2.205 port 8053; }; }; zone "openshift.local" IN { type forward; forward only; forwarders { 192.168.2.205 port 8053; }; };
NOTE1: for openshift internal serivce dnslookup, we just forword it to openshift skydns service.
NOTE2: 192.168.2.205 is mine, change it to yours.
create named.openshift.qyos.com file
vi /var/named/named.openshift.qyos.com
$TTL 1D @ IN SOA openshift.qyos.com. rname.invalid. ( 0 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum @ IN NS @ A 192.168.2.205 master1 A 192.168.2.206 master2 A 192.168.2.207 master3 A 192.168.2.208 node1 A 192.168.2.209 *.route A 192.168.2.205
enable and start named service
systemctl enable named systemctl start named
3, install keepalived on all three master nodes.
yum install -y keepalived
backup old configuration
mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.bak
create new configuration file with contents below.
vi /etc/keepalived/keepalived.conf
global_defs { router_id master1 } vrrp_instance 60 { virtual_router_id 60 advert_int 1 priority 101 state BACKUP interface enp0s3 virtual_ipaddress { 192.168.2.205 dev enp0s3 } unicast_src_ip 192.168.2.206 unicast_peer { 192.168.2.207 192.168.2.208 } }
NOTE:
router_id should be updated for each master node, master1 for master node 1,
master2 for master node 2 .etc.
enp0s3 should be updated to your eth0 interface name.
we configure keepalived to use unicast to communicate with each other, so, the
src ip and peer ip should be updated on different master node. for example, on master 2,
src ip is 192.168.2.207, and peer is 206,208.
enable and start keepalived.
systemctl enable keepalived systemctl start keepalived
to verify it is okay, run “ip a” on every master node, you can see that vip 192.168.2.205 is
on one of your master node.
last, add 192.168.2.205 in /etc/resolv.conf on every node.
vi /etc/resolv.conf
nameserver 192.168.2.205
4, install and setup docker, add parameters to docker daemon, on all nodes
yum install -y docker
vim /etc/sysconfig/docker
OPTIONS=' --selinux-enabled --log-driver=json-file --log-opt max-size=50m' DOCKER_CERT_PATH=/etc/docker
do not start docker service at this time.
Hi Shaun,
Could you please share password of part2 and 3
Hi, could you send me the password for parts 2 and 3,4,5 …
hi
could you give password for protected parts?
thanks
password sent in mail
Hi, could you send me the password for parts 2 and 3,4,5 …
Thanks!
password sent in mail
Hello ,
Can you send me the password for other parts of this article ?
Thanks
hello, could you please share the password for part 2 and part 3 ?
Hello,
Can you send the password for me please?
Thanks.
password sent in mail
Thanks.
L.
Kindly send me the password.
Thanks.
Hello,
Can you send the password for me please?
Thanks.
Can I have the password for the other part ?? thank you
password sent in mail