Deploy Kubernetes cluster 1.11.3 on Centos 7 in Google Cloud Platform

Summary:

Kubernetes is a cluster and orchestration engine for Docker containers. In other words Kubernetes is an open source software or tool which is used to orchestrate and manage Docker containers in cluster environment. 

Kubernetes can be installed and deployed using following methods:

• Minikube ( It is a single node kubernetes cluster)
• Kops ( Multi node kubernetes setup into AWS )
• Kubeadm ( Multi Node Cluster in our own premises)

Master Node components:

• API Server  – It provides kubernetes API using Jason / Yaml over http, states of API objects are stored in etcd
• Scheduler  – It is a program on master node which performs the scheduling tasks like launching containers in worker nodes based on resource availability
• Controller Manager – Main Job of Controller manager is to monitor replication controllers and create pods to maintain desired state.
• etcd – It is a Key value pair data base. It stores configuration data of cluster and cluster state.
• Kubectl utility – It is a command line utility which connects to API Server on port 6443. It is used by administrators to create pods, services etc.

Worker Nodes components:

Kubelet – It is an agent which runs on every worker node, it connects to Docker  and takes care of creating, starting, deleting containers.

• Kube-Proxy – It routes the traffic to appropriate containers based on IP address and port number of the incoming request. In other words we can say it is used for port translation.
• Pod – Pod can be defined as a multi-tier or group of containers that are deployed on a single worker node or Docker host.

Installation steps of Kubernetes on CenOS 7

Environment:
Google Cloud Platform
Google Compute Engine ( not GKE )

On Master Node

Disable swap

# swapoff -a

Edit: /etc/fstab

# vi /etc/fstab

Comment out swap

#/root/swap swap swap sw 0 0

Add the Kubernetes repo

# cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

!!!  Edit /etc/yum.repos.d/google-cloud.repo  and add exclude=kube* to avoid kubectl version update

Disable SELinux

# setenforce 0

Permanently disable SELinux:

# vi /etc/selinux/config

Change enforcing to disabled

SELINUX=disabled

Install Kubernetes 1.11.3 and docker

# yum install -y docker kubelet-1.11.3 kubeadm-1.11.3 kubectl-1.11.3 kubernetes-cni-0.6.0 --disableexcludes=kubernetes

Start and enable the Kubernetes and Docker service

# systemctl start docker && systemctl enable docker
# systemctl start kubelet && systemctl enable kubelet

Create the k8s.conf file:

# cat << EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# sysctl --system

# echo '1' > /proc/sys/net/ipv4/ip_forward



Disable firewall

# systemctl stop firewalld && systemctl disable firewalld

Create kube-config.yml:

# vi kube-config.yml

Add the following to kube-config.yml:

apiVersion: kubeadm.k8s.io/v1alpha1
kind:
kubernetesVersion: "v1.11.3"
networking:
  podSubnet: 10.244.0.0/16
apiServerExtraArgs:
  service-node-port-range: 8000-31274

Initialize Kubernetes

# kubeadm init --config kube-config.yml

Copy admin.conf to your home directory

# mkdir -p $HOME/.kube
# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# chown $(id -u):$(id -g) $HOME/.kube/config

!! Use below command to add worker  nodes !!
# kubeadm join 10.138.0.8:6443 --token 96iv27.yb7jsavab8rwqill --discovery-token-ca-cert-hash sha256:33a196539d423d30c416d46d71127537764c58f671ca08e2326386359ba614cb

Install flannel

# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml

Patch flannel

# vi /etc/kubernetes/manifests/kube-controller-manager.yaml

Add the following to kube-controller-manager.yaml:

--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16

Then reolad kubelete

# systemctl restart kubelet

Verify status of cluster and pods:

kubectl get nodes

kubectl  get pods  --all-namespaces

 

On Worker Node

Disable swap

# swapoff -a

Edit: /etc/fstab

# vi /etc/fstab

Comment out swap

#/root/swap swap swap sw 0 0

Add the Kubernetes repo

# cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

Disable SELinux

# setenforce 0

Permanently disable SELinux:

# vi /etc/selinux/config

Change enforcing to disabled

SELINUX=disabled

Install Kubernetes 1.11.3 and docker

# yum install -y docker kubelet-1.11.3 kubeadm-1.11.3 kubectl-1.11.3 kubernetes-cni-0.6.0 --disableexcludes=kubernetes

Start and enable the Kubernetes and Docker service

# systemctl start docker && systemctl enable docker
# systemctl start kubelet && systemctl enable kubelet

Create the k8s.conf file:

# cat << EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# sysctl --system

# echo '1' > /proc/sys/net/ipv4/ip_forward



Disable firewall

# systemctl stop firewalld && systemctl disable firewalld

Join workder nodes to master node:

kubeadm join < MASTER_IP >:6443 --token < TOKEN > --discovery-token-ca-cert-hash sha256:< HASH >

Verify Nodes status from master node:

kubectl get nodes

Conclusion:

Kubernetes 1.11.3 has been installed successfully and successfully joined two worker nodes.  Now we can create pods and services.