{"id":324,"date":"2022-02-27T11:31:15","date_gmt":"2022-02-27T11:31:15","guid":{"rendered":"https:\/\/blog.hslracing.com\/myhome\/?p=324"},"modified":"2024-01-16T11:47:04","modified_gmt":"2024-01-16T11:47:04","slug":"kubernetes-cluster-install","status":"publish","type":"post","link":"https:\/\/blog.hslracing.com\/myhome\/2022\/02\/kubernetes-cluster-install.html","title":{"rendered":"Kubernetes &#8211; Cluster Install"},"content":{"rendered":"<p>On each server edit the sudoers file such that no password is required for sudo commands (required for k3sup to work)<\/p>\n<pre>%sudo ALL=(ALL:ALL) NOPASSWD: ALL<\/pre>\n<p>Install First Master server<\/p>\n<pre>k3sup install \\\r\n  --host=1of3 \\\r\n  --user=moorest \\\r\n  --k3s-version=v1.21.5+k3s1 \\\r\n  --local-path=borg.yaml \\\r\n  --context borg \\\r\n  --cluster \\\r\n  --tls-san 10.68.0.70 \\\r\n  --k3s-extra-args=\"--disable servicelb --node-taint node-role.kubernetes.io\/master=true:NoSchedule\"<\/pre>\n<p>Setup aliases and variables<\/p>\n<pre>alias k='kubectl'\r\nalias ns='kubectl config set-context --current --namespace '\r\nexport KUBECONFIG=\/Users\/moorest\/borg.yaml<\/pre>\n<p>Log into master node 1 and install RBAC, Pull kube-vip image, setup variable, daemonset and cloud provider<\/p>\n<pre>ssh 1of3\r\ncurl -s https:\/\/kube-vip.io\/manifests\/rbac.yaml &gt; \/var\/lib\/rancher\/k3s\/server\/manifests\/kube-vip-rbac.yaml\r\ncrictl pull docker.io\/plndr\/kube-vip:v0.4.1\r\nexport VIP=192.168.99.70\r\nexport INTERFACE=eno1\r\nalias kube-vip=\"ctr run --rm --net-host docker.io\/plndr\/kube-vip:v0.4.1 vip \/kube-vip\"\r\nkube-vip manifest daemonset --arp --interface $INTERFACE --address $VIP --controlplane --leaderElection --taint --services --inCluster | tee \/var\/lib\/rancher\/k3s\/server\/manifests\/kube-vip.yaml\r\ncurl -sfL https:\/\/raw.githubusercontent.com\/kube-vip\/kube-vip-cloud-provider\/main\/manifest\/kube-vip-cloud-controller.yaml &gt; \/var\/lib\/rancher\/k3s\/server\/manifests\/kube-vip-cloud-controller.yaml\r\nping 192.168.99.70<\/pre>\n<p>Edit borg.yaml and replace server with 192.168.99.70<br \/>\nJoin the other master nodes to the cluster<\/p>\n<pre>k3sup join \\\r\n  --host=2of3 \\\r\n  --server-user=moorest \\\r\n  --k3s-version=v1.21.5+k3s1 \\\r\n  --server-host=192.168.99.70 \\\r\n  --server \\\r\n  --user=moorest \\\r\n  --k3s-extra-args=\"--disable servicelb --node-taint node-role.kubernetes.io\/master=true:NoSchedule\"<\/pre>\n<p>Repeat for 3of3, then join worker nodes<\/p>\n<pre>k3sup join \\\r\n  --host=1of5 \\\r\n  --server-user=moorest \\\r\n  --server-host=192.168.99.70 \\\r\n  --k3s-version=v1.21.5+k3s1 \\\r\n  --user=moorest<\/pre>\n<p>Repeat for 2of5 through 5of5<br \/>\nCreate kube-vip-config.yaml in \/var\/lib\/rancher\/k3s\/server\/manifests<\/p>\n<pre>---\r\napiVersion: v1\r\nkind: ConfigMap\r\nmetadata:\r\nname: kubevip\r\nnamespace: kube-system\r\ndata:\r\nrange-global: 192.168.99.90-192.168.99.99<\/pre>\n<p>Install Helm on each master server<\/p>\n<pre>curl https:\/\/baltocdn.com\/helm\/signing.asc | apt-key add -\r\napt-get install apt-transport-https --yes\r\necho \"deb https:\/\/baltocdn.com\/helm\/stable\/debian\/ all main\" &gt; \/etc\/apt\/sources.list.d\/helm-stable-debian.list\r\napt-get update\r\napt-get install helm<\/pre>\n<p>Configure remaining space on Worker nodes for Longhorn<\/p>\n<pre>vgdisplay -v ubuntu-vg<\/pre>\n<p>Use the free extents number as the length value in the next command:<\/p>\n<pre>lvcreate -n longhorn-lv -l 69403 ubuntu-vg\r\nmkfs -t ext4 \/dev\/mapper\/ubuntu--vg-longhorn--lv\r\nmkdir \/var\/lib\/longhorn\r\necho \"\/dev\/mapper\/ubuntu--vg-longhorn--lv \/var\/lib\/longhorn ext4 defaults 0 1\" &gt;&gt; \/etc\/fstab\r\nmount \/var\/lib\/longhorn<\/pre>\n<p>Install NFS server<\/p>\n<pre>apt-get install nfs-kernel-server<\/pre>\n<p>Here is a screenshot of my cluster from Rancher:<br \/>\n<a href=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Borg-Cluster.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-329 size-medium\" src=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Borg-Cluster-300x130.jpg\" alt=\"\" width=\"300\" height=\"130\" srcset=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Borg-Cluster-300x130.jpg 300w, https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Borg-Cluster-768x332.jpg 768w, https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Borg-Cluster.jpg 800w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><br \/>\nIf a worker node needs to be replaced, completely or just a hard disk then all that is needed is:<\/p>\n<ul>\n<li>Drain and then delete the node from the cluster<\/li>\n<li>Replace the failed hardware<\/li>\n<li>Install a new OS<\/li>\n<li>Add the password-less sudo entry<\/li>\n<li>Configure the remaining disk space for Longhorn<\/li>\n<li>Install the NFS client<\/li>\n<li>Run the k3sup command to add the node to the cluster<\/li>\n<\/ul>\n<p>Here is a current picture of my rack:<\/p>\n<p><a href=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Rack02.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-341 size-medium\" src=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Rack02-300x225.jpg\" alt=\"\" width=\"300\" height=\"225\" srcset=\"https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Rack02-300x225.jpg 300w, https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Rack02-768x576.jpg 768w, https:\/\/blog.hslracing.com\/wp-content\/uploads\/sites\/6\/2022\/02\/Rack02.jpg 800w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>The loose cables are for when I remove the Dell R210 ii and replace it with a second CheckPoint T-180 for my pfSense HA cluster<\/p>\n<p>The extra Lenovo tiny on the far right is an m93p that is running docker containers for Minecraft and Rancher<\/p>\n<p>The various Lenovo tiny PCs and their power supplies are held in place my strips of velcro, the hook part in strips across the shelf and the loop part along the bottom edge of the PC and power supply.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>On each server edit the sudoers file such that no password is required for sudo commands (required for k3sup to work) %sudo ALL=(ALL:ALL) NOPASSWD: ALL Install First Master server k3sup install \\ &#8211;host=1of3 \\ &#8211;user=moorest \\ &#8211;k3s-version=v1.21.5+k3s1 \\ &#8211;local-path=borg.yaml \\ &#8211;context borg \\ &#8211;cluster \\ &#8211;tls-san 10.68.0.70 \\ &#8211;k3s-extra-args=&#8221;&#8211;disable servicelb &#8211;node-taint node-role.kubernetes.io\/master=true:NoSchedule&#8221; Setup aliases and&hellip;<\/p>\n","protected":false},"author":1,"featured_media":329,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[5],"class_list":["post-324","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised","tag-homelab"],"_links":{"self":[{"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/posts\/324","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/comments?post=324"}],"version-history":[{"count":13,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/posts\/324\/revisions"}],"predecessor-version":[{"id":343,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/posts\/324\/revisions\/343"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/media\/329"}],"wp:attachment":[{"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/media?parent=324"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/categories?post=324"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.hslracing.com\/myhome\/wp-json\/wp\/v2\/tags?post=324"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}