K8s 學習筆記 - 在 MacBook Pro M1 安裝 K3s


在 Macbook Pro M1 (Apple Silicon) 安裝 K3s 的筆記。


摘要

K3d vs K3s

K3d 其實是 K3s in Docker 的縮寫,也就是透過 docker 把 K8s 的角色跑起來,包含 api, kube-api, kube-proxy, coredns … 等。兩者的使用對象有所差異,整理如下:

  • 開發者:適合用 K3d 快速建立 cluster,測試自己的應用程式。
    • 如果要在 Apple Silicon 的環境使用 k8s,也可以用 k3d 就可以。
  • K8s admin:適合用 K3s 設計與規劃 K8s cluster,模擬各種 cluster 的需求。

K3s 的架構與原理如下圖:

Source: https://docs.k3s.io/architecture

類似於 dapr 的 hosted mode 與 cluster mode.

Multipasss

multipass 是 ubuntu 的公司出的,針對 ubuntu 在三大作業系統上可以順利運行 ubuntu vm 的工具,本身提供的 ubuntu image 也是輕量化過的。

安裝 multipass

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
❯ brew install --cask multipass
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
aptos cargo-about ecoji grayskull kwctl podsync twm xorg-server
aws-sam-cli cargo-deny fgbio jbang mpfrcx quartz-wm video-compare xrdb
bossa cascadia flagd jreleaser ocm ruby@3.1 xinit
bzip3 cdebug gokey kubent openvino simdutf xmodmap

You have 3 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.

==> Tapping homebrew/cask
Cloning into '/opt/homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 702522, done.
remote: Counting objects: 100% (57/57), done.
remote: Compressing objects: 100% (41/41), done.
Receiving objects: 39% (273984/702522), 98.74 MiB | 3.09 MiB/s


==> Installing Cask multipass
==> Running installer for multipass; your password may be necessary.
Package installers may write to any location; options such as `--appdir` are ignored.
Password:
installer: Package name is multipass
installer: Installing at base path /
installer: The install was successful.
🍺 multipass was successfully installed!

基本的使用參數:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
❯ multipass version
multipass 1.10.1+mac
multipassd 1.10.1+mac

## Help
❯ multipass help
Usage: multipass [options] <command>
Create, control and connect to Ubuntu instances.

This is a command line utility for multipass, a
service that manages Ubuntu instances.

## 查看 launch 的 help
❯ multipass help launch
Usage: multipass launch [options] [[<remote:>]<image> | <url>]
Create and start a new instance.

## 提供哪一些 Images
❯ multipass find
Image Aliases Version Description
snapcraft:core18 18.04 20201111 Snapcraft builder for Core 18
snapcraft:core20 20.04 20210921 Snapcraft builder for Core 20
snapcraft:core22 22.04 20220426 Snapcraft builder for Core 22
18.04 bionic 20230112 Ubuntu 18.04 LTS
20.04 focal 20230111 Ubuntu 20.04 LTS
22.04 jammy,lts 20230107 Ubuntu 22.04 LTS
anbox-cloud-appliance latest Anbox Cloud Appliance
charm-dev latest A development and testing environment for charmers
docker latest A Docker environment with Portainer and related tools
jellyfin latest Jellyfin is a Free Software Media System that puts you in control of managing and streaming your media.
minikube latest minikube is local Kubernetes

啟動虛擬機

透過 multipass 啟動一個虛擬機,指定虛擬機名稱、記憶體、Disk、CPU 數量、網路模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

❯ multipass launch --name k3s-main \
--mem 4G --disk 20G \
--cpus 2

Launched: k3s-main

❯ multipass info k3s-main
Name: k3s-main
State: Running
IPv4: 192.168.64.3
Release: Ubuntu 22.04.1 LTS
Image hash: 3100a27357a0 (Ubuntu 22.04 LTS)
Load: 0.88 0.37 0.14
Disk usage: 2.4G out of 19.2G
Memory usage: 749.0M out of 3.8G
Mounts: --

mkdir -p ~/Temp/K8s

## 掛載磁碟: Host Machine to Virtual Machine
❯ multipass mount ~/Temp/K8s k3s:~/k8s
Enabling support for mounting \


## 刪除虛擬機
❯ multipass delete k3s-main
❯ multipass purge

進去虛擬機

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
❯ multipass shell k3s-main

## 查看 disk 狀況
ubuntu@k3s-main:~$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 393M 864K 392M 1% /run
/dev/vda1 20G 1.9G 18G 10% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/vda15 105M 5.3M 100M 5% /boot/efi
tmpfs 393M 4.0K 393M 1% /run/user/1000

## 查看 CPU 狀況, 需要安裝 apt install cpuinfo
ubuntu@k3s-main:~$ cpu-info
Packages:
0: Intel Core i9-9900K
1: Intel Core i9-9900K
Microarchitectures:
2x Sky Lake
Cores:
0: 1 processor (0), Intel Sky Lake
1: 1 processor (1), Intel Sky Lake
Logical processors (System ID):
0 (0): APIC ID 0x00000000
1 (1): APIC ID 0x00000001

整理幾個 multipass 運作的資訊:

  • multipass 程序主要工作在 /Library/Application Support/com.canonical.multipass
  • multipass 預設使用 qemu 為 Hypervisor,虛擬磁碟為 COW2 格式,網路為 Bridge 模式
  • 虛擬機的預設路徑為:/var/root/Library/Application Support/multipassd/ (需要 root 權限才能存取)
  • multipass 預設透過 luanchd 跟著作業系統啟動
    • 配置檔位置: /Library/LaunchDaemons/com.canonical.multipassd.plist

安裝 K3s

直接安裝

如果在 macOS 直接安裝 k3s 會出現以下訊息:

1
2
❯ curl -sfL https://get.k3s.io | sh -
[ERROR] Can not find systemd or openrc to use as a process supervisor for k3s

不管是 Intel or Apple Silicon 都是。

在 VM 裡面安裝 K3s

透過 multipass 裝起好一台 VM 之後,進去裡面再安裝 K3s 就可以了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
## 在虛擬機裡面
root@k3s:~# curl -sfL https://get.k3s.io | sh -
[INFO] Finding release for channel stable
[INFO] Using v1.25.5+k3s2 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.25.5+k3s2/sha256sum-arm64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.25.5+k3s2/k3s-arm64
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3se

## 確認 K8s 安裝的狀況
ubuntu@k3s-main:~$ sudo kubectl get po,no --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/local-path-provisioner-79f67d76f8-w946b 1/1 Running 0 8m15s
kube-system pod/coredns-597584b69b-9mx2w 1/1 Running 0 8m15s
kube-system pod/metrics-server-5f9f776df5-p826z 1/1 Running 0 8m15s
kube-system pod/helm-install-traefik-crd-bq27h 0/1 Completed 0 8m15s
kube-system pod/helm-install-traefik-b6vb4 0/1 Completed 1 8m15s
kube-system pod/svclb-traefik-09095d96-v9fpk 2/2 Running 0 7m45s
kube-system pod/traefik-66c46d954f-r48tq 1/1 Running 0 7m45s

NAMESPACE NAME STATUS ROLES AGE VERSION
node/k3s-main Ready control-plane,master 8m31s v1.25.5+k3s2

ubuntu@k3s-main:~$ sudo kubectl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
... 略 ...

ubuntu@k3s-main:~$ sudo kubectl get rs --all-namespaces
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system local-path-provisioner-79f67d76f8 1 1 1 10m
kube-system coredns-597584b69b 1 1 1 10m
kube-system metrics-server-5f9f776df5 1 1 1 10m
kube-system traefik-66c46d954f 1 1 1 10m

## 在 Host Machine,取得 K8s 的 Node Token
❯ multipass exec k3s-main sudo cat /var/lib/rancher/k3s/server/node-token
K1082f1ce868a513c55999a7eba68ab888dc2035baf2323131466364f65bf5f0220::server:27b105f74351819669c0481da7c87baa

## 在 Master Node 裏取得 KubeConfig
## 如果要在 Master Node 之外使用,複製檔案內容後,
## 記得把 server: https://127.0.0.1:6443 改成 Master Node 的 IP
cat /etc/rancher/k3s/k3s.yaml

新增 Worker Node

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
multipass launch --name k3s-worker1 --mem 2G --disk 20G
multipass launch --name k3s-worker2 --mem 2G --disk 20G

multipass shell k3s-worker1
multipass shell k3s-worker2

## 進去 VM 裡
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.64.3:6443 K3S_TOKEN="K102a46f234cf614bd49a50092389d8e03fed00618846936782fbdae904c32ee9d5::server:5fc3190151a2de0126199cc6fbd94acf" sh -

## 在 Master Node
ubuntu@k3s-main:~$ sudo kubectl get no
NAME STATUS ROLES AGE VERSION
k3s-main Ready control-plane,master 65m v1.25.5+k3s2
k3s-worker1 Ready <none> 75s v1.25.5+k3s2
k3s-worker2 Ready <none> 2s v1.25.5+k3s2

移除 Worker Node

1
2
3
4
5
kubectl drain k3s-worker --delete-emptydir-data --force --ignore-daemonsets

root@k3s:~# kubectl delete node k3s-worker
node "k3s-worker" deleted


測試

底下都在 k3s-master node 裡面執行。

建立測試網頁: index.html,內容如下:

1
2
3
4
5
6
<html>
<head>
<title>Hello World!</title>
</head>
<body>Hello World!</body>
</html>

把這個 HTML 設定成 configmap:

1
~$ kubectl create configmap hello-world --from-file index.html

建立 K8s YAML:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: hello-world
port:
number: 80

---
apiVersion: v1
kind: Service
metadata:
name: hello-world
spec:
ports:
- port: 80
protocol: TCP
selector:
app: hello-world

---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-nginx
spec:
selector:
matchLabels:
app: hello-world
replicas: 3
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: hello-world-volume
mountPath: /usr/share/nginx/html
volumes:
- name: hello-world-volume
configMap:
name: hello-world

執行這段 YAML

1
2
3
4
5
6
~$ kubectl apply -f hello-world.yml
~$ kubectl get po
NAME READY STATUS RESTARTS AGE
hello-world-nginx-5876d86d96-k2mm6 1/1 Running 0 4m25s
hello-world-nginx-5876d86d96-krsrh 1/1 Running 0 4m25s
hello-world-nginx-5876d86d96-wq9hf 1/1 Running 0 4m25s

測試:

1
2
3
4
5
6
7
~$ curl localhost:80
<html>
<head>
<title>Hello World!</title>
</head>
<body>Hello World!</body>
</html>

Q & A

Q: 可以把 master 和 worker node 裝在同一台 VM?

不可以,Port 會出現衝突,要裝在同一台 VM,就用 K3d

如果已經執行了,記得跑 k3s-agent-uninstall.sh 或者 k3s-uninstall.sh 刪掉造成衝突的一方,然後重新啟動 service 即可:

1
2
systemctl start k3s.service
systemctl status k3s.service

Comments

  • 全站索引
  • 關於這裏
  • 關於作者
  • 學習法則
  • 思考本質
  • 一些領悟
  • 分類哲學
  • ▲ TOP ▲