Описание системы
Гостевая версия операционной системы.
$ lsb_release -a
No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal
Версия LXD на операционной системе хоста.
$ lxd --version
4.0.2
Установим apparmor-utils внутри гостевой операционной системы.
$ sudo apt install apparmor-utils
Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: python3-apparmor python3-libapparmor Suggested packages: vim-addon-manager The following NEW packages will be installed: apparmor-utils python3-apparmor python3-libapparmor 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 157 kB of archives. After this operation, 966 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-libapparmor amd64 2.13.3-7ubuntu5.1 [26.7 kB] Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-apparmor amd64 2.13.3-7ubuntu5.1 [78.6 kB] Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apparmor-utils amd64 2.13.3-7ubuntu5.1 [51.4 kB] Fetched 157 kB in 0s (589 kB/s) Selecting previously unselected package python3-libapparmor. (Reading database ... 18379 files and directories currently installed.) Preparing to unpack .../python3-libapparmor_2.13.3-7ubuntu5.1_amd64.deb ... Unpacking python3-libapparmor (2.13.3-7ubuntu5.1) ... Selecting previously unselected package python3-apparmor. Preparing to unpack .../python3-apparmor_2.13.3-7ubuntu5.1_amd64.deb ... Unpacking python3-apparmor (2.13.3-7ubuntu5.1) ... Selecting previously unselected package apparmor-utils. Preparing to unpack .../apparmor-utils_2.13.3-7ubuntu5.1_amd64.deb ... Unpacking apparmor-utils (2.13.3-7ubuntu5.1) ... Setting up python3-libapparmor (2.13.3-7ubuntu5.1) ... Setting up python3-apparmor (2.13.3-7ubuntu5.1) ... Setting up apparmor-utils (2.13.3-7ubuntu5.1) ...
Проблема
microk8s не запускается с ошибкой:
cannot change apparmor profile
$ microk8s
cannot change profile for the next exec call: No such file or directory
Гостевые логи будут указывать на ту же проблему.
Детали вопроса
Выполним snap microk8s, используя режим отладки, чтобы подтвердить, что он не может изменить профиль AppArmor на snap.microk8s.microk8.
$ SNAPD_DEBUG=1 SNAP_DEBUG_CONFINE=1 microk8s
2020/07/04 21:37:58.884019 cmd_linux.go:207: DEBUG: restarting into "/snap/core/current/usr/bin/snap" 2020/07/04 21:37:58.900289 cmd_run.go:398: DEBUG: SELinux not enabled DEBUG: umask reset, old umask was 02 DEBUG: security tag: snap.microk8s.microk8s DEBUG: executable: /snap/core/9665/usr/lib/snapd/snap-exec DEBUG: confinement: classic DEBUG: base snap: core DEBUG: ruid: 2018, euid: 0, suid: 0 DEBUG: rgid: 2018, egid: 2018, sgid: 2018 DEBUG: apparmor label on snap-confine is: /snap/core/9665/usr/lib/snapd/snap-confine DEBUG: apparmor mode is: enforce DEBUG: preparing classic execution environment DEBUG: creating lock directory /run/snapd/lock (if missing) DEBUG: set_effective_identity uid:0 (change: no), gid:0 (change: yes) DEBUG: opening lock directory /run/snapd/lock DEBUG: set_effective_identity uid:0 (change: no), gid:2018 (change: yes) DEBUG: opening lock file: /run/snapd/lock/microk8s.lock DEBUG: set_effective_identity uid:0 (change: no), gid:0 (change: yes) DEBUG: set_effective_identity uid:0 (change: no), gid:2018 (change: yes) DEBUG: sanity timeout initialized and set for 30 seconds DEBUG: acquiring exclusive lock (scope microk8s, uid 0) DEBUG: sanity timeout reset and disabled DEBUG: releasing lock 5 DEBUG: set_effective_identity uid:2018 (change: yes), gid:2018 (change: yes) DEBUG: creating user data directory: /home/ansible/snap/microk8s/1496 DEBUG: requesting changing of apparmor profile on next exec to snap.microk8s.microk8s cannot change profile for the next exec call: No such file or directory
AppArmor включен в гостевой операционной системе.
$ aa-enabled
Yes
В настоящее время политика AppArmor внутри гостевой операционной системы настроена.
$ aa-status
apparmor module is loaded. 17 profiles are loaded. 17 profiles are in enforce mode. /snap/core/9436/usr/lib/snapd/snap-confine /snap/core/9436/usr/lib/snapd/snap-confine//mount-namespace-capture-helper snap-update-ns.core snap-update-ns.lxd snap.core.hook.configure snap.lxd.activate snap.lxd.benchmark snap.lxd.buginfo snap.lxd.check-kernel snap.lxd.daemon snap.lxd.hook.configure snap.lxd.hook.install snap.lxd.hook.remove snap.lxd.lxc snap.lxd.lxc-to-lxd snap.lxd.lxd snap.lxd.migrate 0 profiles are in complain mode. 0 processes have profiles defined. 0 processes are in enforce mode. 0 processes are in complain mode. 0 processes are unconfined but have a profile defined.
Профили microk8s отсутствуют, поскольку гостевая операционная система использует профиль unconfined AppArmor, поэтому добавьте их вручную, чтобы убедиться, что это так.
$ apparmor_parser --add /var/lib/snapd/apparmor/profiles/snap.microk8s.*
Политика AppArmor после добавления определений.
$ aa-status
apparmor module is loaded. 53 profiles are loaded. 18 profiles are in enforce mode. /snap/core/9436/usr/lib/snapd/snap-confine /snap/core/9436/usr/lib/snapd/snap-confine//mount-namespace-capture-helper snap-update-ns.core snap-update-ns.lxd snap-update-ns.microk8s snap.core.hook.configure snap.lxd.activate snap.lxd.benchmark snap.lxd.buginfo snap.lxd.check-kernel snap.lxd.daemon snap.lxd.hook.configure snap.lxd.hook.install snap.lxd.hook.remove snap.lxd.lxc snap.lxd.lxc-to-lxd snap.lxd.lxd snap.lxd.migrate 35 profiles are in complain mode. snap.microk8s.add-node snap.microk8s.cilium snap.microk8s.config snap.microk8s.ctr snap.microk8s.daemon-apiserver snap.microk8s.daemon-apiserver-kicker snap.microk8s.daemon-cluster-agent snap.microk8s.daemon-containerd snap.microk8s.daemon-controller-manager snap.microk8s.daemon-etcd snap.microk8s.daemon-flanneld snap.microk8s.daemon-kubelet snap.microk8s.daemon-proxy snap.microk8s.daemon-scheduler snap.microk8s.disable snap.microk8s.enable snap.microk8s.helm snap.microk8s.helm3 snap.microk8s.hook.configure snap.microk8s.hook.install snap.microk8s.hook.remove snap.microk8s.inspect snap.microk8s.istioctl snap.microk8s.join snap.microk8s.juju snap.microk8s.kubectl snap.microk8s.leave snap.microk8s.linkerd snap.microk8s.microk8s snap.microk8s.refresh-certs snap.microk8s.remove-node snap.microk8s.reset snap.microk8s.start snap.microk8s.status snap.microk8s.stop 0 processes have profiles defined. 0 processes are in enforce mode. 0 processes are in complain mode. 0 processes are unconfined but have a profile defined.
microk8s теперь можно использовать.
$ microk8s
Available subcommands are: add-node cilium config ctr disable enable helm helm3 istioctl join juju kubectl leave linkerd refresh-certs remove-node reset start status stop inspect
Решение
Это самое простое решение.
Скопируйте профили AppArmor microk8s в операционную систему хоста.
Создайте временный каталог в гостевой операционной системе.
$ sudo lxc exec kube-worker-3 -- mkdir /tmp/microk8s
Скопируйте профили AppArmor microk8s.
$ sudo lxc exec kube-worker-3 -- find /var/lib/snapd/apparmor/profiles/ -name "snap.microk8s.*" -exec cp {} /tmp/microk8s/ \;
Список файлов внутри гостевой операционной системы.
$ sudo lxc exec kube-worker-3 -- ls /tmp/microk8s/
snap.microk8s.add-node snap.microk8s.daemon-apiserver snap.microk8s.daemon-controller-manager snap.microk8s.daemon-proxy snap.microk8s.helm snap.microk8s.hook.remove snap.microk8s.juju snap.microk8s.microk8s snap.microk8s.start snap.microk8s.cilium snap.microk8s.daemon-apiserver-kicker snap.microk8s.daemon-etcd snap.microk8s.daemon-scheduler snap.microk8s.helm3 snap.microk8s.inspect snap.microk8s.kubectl snap.microk8s.refresh-certs snap.microk8s.status snap.microk8s.config snap.microk8s.daemon-cluster-agent snap.microk8s.daemon-flanneld snap.microk8s.disable snap.microk8s.hook.configure snap.microk8s.istioctl snap.microk8s.leave snap.microk8s.remove-node snap.microk8s.stop snap.microk8s.ctr snap.microk8s.daemon-containerd snap.microk8s.daemon-kubelet
Скопируйте эти файлы в операционную систему хоста.
$ sudo lxc file pull -r kube-worker-3/tmp/microk8s /etc/apparmor.d/
Список файлов, скопированных в операционную систему хоста.
$ ls /etc/apparmor.d/microk8s/
snap.microk8s.add-node snap.microk8s.daemon-apiserver snap.microk8s.daemon-controller-manager snap.microk8s.daemon-proxy snap.microk8s.helm snap.microk8s.hook.remove snap.microk8s.juju snap.microk8s.microk8s snap.microk8s.start snap.microk8s.cilium snap.microk8s.daemon-apiserver-kicker snap.microk8s.daemon-etcd snap.microk8s.daemon-scheduler snap.microk8s.helm3 snap.microk8s.inspect snap.microk8s.kubectl snap.microk8s.refresh-certs snap.microk8s.status snap.microk8s.config snap.microk8s.daemon-cluster-agent snap.microk8s.daemon-flanneld snap.microk8s.disable snap.microk8s.hook.configure snap.microk8s.istioctl snap.microk8s.leave snap.microk8s.remove-node snap.microk8s.stop snap.microk8s.ctr snap.microk8s.daemon-containerd snap.microk8s.daemon-kubelet
$ sudo apparmor_parser --replace /etc/apparmor.d/microk8s/