RKE2
lastmod 2026-06-11created 2024-06-138 edits
RKE2, also known as RKE Government, is Rancher's next-generation Kubernetes distribution.
Install
If NetworkManager is installed and enabled on the host, make sure it is configured to ignore CNI-managed interfaces. -> See #NetworkManager Issues
Firewalld conflicts with default networking
Firewalld conflicts with RKE2's default Canal (Calico + Flannel) networking stack. To avoid unexpected behavior, firewalld should be disabled on systems running RKE2.
NetworkManager Issues
NetworkManager manipulates the routing table for interfaces in the default network namespace where many CNIs, including RKE2's default, create veth pairs for connections to containers. This can interfere with the CNI's ability to route correctly. Therefore, if installing RKE2 on a NetworkManager-enabled system, it is highly recommended to configure NetworkManager to ignore Calico/flannel-related network interfaces.
To do this, create a configuration file called /etc/NetworkManager/conf.d/rke2-canal.conf with the following contents:
- If you have not yet installed RKE2, simply installing the configuration with
systemctl reload NetworkManageris sufficient. - If you are making this configuration change on a system where RKE2 is already installed, you must reboot the node for the change to take effect.
| INFORMATION |
| In some operating systems, such as RHEL 8.4, NetworkManager includes two extra services called |
| INFORMATION |
| Prior to NetworkManager-1.30.0-11.el8_4, the node must also be rebooted after disabling the extra services. |
Air-Gap Install
Troubleshooting
Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
When running the service restart command
it hangs for a long time and then prints the following error message:
Job for rke2-server.service failed because the control process exited with error code.
See "systemctl status rke2-server.service" and "journalctl -xe" for details.
Checking the log with the journalctl -xe command:
--
-- The unit run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-c6d4f5b0a29a907870230685397e3b39f0e4cfaa6c90b57cc18ed91937ade090-rootfs.mount has successfully entered the 'dead' state.
Jun 13 21:23:55 server rke2[973979]: time="2024-06-13T21:23:55+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:23:56 server systemd[958]: var-lib-rancher-rke2-agent-containerd-tmpmounts-containerd\x2dmount1672672385.mount: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit UNIT has successfully entered the 'dead' state.
Jun 13 21:23:56 server systemd[1]: Started libcontainer container 389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.
-- Subject: A start job for unit cri-containerd-389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.scope has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit cri-containerd-389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.scope has finished successfully.
--
-- The job identifier is 56217.
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=error msg="Kubelet exited: exit status 255"
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:00 server rke2[973979]: time="2024-06-13T21:24:00+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:00 server rke2[973979]: time="2024-06-13T21:24:00+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:05 server rke2[973979]: time="2024-06-13T21:24:05+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:10 server rke2[973979]: time="2024-06-13T21:24:10+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:15 server rke2[973979]: time="2024-06-13T21:24:15+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:15 server rke2[973979]: time="2024-06-13T21:24:15+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:20 server rke2[973979]: time="2024-06-13T21:24:20+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:25 server rke2[973979]: time="2024-06-13T21:24:25+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=error msg="Kubelet exited: exit status 255"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:30 server rke2[973979]: time="2024-06-13T21:24:30+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:30 server rke2[973979]: time="2024-06-13T21:24:30+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:35 server rke2[973979]: time="2024-06-13T21:24:35+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:40 server rke2[973979]: time="2024-06-13T21:24:40+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:40 server sudo[977843]: your : TTY=pts/0 ; PWD=/home/your ; USER=root ; COMMAND=/usr/bin/systemctl daemon-reload
Jun 13 21:24:40 server sudo[977843]: pam_unix(sudo:session): session opened for user root by your(uid=0)
Jun 13 21:24:40 server systemd[1]: Reloading.
Jun 13 21:24:41 server sudo[977843]: pam_unix(sudo:session): session closed for user root
Jun 13 21:24:43 server sudo[977903]: your : TTY=pts/0 ; PWD=/home/your ; USER=root ; COMMAND=/usr/bin/systemctl start rke2-server.service
Jun 13 21:24:43 server sudo[977903]: pam_unix(sudo:session): session opened for user root by your(uid=0)
Jun 13 21:24:45 server rke2[973979]: time="2024-06-13T21:24:45+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:45 server rke2[973979]: time="2024-06-13T21:24:45+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:50 server rke2[973979]: time="2024-06-13T21:24:50+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
This seems to be an issue related to port 9345. Check firewalls such as iptables.