Kubernetes:Basic

lastmod 2026-08-26created 2022-10-0178 edits

Kubernetes basics.

Getting started

• Kubernetes:Basic:Wordpress - Starting WordPress with k3s installed.

Basic Kubectl commands

The API server supports http communication using json or protobuf format. Using it directly is inconvenient, so we usually use a command-line tool called kubectl. You'll be using it a tediously huge amount from now on. It's confusing how to pronounce it; officially it is read as cube control, but cube-C-T-L, koob control, koob-C-T-L, etc. are also widely used.

• apply - Applies the desired state. Usually used with a file via the -f option.
• get - Shows a list of resources.
• describe - Shows the state of a resource in detail.
• delete - Removes a resource.
• logs - Views a container's logs.
  • kubectl logs -f xxxx --since=5m - Check logs from the last 5 minutes
  • kubectl logs -f deployment/mobile-php --all-containers=true --since=5m - Check logs of all pods (used when, with replicas set, identical containers are up and you don't know which pod to check)
• exec - Sends a command to a container. Mainly used to access a container.
• config - Manages kubectl settings.

Roadmap for beginners

1. Why use Kubernetes? -> Kubernetes#Why do we need Kubernetes?
2. Not sure? Then have you used Swarm? -> It mainly solves problems caused by network connections in Docker.
3. Not sure? Then have you used Docker?
4. Not sure? Have you deployed node?
5. Not sure? Then you're not the target audience yet.

What was hard when deploying?

• In an environment like APM (Apache+PHP+MySQL), connecting the services to each other is essential.
• If you go as far as MSA, things like Redis and grpc also need consideration.

Have you ever made a single package for deployment?

• Windows Install Shield
• dpkg
• AppImage

People usually use it for interoperability

1. Usually because you have to connect to the DB on localhost using a PN (Private Network)
2. So installing the DB is a hassle, running the server (Apache) is a hassle, etc. ...
3. So people somehow manage up to Docker Compose

But have you ever operated a service?

1. How do you respond when an outage occurs?
2. (Usually) you'd look at the logs and respond ... but what will you do if the service is down?
3. Frantic reboot -> lucky if it works... if not, roll back with tears in your eyes
   • Lucky if you at least have checkpoints for rollback (version control) ...
4. If you weren't prepared, recover quickly ... but meanwhile the service is down, and the all-nighter shatters your mind

Now, even if you resolve to do version control well~ ... how do you respond immediately when the service goes down?

1. Route across multiple nodes
2. Round-Robin DNS -> but this ultimately only distributes traffic; it's not failure recovery.
3. Use something like HAProxy (High Availability Proxy)

But at first people usually run the service on a single computer ...

1. What if one process (httpd-1) suddenly uses a lot of resources? -> another process (httpd-2) suddenly slows down
2. Or query speed slows down as data piles up ...
3. Then somewhere things suddenly slow down, Timeouts appear and errors get reported, or complaints come in that it's slow
4. You can limit memory with a VM, or limit resources with cgroup, but

From this point you add PCs. -> And then? .... hellfire begins

• IP addresses connected via localhost get tangled
• Filesystem isn't shared -> use NFS or S3 -> it gets slow
• What was the DB password we used before? -> somehow find it and paste it -> maybe try something like Vault?

Users are increasing ... -> add more PCs -> more hellfire begins

1. IP address management hell -> let's hook up DNS ...

So what matters technically?

• Network technology in general
• File systems like Overlayfs
• Resource limiting like cgroup
• CS (Computer Sciences) topics in general (Virtual Memory, etc ...)

Kubernetes architecture

Containers work very simply and elegantly. Run starts them and stop stops them.

If you know the server-client architecture, you can easily picture building an agent that manages containers and managing it remotely from a central point through an API.

Kubernetes too has a simple structure: an API server and state store at the center (Master), communicating with the agent (kubelet) on each server (Node).

However, because it implements the concepts above split into many modules and uses various open-source components, installation is tricky and the configuration looks complex at first glance.

Master - Node structure

Kubernetes consists of a master that manages the whole cluster and nodes where containers are deployed.

Every command calls the master's API server, and nodes perform the necessary work while communicating with the master.

Even when issuing a command to a container on a specific node or viewing its logs, you don't command the node directly; you command the master, and the master connects to the node and responds with the result on your behalf.

Desired State

Moves from the Current State to the Desired State.

The most important thing in Kubernetes is the concept of the desired state.

The desired state

means the environment the administrator wants,

and more specifically, things like how many web servers should be running and which port you want to serve on.

Kubernetes does complex and varied work, but if you look closely, it has a simple(?) logic: it monitors the "current state" and internally does various things to maintain the desired state set by the administrator.

Because of this concept, when an administrator deploys servers, they declare a state instead of commanding direct actions.

For example, "Run an nginx container. And open it on port 80." is a command (imperative) to change the current state into the desired state, while "Keep one nginx container running with port 80 open" declares the desired state (declarative). Now it makes even less sense

At first glance it may seem like wordplay that slightly rewords the same request, but this difference shows up in CLI commands too.

$ docker run # 명령
$ kubectl create # 상태 생성 (물론 kubectl run 명령어도 있지만 잘 사용하지 않습니다)

The core of Kubernetes is state, and to use Kubernetes you need to know what states exist and how to declare them.

• Replication Controller
• Endpoint Controller
• Namespace Controller
• Custom Controller
  • ML Controller
  • CI/CI Controller

Master

A characteristic pain of the master server is that its various modules are split by function with scalability in mind.

It must be secured so that only administrators can access it, and since the cluster can't be managed if the master server dies, usually three are set up for higher reliability.

In the case of AWS EKS, AWS manages the master itself for higher reliability (the master is not accessible), and in development or small-scale environments the master and nodes are sometimes set up on the same server without separating them.

API Server (kube-apiserver)

REST API, the only module that communicates with etcd, permission checks, etc. ...

The API server is the core module of the master that handles aaall requests. It handles not only kubectl requests but also requests from internal modules, and can check permissions and reject requests. What it actually does is the very simple job of storing the desired state in a key-value store and reading the stored state. Assigning Pods to nodes and checking their state is split off into other modules. It also acts as a debugger, showing logs of containers running on nodes and sending commands.

When the API server receives a request, it only communicates with the etcd store; it's the scheduler and controllers that actually change the state.

They monitor the current state, and if it differs from the desired state, each performs its assigned work and updates the state.

Distributed data store etcd

A key-value store using the RAFT algorithm. Because it can be distributed and replicated across several instances, it is highly reliable and fairly fast. Besides simply storing and reading values, it has a watch feature, so when some state changes it can check immediately and run logic.

All of the cluster's configuration and state data is stored here and the other modules run stateless, so as long as you back up etcd well, you can restore the cluster at any time. etcd communicates only with the API server, and other modules access etcd data through the API server.

Ultra-lightweight Kubernetes distributions such as k3s sometimes use sqlite instead of etcd.

Scheduler (kube-scheduler)

Watches newly created pods, checks the node's current state and the pod's requirements (e.g., assign labels to nodes, deploy to nodes with a specific label - gpu-zone -)

The scheduler is the module that assigns unassigned Pods to suitable node servers based on various conditions (required resources, labels).

Kube controller (kube-controller-manager)

Various controllers exist; run as a single process to reduce complexity.

The kube controller is a very busy module that plays many roles. It manages the state of almost every object in Kubernetes.

Work is strictly divided by object: a Deployment creates a ReplicaSet, a ReplicaSet creates Pods, and Pods are managed by the scheduler.

• Replication controller
• Node controller
• Endpoint controller

Cloud controller (cloud-controller-manager)

The cloud controller is a module specialized for clouds such as AWS, GCE, and Azure.

It can add/remove nodes, attach load balancers, and attach volumes.

Each cloud vendor just needs to implement it according to the interface, so it is highly extensible, and many vendors provide their own modules.

Node

A node server communicates with the master server, creates the necessary Pods, and configures networks and volumes. It is where the actual containers are created, and can scale to hundreds or thousands of machines. You can label each server to define its purpose (GPU-specialized, SSD server, etc.).

A Pod always runs on a Node. A Node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster. Each Node is managed by the control plane. A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. The control plane's automatic scheduling takes into account the available resources on each Node.

Every Kubernetes Node runs at least:

• Kubelet, a process responsible for communication between the Kubernetes control plane and the Node; it manages the Pods and the containers running on a machine.
• A container runtime (like Docker) responsible for pulling the container image from a registry, unpacking the container, and running the application.

The most common operations can be done with the following kubectl commands:

• kubectl get - list resources
• kubectl describe - show detailed information about a resource.
• kubectl logs - print the logs from a container in a pod
• kubectl exec - execute a command on a container in a pod.

You can use these commands to see when applications were deployed, what their current statuses are, what their configurations are, and so on.

Node components

Kubelet (kubelet)

Manages the lifecycle of Pods assigned to the node. It creates Pods, checks that the containers in a Pod are healthy, and periodically reports their state to the master. It also receives requests from the API server to deliver container logs or execute specific commands on its behalf.

Proxy (kube-proxy)

If the kubelet manages Pods, the proxy manages the network connected to Pods.

It forwards TCP, UDP, and SCTP streams and can bundle multiple Pods in round-robin fashion to provide a service.

Initially kube-proxy itself acted as a proxy server, receiving actual requests and passing them to each Pod, but over time it changed to configuring iptables.

Because of the problem of slowing down as the number of rules registered in iptables grows, it recently began supporting IPVS.

Container runtime interface (Container runtime interface; CRI)

It's reasonable to think containers are Docker and Docker is containers, but Kubernetes supports various container runtimes that implement the CRI (Container runtime interface).

These include containerd (practically Docker..), rkt, and CRI-O.

CNI (network)

CSI (storage)

How a single Pod is created

Let's put the pieces discussed above together one by one and look at the overall flow. When an administrator creates a ReplicaSet to deploy an application, a Pod is created through the following process.

Looking at the flow, you can see that the modules don't communicate with each other; they communicate only with the API Server. Through the API Server they check the state stored in etcd, and if the current state differs from the desired state, they perform the necessary work. Here is what each module does.

• kubectl
  • Define the ReplicaSet spec in a yml file and send the command to the API Server using the kubectl tool
  • The API Server stores the new ReplicaSet Object in etcd
• Kube Controller
  • The ReplicaSet Controller included in the Kube Controller watches ReplicaSets and checks whether Pods matching the Label Selector condition defined in the ReplicaSet exist
  • If there is no Pod with the matching Label, it creates a new Pod (no assign) from the ReplicaSet's Pod template. The creation is likewise sent to the API Server, which stores it in etcd
• Scheduler
  • The Scheduler checks whether there are unassigned (no assign) Pods
  • If there are unassigned Pods, it finds a Node that meets the conditions and assigns the Pod to it
• Kubelet
  • The Kubelet checks whether there are Pods assigned to its Node that have not yet been created
  • If there are Pods not yet created, it creates them according to the spec
  • Periodically reports the Pod's state to the API Server

Does the complicated diagram make sense now? The example above dealt with a ReplicaSet, but a DaemonSet, which deploys Pods to every node, works the same way. The DaemonSet controller and Scheduler assign Pods across all nodes, and the kubelet creates the Pods assigned to its own node.

Seeing each module check the state it is responsible for and operate independently, I think it's a really well-built microservice architecture.

kubelet

• Runs on every node.
• Runs/stops pods and checks their state
• Container Runtime Interface (CRI)
  • docker
  • containerd
  • cri-o

kube-proxy

• Network proxy and load balancing
• It used to use a proxy program, but for performance reasons it uses system-level iptables or ipvs. (It only manages the configuration.)

DNS

Dashboard

RBAC (role-based access control)

An access permission system. You can easily assign CRUD-like permissions per user for each resource. It can apply to the whole cluster or to a specific namespace. On AWS, it can also be used integrated with IAM.

CRD (Custom Resource Definitaion)

Lets you apply and use features that Kubernetes doesn't provide in the same way as the built-in features.

For example, Kubernetes doesn't provide SSL certificate management by default, but if you install cert-manager and use the Certificate resource, you can manage certificates with familiar Kubernetes commands.

You can easily extend various features without having to learn yet another tool or method.

Auto Scaling

Scaling based on CPU and memory usage is a given, and you can also use values such as the current number of connected users.

• Horizontal Pod Autoscaler (HPA), which adjusts the number of containers,
• Vertical Pod Autoscaler (VPA), which adjusts containers' resource allocation,
• Cluster Autosclaer (CA), which adjusts the number of servers

are the available approaches.

Federation, Multi Cluster

You can combine a Kubernetes cluster installed in the cloud with Kubernetes installed on your own servers and use them as one.

With Anthos, announced by Google, you can manage multiple clusters across multiple clouds from one place.

Cluster (Kubernetes cluster)

Kubernetes coordinates a highly available cluster of computers that are connected to work as a single unit.

The abstractions in Kubernetes allow you to deploy containerized applications to a cluster without tying them specifically to individual machines. To make use of this new model of deployment, applications need to be packaged in a way that decouples them from individual hosts: they need to be containerized. Containerized applications are more flexible and available than in past deployment models, where applications were installed directly onto specific machines as packages deeply integrated into the host.

Kubernetes automates the distribution and scheduling of application containers across a cluster in a more efficient way.

Kubernetes is an open-source platform and is production-ready.

A Kubernetes cluster consists of two types of resources:

• The Control Plane coordinates the cluster.
• Nodes are the workers that run applications.

Kubernetes Object

• Kubernetes/Deployment, ReplicaSet and Pod - Ever changing code

Pod

• The smallest unit of deployment.
• Several containers can exist inside it.

A Pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker), and some shared resources for those containers. Those resources include:

• Shared storage, as Volumes
• Networking, as a unique cluster IP address
• Information about how to run each container, such as the container image version or specific ports to use

A Pod models an application-specific "logical host" and can contain different application containers which are relatively tightly coupled. For example, a Pod might include both the container with your Node.js app as well as a different container that feeds the data to be published by the Node.js webserver. The containers in a Pod share an IP Address and port space, are always co-located and co-scheduled, and run in a shared context on the same Node.

Pods are the atomic unit on the Kubernetes platform. When we create a Deployment on Kubernetes, that Deployment creates Pods with containers inside them. Each Pod is tied to the Node where it is scheduled, and remains there until termination (according to restart policy) or deletion. In case of a Node failure, identical Pods are scheduled on other available Nodes in the cluster.

ReplicaSet

• Used to create replicas

Deployment

• (Kubernetes) Creating a Deployment

Version updates, and rollback

A Deployment instructs Kubernetes how to create and update instances of your application.

Once you've created a Deployment, the Kubernetes Master schedules the application instances included in that Deployment to run on individual Nodes in the cluster.

Service

• 3. 10 steps for practical use of K8s

ClusterIP

A proxy used inside the cluster. It is the default if no type is specified, and pods inside the cluster can access it by the service's name.

NodePort

In addition to the access scope of ClusterIP, it can also be accessed from outside the K8s cluster via a node's IP address and port number.

It is created on the same port on every node.

If you specify NodePort as the service type, in addition to the ClusterIP functionality described earlier, a public port is opened on the node's IP address. This allows requests to be sent from outside the K8s cluster to the pods inside.

The public port number range is 30000 - 32767 by default.

Requests a client sends to a node's IP and port are ultimately delivered to a pod.

LoadBalancer

In addition to the access scope of NodePort, it can also be accessed from outside the K8s cluster via a representative IP address.

Integrates with a load balancer to expose a pod's application externally.

ExternalName

Resolves names for pods to connect to endpoints outside the K8s cluster.

For example, it can be used to access a public cloud database or an AI API service.

ExternalName sets a mapping between the service name and an external DNS name in the internal DNS. This lets pods access endpoints on the external network by the service's name. The port number cannot be specified, however.

This service type is convenient when pods connect to external endpoints (DB servers), because the IP address can be obtained from the service name in the namespace. It is also easy to replace with a service inside the K8s cluster.

Ingress

• (Translation) Understanding Kubernetes networking #3: Ingress | Coffee Whale's Notes

Volume

• Storage (e.g. EBS, NFS, ...)

Namespace

• Namespaces | Kubernetes
• Share a Cluster with Namespaces | Kubernetes

In Kubernetes, namespaces provide a mechanism for isolating groups of resources within a single cluster. Names of resources need to be unique within a namespace, but not across namespaces. Namespace-based scoping is applicable only for namespaced objects (e.g. Deployments, Services, etc.) and not for cluster-wide objects (e.g. StorageClass, Nodes, PersistentVolumes, etc.).

You can logically divide a single cluster. Since various frameworks and applications are installed in one cluster, it is common to use several namespaces besides the defaults (system, default). For finer-grained configuration, you can actively use labels to manage resources flexibly and scalably.

List the current namespaces in a cluster using the kubectl get namespaces command.

NAME          STATUS    AGE
default       Active    11d
kube-system   Active    11d
kube-public   Active    11d

Kubernetes starts with three initial namespaces.

• default - The default namespace for objects with no other namespace
• kube-system - The namespace for objects created by the Kubernetes system
• kube-public - This namespace is created automatically and is readable by all users (including those not authenticated). This namespace is mostly reserved for cluster usage, in case some resources should be visible and readable publicly throughout the whole cluster. The public aspect of this namespace is only a convention, not a requirement.

Creating a new namespace

• Create a new YAML file called my-namespace.yaml with the contents below.

apiVersion: v1
kind: Namespace
metadata:
  name: <insert-namespace-name-here>

Then run:

kubectl create -f ./my-namespace.yaml

Alternatively, you can create a namespace using the command below.

kubectl create namespace <insert-namespace-name-here>

ConfigMap/Secret

• Configuration

ServiceAccount

• Permission account

Role/ClusterRole

• Permission settings (get, list, watch, create, ...)

Typical configuration

• 🌌 (Kubernetes architecture) 2. Objects

Exposing the app externally

• Using a Service to Expose Your App | Kubernetes

Scaling the app

• Running Multiple Instances of Your App | Kubernetes

Updating the app

• Performing a Rolling Update | Kubernetes

Installing k3s

Using kubectl

Helm

Building a web server with k8s

• Building a web server with k8s - Part 1. install k3s with Calico - Oznote.io_-_k8s_web_server_1.pdf
• Building a web server with k8s - Part 2. setup nginx-ingress, ipvs, metalb - Oznote.io_-_k8s_web_server_2.pdf
• Building a web server with k8s - Part 3. deploy nextjs (with. bun.js) - Oznote.io_-_k8s_web_server_3.pdf
• Building a web server with k8s - Part 4. Applying https to nginx ingress (using cert-manager) - Oznote.io_-_k8s_web_server_4.pdf
• Building a web server with k8s - Part 5. Deploying and integrating MariaDB - Oznote.io_-_k8s_web_server_5.pdf
• Building a web server with k8s - Part 5-1. (Extra) Deploying and integrating PostgreSQL - Oznote.io_-_k8s_web_server_5-1.pdf
• Building a web server with k8s - Part 6. Setting up MiniO - Oznote.io_-_k8s_web_server_6.pdf

How to deploy a Node.js application and run it with Kubernetes

• Deploying NodeJs Application Using Kubernetes | by Somansh Kumar | Akeo | Medium

Write the Dockerfile:

FROM somansh1206/nodejs
WORKDIR /usr/src/app
COPY . .
RUN \
  npm install && \
  npm install gulp -g && \
  npm install swagger -g && \
  gulp clean && \
  gulp tsc
CMD [ "node", "app.js" ]

Write the Kubernetes yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: sample-nodejs
 labels:
   app: sample-nodejs
spec:
 selector:
  matchLabels:
   app: sample-nodejs
 template:
  metadata:
   labels:
    app: sample-nodejs
  spec:
   containers:
   - image: ip_address_of_the_machine:5000/nodejs-api-image:latest
     name: nodejs-api
     imagePullPolicy: Always
     ports:
       - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
    name: sample-nodejs-svc
spec:
    ports:
    - name: “sample-nodejs”
      targetPort: 3000
      port: 3000
      nodePort: 30253
      protocol: TCP
    selector:
       app: sample-nodejs
    type: NodePort

Building single-node Kubernetes with k3s and opening ports and deploying with Traefik

• Building single-node Kubernetes with k3s

See also

• k8s
• k3s

Favorite site

• (Kubernetes guide for beginners) What is container orchestration? - YouTube
• How kubeadm Initializes Your Kubernetes Master - Ian Lewis
• [Recommended] Getting started with Kubernetes - What is Kubernetes?
  • [Recommended] Getting started | Kubernetes guide
• (Translation) Understanding Kubernetes networking #3: Ingress | Coffee Whale's Notes