Talos Linux: Immutable & Secure K8s Cluster

๐Ÿ’ก Talos Linux: Immutable & Secure K8s Cluster โ€” Executive Summary

Technical architectural analysis, practical configuration manifests, and deployment instructions for Talos Linux: Immutable & Secure K8s Cluster.

Binary Installation

mac
bash
brew install siderolabs/tap/talosctl

echo 'source <(talosctl completion zsh)' | tee -a ~/.zshrc

#if using colima
sudo ln -s "$HOME/.colima/default/docker.sock" /var/run/docker.sock
#if using docker-desktop
sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

Testing Cluster Management

Cluster Init
bash
# docker ๊ธฐ๋ฐ˜ cluster ์ƒ์„ฑ
talosctl cluster create
# ํด๋Ÿฌ์Šคํ„ฐ ๋ฐ ๋…ธ๋“œ ์ •๋ณด
talosctl cluster show

# ํด๋Ÿฌ์Šคํ„ฐ ์ธ์ฆ์ƒ์„ฑ: admin@talos-default context ์ƒ์„ฑ ํ›„ ๋‚ด kubeconfig์— ์ถ”๊ฐ€
talosctl kubeconfig --nodes $ep_or_nodename

# kubeconfig ์ˆ˜์ • (proxy Point๋กœ ๋ณ€๊ฒฝ)
yq -i "(.clusters[] | select(.name==\"talos-default\") | .cluster.server) 
    = \"$(talosctl cluster show | grep 'KUBERNETES ENDPOINT' | awk '{print $3}')\"
" ~/.kube/config
Cluster Destroy
bash
export default_context=rancher-desktop
talosctl cluster destroy

# kubeconfig cleanning
yq -i 'del(.clusters[] | select(.name=="talos-default"))' ~/.kube/config
yq -i 'del(.contexts[] | select(.name=="admin@talos-default"))' ~/.kube/config
yq -i 'del(.users[] | select(.name=="admin@talos-default"))' ~/.kube/config
yq -i ".[\"current-context\"]=\"${default_context}\"" ~/.kube/config

Production Cluster Management

Step 1: Prepare your infrastructure

To create the production cluster infrastructure:

  1. Boot the machine using the Talos ISO image
  2. Ensures network access on the node.

Here’s how to perform each step.

Boot your computer using the Talos ISO image

To boot your computer using the Talos ISO image, follow these steps:

  1. Talos Image Factory

    Download the latest ISO for your infrastructure based on your hardware type from .

Note: For network booting and self-build media using public kernels, there are some required kernel parameters. For more information, see [Kernel Documentation](https://docs.siderolabs.com/talos/v1.11/reference/kernel) โ€˜Getting Startedโ€™.

See โ€˜Getting Startedโ€™.

  1. Boot the three control planes using the ISO images you just downloaded.
  2. Boot additional machines as worker nodes.
Ensuring network access

If your nodes are behind a firewall, on a private network, or cannot be reached directly for other reasons, you must configure your load balancer to forward TCP port 50000 to reach them for Talos API access.

Note: The Talos Linux API uses gRPC and mutual TLS, so it cannot be proxied through an HTTP/S proxy, only through a TCP load balancer.

Once the control plane and worker nodes are booted, next configure the Kubernetes endpoints.

Step 2: Store the IP address in a variable

To save a variable for your computer’s IP address:

  1. Copy the IP addresses displayed on each machine console, including the control plane and any worker nodes you created.

    If the display is not connected, retrieve the IP address from the DHCP server.

  2. Create a Bash array for the control plane node IP addresses, replacing each placeholder with the IP address of the control plane node. You can include as many IP addresses as you need.

bash
CONTROL_PLANE_IP=("<control-plane-ip-1>" "<control-plane-ip-2>" "<control-plane-ip-3>")

For example: If the IP addresses of the control plane nodes are 192.168.0.2, 192.168.0.3, 192.168.0.4, then the command would be:

bash
CONTROL_PLANE_IP=("192.168.0.2" "192.168.0.3" "192.168.0.4")
  1. If you have worker nodes, store their IP addresses in a Bash array. Replace each placeholder with the actual IP address of your worker node. You can include as many IP addresses as you need.
bash
WORKER_IP=("<worker-ip-1>" "<worker-ip-2>" "<worker-ip-3>")
Step 3: Determine the Kubernetes endpoint

You have set up multiple control planes for high availability, but true high availability is only provided if the Kubernetes API server endpoint can reach all control plane nodes.

There are two common ways to configure this:

  • Dedicated Load Balancer: Set up a dedicated load balancer that routes to the control plane nodes.
  • DNS Records: Create several DNS records pointing to all control plane nodes.

This allows you to pass one IP address or DNS name to all control plane nodes during setup.

Here’s how to configure each option.

Dedicated Load Balancer

If you use a cloud provider or have your own load balancer (such as HAProxy, NGINX reverse proxy, or F5 load balancer), setting up a dedicated load balancer is a natural choice.

It is also important to note that if you create a cluster with Omni, Omni automatically becomes a load balancer for Kubernetes endpoints.

Configure the frontend to listen on TCP port 6443 and forward traffic to the address of the Talos control plane node.

A Kubernetes endpoint is the IP address or DNS name of the load balancer frontend followed by a port (e.g., https://myK8s.mydomain.io:6443)

Note: You cannot use an HTTP load balancer because the Kubernetes API server handles TLS termination and mutual TLS authentication.

DNS records

You can also configure Kubernetes endpoints using DNS records. Simply add multiple A or AAAA records to your DNS name, one for each control plane.

For example, you could add:

txt
kube.cluster1.mydomain.com  IN  A  192.168.0.10
kube.cluster1.mydomain.com  IN  A  192.168.0.11
kube.cluster1.mydomain.com  IN  A  192.168.0.12

Then the endpoint is:

txt
https://kube.cluster1.mydomain.com:6443
References and Guidelines (NOTE)

Note: This endpoint is used as a fallback if KubePrism becomes unavailable, such as due to a reboot. It also ensures that Talos nodes can continue to find and communicate with the control plane even when other discovery services are unavailable.

Step 4: Store the endpoint in a variable

Set a variable to store the endpoint selected in step 3. Placeholder into actual endpoint

bash
export YOUR_ENDPOINT=<your_endpoint>
Step 5: Create Secret Bundle

The secret bundle is a file that contains all the encryption keys, certificates, and tokens needed to secure your Talos Linux cluster.

To create a secret bundle, run:

bash
talosctl gen secrets -o secrets.yaml
Step 6: Create Machine Configuration

Follow these steps to create a machine configuration.

  1. Run the following command to set a variable for the cluster name. Replace with the name you want to give your cluster.
bash
export CLUSTER_NAME=<your_cluster_name>
  1. To generate a machine configuration file using the secret bundle, run the following command:
bash
talosctl gen config --with-secrets secrets.yaml $CLUSTER_NAME https://$YOUR_ENDPOINT:6443

This command creates three files.

  • controlplane.yaml: Configuration for the control plane.
  • worker.yaml: Configuration for the worker node.
  • talosconfig: Configuration file used by talosctl to connect to and authenticate to the cluster.
Step 7: Unmount the ISO

Remove the installation USB drive or unmount the ISO from all control plane and worker nodes. This will prevent you from accidentally installing to a USB drive and will give you a clearer idea of which disk to install to.

Step 8: Understand Nodes

Generally, the basic machine configuration of the control plane and worker nodes is sufficient to run the cluster. However, depending on your specific environment, you may need to customize certain settings such as network interfaces and disk configuration.

Follow these steps to ensure your machine configuration is set up correctly.

  1. Check network interfaces: Running this command will allow you to see all network interfaces on all nodes, whether control plane or workers.
    Replace the IP of the node you want to check.
    Note: Network ID with the Operation Status (OPER) value up
    Copy .
bash
talosctl --nodes <node-ip-address> get links --insecure
  1. Check available disks: Run this command to check all available disks on all nodes. Replace with the IP address of the node you want to check.
bash
talosctl get disks --insecure --nodes <node-ip-address>
  1. Check the configuration files: Open the configuration files worker.yaml, controlplane.yaml in your preferred editor.

    Verify that the values match the network and disk settings of your worker and control plane nodes. If the values โ€‹โ€‹do not match, you may need to update your machine configuration.

References and Guidelines (NOTE)

Note: See Talos CLI Reference for additional commands to gather detailed information about nodes and clusters.

Step 9: Patch Machine Configuration (Optional)

You can patch your worker and control plane machine configurations to reflect the correct network interfaces and disks on your control plane nodes.

Follow these steps to patch your machine configuration.

  1. Create a patch file for the configuration you want to modify.
bash
touch controlplane-patch-1.yaml # For patching the control plane nodes configuration
   touch worker-patch-1.yaml # For patching the worker nodes configuration

Note: You do not need to create both patch files. Only create patches for configurations that actually need to be fixed.

If you want to create multiple subsequent patches for the same machine configuration, you can also create multiple patch files (e.g. controlplane-patch-2.yaml, controlplane-patch-3.yaml).

  1. Copy and paste this YAML code block and add the correct hardware values to each patch file.

    For example, to use the network interface and disk information collected from the controlplane-patch-1control plane node:

bash
# controlplane-patch-1 file
   machine:
     network:
       interfaces:
         - interface: <control-plane-network-interface>  # From control plane node
           dhcp: true
     install:
       disk: /dev/<control-plane-disk-name> # From control plane node

For worker-patch-1.yaml, use the worker node’s network interface and disk information.

bash
# worker-patch-1.yaml file
   machine:
     network:
       interfaces:
         - interface: <worker-network-interface>  # From worker node
           dhcp: true
     install:
       disk: /dev/<worker-disk-name> # From worker node
  1. Apply various patch files to suit different machine configurations.
    โ€“ for Control-plane
bash
talosctl machineconfig patch controlplane.yaml --patch @controlplane-patch-1.yaml --output controlplane.yaml
  • for Worker
bash
talosctl machineconfig patch worker.yaml --patch @worker-patch-1.yaml --output worker.yaml

You can also learn more about patching in the Configuration Patching documentation.

Step 10: Configure multihomed machines

If the machine is multihomed (that is, it has more than one IPv4 and/or IPv6 address in addition to the loopback address), additional configuration is required.ย For more information, see Multihoming

Step 11: Apply machine configuration

To apply machine configuration:

  1. Run the following command to apply the controlplane.yaml configuration to the control plane nodes.
bash
for ip in "${CONTROL_PLANE_IP[@]}"; do
     echo "=== Applying configuration to node $ip ==="
     talosctl apply-config --insecure \
       --nodes $ip \
       --file controlplane.yaml
     echo "Configuration applied to $ip"
     echo ""
   done
  1. Run the following command to apply the worker.yaml configuration to your worker nodes.
bash
for ip in "${WORKER_IP[@]}"; do
     echo "=== Applying configuration to node $ip ==="
     talosctl apply-config --insecure \
       --nodes $ip \
       --file worker.yaml
     echo "Configuration applied to $ip"
     echo ""
   done
Step 12: Manage Talos Configuration File

talosconfig is the key to managing your Talos Linux cluster. Without it, talosctl cannot be used to authenticate or communicate with cluster nodes.

There are two options for managing:

  1. Merge the new talosconfig file into the default configuration file located at ~/.talos/configย 
    bash
    talosctl config merge ./talosconfig
  2. Copy the configuration file to your ~/.talos directory and set the TALOSCONFIG environment variable
  3. Copy the configuration file to the ~/.talos directory and set the TALOSCONFIG environment variable
    bash
    mkdir -p ~/.talos
    cp ./talosconfig ~/.talos/config
    export TALOSCONFIG=~/.talos/config
Step 13: Set the endpoints of the control plane nodes

If an individual node becomes unavailable, talosctl automatically configures endpoints to load balance requests and failover between control plane nodes.

Run the following command to configure the endpoint. Replace with the IP address of your control plane node.

bash
talosctl config endpoint <control_plane_IP_1> <control_plane_IP_2> <control_plane_IP_3>

For example:

If the IP address of the control plane node is 192.168.0.2, 192.168.0.3, 192.168.0.4, the command is:

bash
talosctl config endpoint 192.168.0.2 192.168.0.3 192.168.0.4
Step 14: Bootstrap the Kubernetes Cluster

Wait for the control plane nodes to finish booting, then run the command below to bootstrap the etcd cluster.

Replace the placeholder with the IP address of one of the three control plane nodes.

bash
talosctl bootstrap --nodes <control-plane-IP>
Step 15: Obtain Kubernetes access

To use kubectl on your cluster, download the kubeconfig file. This command must be run against a single control plane node.

There are two options for managing kubeconfig: Replace with the IP address of one of your control plane nodes.

  • Merge to default kubeconfig:
bash
talosctl kubeconfig --nodes <control-plane-IP>
  • Create a separate file kubeconfig
bash
talosctl kubeconfig alternative-kubeconfig --nodes <control-plane-IP>
  export KUBECONFIG=./alternative-kubeconfig
Step 16: Verify that the node is running

To check if the node is running, run the following command.

bash
kubectl get nodes

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *