Fredric News

홈랩 Harvester HCI 구축: KubeVirt 가상머신 및 스토리지

💡 핵심 요약 (Executive Summary)

본 가이드는 Harvester HCI 가상화 핵심 가이드 및 실전 아키텍처의 핵심 개념, 실전 구성 명령어 및 아키텍처 베스트 프랙티스를 다룹니다. 클라우드 네이티브 및 인프라 엔지니어링 실무에 즉시 활용 가능하도록 체계적으로 정리했습니다.

📑 Table of Contents

Installation

HCI: Host

Reference & Guidelines (NOTE)

Hosts → \ → Edit Config 클릭

Disk 마운트 및 Longhorn Join
  1. Storage → Add Disk 클릭 → Disk 선택 → 저장
  • Longhorn 스케줄이 끝나면 로드됨

HCI: Creation

SSH Keys 관리
  • 비밀키는 잘 가지고 있고, 공개키 등록하면 됨
Upload Image
🔗 https://docs.harvesterhci.io/v1.1/upload-image
Reference & Guidelines (NOTE)

Images → Create

Cloud Image repositories
bash
customization:
          extraKernelArgs:
              - talos.platform=kvm
              - console=ttyS0,115200n8
          systemExtensions:
              officialExtensions:
                  - siderolabs/iscsi-tools
                  - siderolabs/qemu-guest-agent
                  - siderolabs/util-linux-tools
URL을 통한 이미지 업로드

  1. .iso / .qcow / .qcow2 / .raw / .img 이미지 링크 복사

  2. Harvester에 삽입 및 Name 지정. (자동 라벨링) → Create

  3. Storage Class 선택

    1. 참고: Standalone용 Storage Class 생성

  4. Download 확인

Bootable Volume 생성
Reference & Guidelines (NOTE)

호스트 kubernetes에 PVC로 생성됨 (pre-bounded)

VM: Windows VDI Server Setup
Reference & Guidelines (NOTE)

Windows는 VirtIo 타입 볼륨 드라이버가 기본이미지에 없으므로, virtio-win 이미지를 같이 마운트해준다. (Virtio-win)


Windows Server 2025 Installation

  • Basics

  • Volumes

  • WebVNC 접근

  • 적당히 Windows 설정 하다가 에디션 선택창에서 Desktop Experience를 선택. (GUI)

  • 이후 Location to Install Windows Server 화면 등장함

  • Load Driver 클릭

  • virtio-win disk → amd64 → 최신 드라이브 선택

  • Red Hat VirtIO SCSI controller 선택 후 Install

  • 이제 잘 조회되는 VirtIO 디스크에 Install

Network Setup
powershell
Get-PSDrive -PSProvider FileSystem #확인된 virtio drive=E:
pnputil /add-driver "E:\NetKVM\2k25\amd64\*.inf" /install

# 인터넷 연결및 ip할당 확인
ipconfig
ping 8.8.8.8
All-in-one Virtio Drivers Setup
  • virtio-win-gt-x64.msi 설치

장치관리자에서 확인

Enable RDP
  • 시작 버튼 클릭 > 설정(Settings) > 시스템(System) > **원격 데스크톱(Remote Desktop)**으로 이동합니다.
  • ‘켬(On)’으로 토글을 바꿉니다.
서비스 노출 / 접근
Reference & Guidelines (NOTE)

RDP 기본 포트: 3389

기본 접근계정: Administrator

  • LoadBalancer
부팅디스크 및 드라이버 디스크 제거
  • VM: Volume Unmount 및 제거
VM: Ubuntu Server
Configuration
서비스 노출 / 접근
Reference & Guidelines (NOTE)

SSH 기본 포트: 22

기본 접근계정: ubuntu

디스크 마운트
bash
# lvm없이 ext4
sudo mkfs.ext4 /dev/vdb
sudo mkdir /workspace
sudo mount /dev/vdb /workspace
Network
Cluster Network
Reference & Guidelines (NOTE)

Harvester Network Options 이해

  • Cluster Network – Masquerade (L3)

  • 마스커레이드 모드는 VM을 외부 네트워크에 노출하지 않기 때문에 외부에서 서비스 접근을 위해서는 LoadBalancer 설정으로 NAT를 구성해야 함

  • Cluster Network – Bridge (L2)

  • 브릿지 모드는 VM을 업스트림 라우터에 노출하여 DHCP를 할당받을 수 있음.
VM Network 생성
Reference & Guidelines (NOTE)

Harvester Network Options 이해

  • VM Network – Bridge (L2)
  • 브릿지 모드는 VM을 업스트림 라우터에 노출하여 DHCP를 할당받을 수 있음.
Harvester 용어 표준 네트워크 용어 핵심 메커니즘
Untagged Standard L2 LAN 패킷에 아무런 꼬리표 없이 물리 스위치(Access Port)와 직접 통신
L2VLAN 802.1Q VLAN 패킷 헤더에 VLAN ID(Tag)를 붙여 물리 스위치(Trunk Port)와 통신
Overlay VXLAN L2 패킷을 UDP(L3/L4) 패킷으로 감싸서(Encapsulation) 전송
  • untagged: 아주 단순함
LoadBalancer
Reference & Guidelines (NOTE)

– DHCP: Host-dependent
– VM NetworksIP Pools: LoadBalancer ← IP Pools ← VM Networks (VLAN or Else)

  • 서비스 포트 포워딩

  • 대상 VM 라벨셀렉트

HCI: Management

VM: Resizing CPU / Memory
  • edit config
VM: Resizing Disk
Reference & Guidelines (NOTE)

LVM구성된 경우에는 디스크를 새로 추가하여 논리적으로 결합하는게 좋음

VM: Volume Unmount 및 제거

  • Edit Config

  • Drive 이미지나 설치 이미지 제거

Features

Standalone용 Storage Class 생성
Reference & Guidelines (NOTE)

Harvester의 기본 Longhorn 구성은Replicas=3 으로 되어있으나, 싱글노드에서는 레플리카 스케쥴이 불가하므로 VM생성시 Volume degreaded가 발생함.

  • 현재 Degreaded인 볼륨 즉시 복구
    1. Longhorn Dashboard 접근
    2. Volume의 설정에서 Update Replicas Count: 1로 변경

  • 기존 harvester-longhorn 등의 SC는 수정불가이므로 신규 SC를 생성

  • Number of Replica = 1

  • set Default

Enable Rancher & Longhorn UI
🔗 https://docs.harvesterhci.io/v1.6/troubleshooting/harvester#access-embedded-rancher-and-longhorn-dashboards
Reference & Guidelines (NOTE)

디버깅 및 유효성 검사 목적으로만 내장된 Rancher 및 Longhorn 대시보드 사용을 지원

  • Preference > Advanced Features > Enable Extension developer features

  • Support
Enable Vim Key Mapping
  • Preference > YAML Editor Setting > Vim

GitOps
🔗 https://www.youtube.com/watch?v=CrzsIXXY5zo
Rancher Connection

  1. Rancher

  2. Harvester

Solution
상세 내용 펼치기

Get "https://rancher.mgnt.fredric18.uk/v3/import/6v5bg947vq7zm2tb8wjxbbxp9tgq8w6rcx7bskdd6hw4h2lt4r4gcp_c-dm6zk.yaml": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

– In Harvester CLI DNS Setting

bash
# Unbound 동작 확인
  rancher@fredric18:~> nslookup rancher.mgnt.fredric18.uk 192.168.1.1
  Server:		192.168.1.1
  Address:	192.168.1.1#53

  Name:	rancher.mgnt.fredric18.uk
  Address: 192.168.118.13

  # CoreDNS에 Unbound 적용
  k exec -it -n harvester-system deployments/harvester --container apiserver -- curl -kv https://rancher.mgnt.fredric18.uk/v3/import/6v5bg947vq7zm2tb8wjxbbxp9tgq8w6rcx7bskdd6hw4h2lt4r4gcp_c-dm6zk.yaml
  >>>
  * Could not resolve host: rancher.mgnt.fredric18.uk
  * shutting down connection #0
  curl: (6) Could not resolve host: rancher.mgnt.fredric18.uk
  command terminated with exit code 6
  <<<

  k get cm -n kube-system -l k8s-app=kube-dns
  k edit cm -n kube-system rke2-coredns-rke2-coredns
  >>>edit
  forward  . 192.168.1.1 8.8.8.8
  <<<
  k delete pods -n kube-system -l k8s-app=kube-dns
  k exec -it -n harvester-system deployments/harvester --container apiserver -- curl -kv https://rancher.mgnt.fredric18.uk/v3/import/6v5bg947vq7zm2tb8wjxbbxp9tgq8w6rcx7bskdd6hw4h2lt4r4gcp_c-dm6zk.yaml
  >>>
  * Host rancher.mgnt.fredric18.uk:443 was resolved.
  * IPv6: (none)
  * IPv4: 192.168.118.13
  *   Trying 192.168.118.13:443...
  <<<

CLI

Generate VM
Masquerade 모드
Reference & Guidelines (NOTE)

SNAT vs Masquerade

  • SNAT (Source Network Address Translation): 출발지 IP를 변경하는 모든 기술을 통칭하는 상위 개념입니다. 관리자가 변환할 목적지 IP를 명시적으로 지정하여 사용하는 것이 일반적입니다.
  • Masquerade: SNAT의 동적(Dynamic) 버전입니다. 관리자가 일일이 IP를 지정하지 않아도, 패킷이 나가는 네트워크 인터페이스의 IP를 시스템이 자동으로 감지(Lookup)하여 출발지 IP로 적용합니다.
VMSet + LB for SSH
yaml
apiVersion: loadbalancer.harvesterhci.io/v1beta1
kind: LoadBalancer
metadata:
  annotations:
    loadbalancer.harvesterhci.io/namespace: harvester-public
    loadbalancer.harvesterhci.io/network: ""
    loadbalancer.harvesterhci.io/project: ""
  finalizers:
  - wrangler.cattle.io/harvester-lb-controller
  name: cks-inkang-01
  namespace: harvester-public
spec:
  backendServerSelector:
    harvesterhci.io/vmName:
    - cks-inkang-01
  healthCheck: {}
  ipam: dhcp
  listeners:
  - backendPort: 22
    name: ssh
    port: 22
    protocol: TCP
  workloadType: vm

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
    volume.kubernetes.io/storage-provisioner: driver.longhorn.io
  finalizers:
  - kubernetes.io/pvc-protection
  - wrangler.cattle.io/persistentvolumeclaim-controller
  name: cks-inkang-01-root
  namespace: harvester-public
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: harvester-longhorn-standalone
  volumeMode: Block

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    harvesterhci.io/imageId: harvester-public/image-jb4b8
    volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
    volume.kubernetes.io/storage-provisioner: driver.longhorn.io
  finalizers:
  - kubernetes.io/pvc-protection
  - wrangler.cattle.io/persistentvolumeclaim-controller
  name: cks-inkang-01-bootable
  namespace: harvester-public
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: longhorn-image-jb4b8
  volumeMode: Block

---

apiVersion: v1
data:
  networkdata: ""
  userdata: I2Nsb3VkLWNvbmZpZwpwYWNrYWdlX3VwZGF0ZTogdHJ1ZQpwYWNrYWdlczoKICAtIHFlbXUtZ3Vlc3QtYWdlbnQKcnVuY21kOgogIC0gLSBzeXN0ZW1jdGwKICAgIC0gZW5hYmxlCiAgICAtIC0tbm93CiAgICAtIHFlbXUtZ3Vlc3QtYWdlbnQuc2VydmljZQpzc2hfYXV0aG9yaXplZF9rZXlzOgogIC0gc3NoLWVkMjU1MTkKICAgIEFBQUFDM056YUMxbFpESTFOVEU1QUFBQUlLSkxpb0tESFZqRUtnTFBwbEpkdVh4YzlHR0xaa0RtaVUwNDlRRFFEYTd1CiAgICBhZG1pbkBtYWlsLmZyZWRyaWMxOC51awoKcGFzc3dvcmQ6ICJpbmthbmcxMjMjIgpjaHBhc3N3ZDoKICBsaXN0OiB8CiAgICByb290Omlua2FuZzEyMyMKICAgIHVidW50dTppbmthbmcxMjMjCiAgZXhwaXJlOiBGYWxzZQpzc2hfcHdhdXRoOiBUcnVlCg==
kind: Secret
metadata:
  labels:
    harvesterhci.io/cloud-init-template: harvester
  name: cks-inkang-01
  namespace: harvester-public
type: secret

---

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
    harvesterhci.io/vmRunStrategy: RerunOnFailure
    kubevirt.io/latest-observed-api-version: v1
    kubevirt.io/storage-observed-api-version: v1
    network.harvesterhci.io/ips: '[]'
    template-version.harvesterhci.io/customName: QK9h2nZlKw
  creationTimestamp: "2026-04-17T06:03:53Z"
  finalizers:
  - kubevirt.io/virtualMachineControllerFinalize
  - wrangler.cattle.io/VMController.CleanupPVCAndSnapshot
  generation: 1
  labels:
    harvesterhci.io/creator: harvester
    harvesterhci.io/os: ubuntu
    template.harvesterhci.io/templateID: ubuntu-tiny
  name: cks-inkang-01
  namespace: harvester-public
spec:
  runStrategy: RerunOnFailure
  template:
    metadata:
      annotations:
        harvesterhci.io/sshNames: '["harvester-public/admin"]'
      creationTimestamp: null
      labels:
        harvesterhci.io/vmName: cks-inkang-01
        harvesterhci.io/vmNamePrefix: cks-inkang
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchLabels:
                  harvesterhci.io/vmNamePrefix: cks-inkang
              topologyKey: kubernetes.io/hostname
            weight: 1
      architecture: amd64
      domain:
        cpu:
          cores: 1
          maxSockets: 1
          sockets: 1
          threads: 1
        devices:
          disks:
          - bootOrder: 1
            disk:
              bus: virtio
            name: bootable
          - bootOrder: 2
            disk:
              bus: virtio
            name: root
          - disk:
              bus: virtio
            name: cloudinitdisk
          inputs:
          - bus: usb
            name: tablet
            type: tablet
          interfaces:
          - masquerade: {}
            model: virtio
            name: default
        features:
          acpi:
            enabled: true
        machine:
          type: q35
        memory:
          guest: 4Gi
        resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: 125m
            memory: 2730Mi
      evictionStrategy: LiveMigrateIfPossible
      networks:
      - name: default
        pod: {}
      terminationGracePeriodSeconds: 120
      volumes:
      - name: bootable
        persistentVolumeClaim:
          claimName: cks-inkang-01-bootable
      - name: root
        persistentVolumeClaim:
          claimName: cks-inkang-01-root
      - cloudInitNoCloud:
          networkDataSecretRef:
            name: cks-inkang-01
          secretRef:
            name: cks-inkang-01
        name: cloudinitdisk
Bridge 모드
Reference & Guidelines (NOTE)

Bridge는 Harvester에서 VM IP를 관리하지 않고, 업스트림의 DHCP서버에 할당을 요청하므로 Harvester와 동일레벨의 IP를 할당하는것이 가능하다.

일반적으로 vLAN 인터페이스를 만들어 NetworkAttachmentDefinition 를 선언한다.

Upstream DHCP vLAN Interface
Bridge Network 구성 (NetworkAttachmentDefinition)
yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: vlan1
  namespace: harvester-public
  annotations:
    network.harvesterhci.io/clusternetwork: mgmt
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "vlan1",
    "type": "bridge",
    "bridge": "mgmt-br",
    "vlan": 30
  }'

– clusterNetwork (static)

bash
apiVersion: v1
  items:
  - apiVersion: network.harvesterhci.io/v1beta1
    kind: ClusterNetwork
    metadata:
      creationTimestamp: "2025-12-07T14:48:39Z"
      finalizers:
      - wrangler.cattle.io/harvester-network-manager-cn-controller
      generation: 1
      name: mgmt
      resourceVersion: "4969"
      uid: 1c602bce-a7c2-4fa5-8a28-95d17af5ca25
    status:
      conditions:
      - lastUpdateTime: "2025-12-07T14:48:39Z"
        status: "True"
        type: ready
  kind: List
  metadata:
    resourceVersion: ""
VMSet
yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
    volume.kubernetes.io/storage-provisioner: driver.longhorn.io
  finalizers:
  - kubernetes.io/pvc-protection
  - wrangler.cattle.io/persistentvolumeclaim-controller
  name: cks-inkang-03-root
  namespace: harvester-public
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: harvester-longhorn-standalone
  volumeMode: Block

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    harvesterhci.io/imageId: harvester-public/image-jb4b8
    volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
    volume.kubernetes.io/storage-provisioner: driver.longhorn.io
  finalizers:
  - kubernetes.io/pvc-protection
  - wrangler.cattle.io/persistentvolumeclaim-controller
  name: cks-inkang-03-bootable
  namespace: harvester-public
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: longhorn-image-jb4b8
  volumeMode: Block

---

apiVersion: v1
data:
  networkdata: ""
  userdata: I2Nsb3VkLWNvbmZpZwpwYWNrYWdlX3VwZGF0ZTogdHJ1ZQpwYWNrYWdlczoKICAtIHFlbXUtZ3Vlc3QtYWdlbnQKcnVuY21kOgogIC0gLSBzeXN0ZW1jdGwKICAgIC0gZW5hYmxlCiAgICAtIC0tbm93CiAgICAtIHFlbXUtZ3Vlc3QtYWdlbnQuc2VydmljZQpzc2hfYXV0aG9yaXplZF9rZXlzOgogIC0gc3NoLWVkMjU1MTkKICAgIEFBQUFDM056YUMxbFpESTFOVEU1QUFBQUlLSkxpb0tESFZqRUtnTFBwbEpkdVh4YzlHR0xaa0RtaVUwNDlRRFFEYTd1CiAgICBhZG1pbkBtYWlsLmZyZWRyaWMxOC51awoKcGFzc3dvcmQ6ICJpbmthbmcxMjMjIgpjaHBhc3N3ZDoKICBsaXN0OiB8CiAgICByb290Omlua2FuZzEyMyMKICAgIHVidW50dTppbmthbmcxMjMjCiAgZXhwaXJlOiBGYWxzZQpzc2hfcHdhdXRoOiBUcnVlCg==
kind: Secret
metadata:
  labels:
    harvesterhci.io/cloud-init-template: harvester
  name: cks-inkang-03
  namespace: harvester-public
type: secret

---

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
    harvesterhci.io/vmRunStrategy: RerunOnFailure
    kubevirt.io/latest-observed-api-version: v1
    kubevirt.io/storage-observed-api-version: v1
    network.harvesterhci.io/ips: '[]'
    template-version.harvesterhci.io/customName: QK9h2nZlKw
  creationTimestamp: "2026-04-17T06:03:53Z"
  finalizers:
  - kubevirt.io/virtualMachineControllerFinalize
  - wrangler.cattle.io/VMController.CleanupPVCAndSnapshot
  generation: 1
  labels:
    harvesterhci.io/creator: harvester
    harvesterhci.io/os: ubuntu
    template.harvesterhci.io/templateID: ubuntu-tiny
  name: cks-inkang-03
  namespace: harvester-public
spec:
  runStrategy: RerunOnFailure
  template:
    metadata:
      annotations:
        harvesterhci.io/sshNames: '["harvester-public/admin"]'
      creationTimestamp: null
      labels:
        harvesterhci.io/vmName: cks-inkang-03
        harvesterhci.io/vmNamePrefix: cks-inkang
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchLabels:
                  harvesterhci.io/vmNamePrefix: cks-inkang
              topologyKey: kubernetes.io/hostname
            weight: 1
      architecture: amd64
      domain:
        cpu:
          cores: 1
          maxSockets: 1
          sockets: 1
          threads: 1
        devices:
          disks:
          - bootOrder: 1
            disk:
              bus: virtio
            name: bootable
          - bootOrder: 2
            disk:
              bus: virtio
            name: root
          - disk:
              bus: virtio
            name: cloudinitdisk
          inputs:
          - bus: usb
            name: tablet
            type: tablet
          interfaces:
          - bridge: {}
            model: virtio
            name: default
        features:
          acpi:
            enabled: true
        machine:
          type: q35
        memory:
          guest: 4Gi
        resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: 125m
            memory: 2730Mi
      evictionStrategy: LiveMigrateIfPossible
      networks:
      - name: default
        multus:
          networkName: harvester-public/vlan1
      terminationGracePeriodSeconds: 120
      volumes:
      - name: bootable
        persistentVolumeClaim:
          claimName: cks-inkang-03-bootable
      - name: root
        persistentVolumeClaim:
          claimName: cks-inkang-03-root
      - cloudInitNoCloud:
          networkDataSecretRef:
            name: cks-inkang-03
          secretRef:
            name: cks-inkang-03
        name: cloudinitdisk
VM Diffs with Masquerade VS Bridge
yaml
# ... (앞부분 생략)
    spec:
      # ...
      networks:
      - name: default
        # pod: {}  <-- 이 부분을 지우고 아래 multus를 추가합니다.
        multus:
          networkName: harvester-public/vlan1 # <-- 실제 사용하려는 네트워크 이름 (Namespace/Name)
      domain:
        # ...
        devices:
          interfaces:
          - bridge: {} # <-- masquerade에서 bridge로 변경
            model: virtio
            name: default
# ... (뒷부분 생략)
Backup
bash
# vm 목록
k get pods -n harvester-public

# virsh vm listing
k exec -it virt-launcher-management-00-kh82s -n harvester-public -- virsh list --all

# dump
k exec -it virt-launcher-management-00-kh82s -n harvester-public -- virsh dumpxml harvester-public_management-00 > harvester-public_management-00.xml

# 디스크 경로 확인
k exec -it virt-launcher-management-00-kh82s -n harvester-public -- virsh domblklist

Add-ons

Reference & Guidelines (NOTE)

Advanced → Add-ons

Observability
Logging
Monitoring

  • Prometheus UI로 이동

  • Grafana 대시보드로 이동

  • 로그인 없이 조회하는 목적이나, 혹시 개발이 필요한 경우 초기비밀번호 확인

bash
# Grafana admin 패스워드 확인
  kubectl get secret -n cattle-monitoring-system rancher-monitoring-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
  • Change Password로 Harvester와 동일하게 변경

Posted:

/

Category:

Author:

fredric

Tags:

, , , , , , , , , , , , , , ,

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *