Repository

helm repo add argo https://argoproj.github.io/argo-helm

Pipelines

RBAC

• Kubernetes

wait: Error (exit code 64): workflowtaskresults.argoproj.io is forbidden: User "system:serviceaccount:deploy-system:default" cannot create resource "workflowtaskresults" in API group "argoproj.io" in the namespace "deploy-system"

# 최고권한 클러스터롤은 가지고 있으나, 해당 네임스페이스에 sa가 없는 경우
  k create sa deploy-admin -n deploy-system
  k edit clusterrolebindings.rbac.authorization.k8s.io cmp-admin-rolebinding
  >>>
  ...
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cmp-admin-role
  subjects:
  - kind: ServiceAccount
    name: cmp-admin
    namespace: default
  ######## Insert ########
  - kind: ServiceAccount
    name: deploy-admin
    namespace: deploy-system
  <<<

• Argo-workflows

spec:
    serviceAccountName: deploy-admin

Vault에서 값 주입받기

• – vault에 deploy-admin용 role 생성

  bash
  
  
  Copy
  

  k exec -it -n portal-system sts/vault -- sh
    ### START IN CONTAINER
    vault login
  
    role_name=deploy-admin-role
    policy_name=admin-policy
    kube_sa=deploy-admin
    kube_sa_ns=deploy-system
  
    vault write auth/kubernetes/role/${role_name} \
        bound_service_account_names=${kube_sa} \
        bound_service_account_namespaces=${kube_sa_ns} \
        policies=${policy_name} \
        ttl=24h \
        audience="https://kubernetes.default.svc.cluster.local"

  – Vault – Ingejctor

  yaml
  
  
  Copy
  

  spec:
      template:
        metadata:
          annotations:
          labels:
            vault.hashicorp.com/agent-inject: "true"
            vault.hashicorp.com/role: "deploy-admin-role"
            vault.hashicorp.com/agent-inject-secret-common: "secret/mgt/pipelines/common"
            vault.hashicorp.com/agent-inject-template-common: |
              {{- with secret "secret/mgt/pipelines/common" -}}
              export image_registry_id="{{ .Data.data.username }}"
              export image_registry_pw="{{ .Data.data.password }}"
              {{- end -}}
    		    vault.hashicorp.com/agent-inject-secret-exception: "secret/mgt/pipelines/build1"
    		    vault.hashicorp.com/agent-inject-template-exception: |
    		      {{- with secret "secret/mgt/pipelines/build1" -}}
    		      export NPM_VERSION="{{ .Data.data.version }}"
    		      {{- end -}}

Sample

• UI with Vault-agent-injector
  yaml
  
  
  Copy
  

  metadata:
    name: workflow-vault-test
    namespace: deploy-system
    labels:
      example: "true"
  spec:
    serviceAccountName: deploy-admin
    entrypoint: argosay
    templates:
      - name: argosay
        metadata:
          annotations:
            vault.hashicorp.com/agent-inject: "true"
            vault.hashicorp.com/role: "deploy-admin-role"
            vault.hashicorp.com/agent-inject-secret-common: "secret/mgt/pipelines/common"
            vault.hashicorp.com/agent-inject-template-common: |
              {{- with secret "secret/mgt/pipelines/common" -}}
              export registry="{{ .Data.data.registry }}"
              {{- end -}}
        container:
          name: main
          image: argoproj/argosay:v2
          command:
            - /bin/bash
          args:
            - -ec
            - |
              source /vault/secrets/common
              echo $registry
    ttlStrategy:
      secondsAfterCompletion: 300
    podGC:
      strategy: OnPodCompletion

Features

인증

• Server 인증 (ID / PW)
• Client 인증 (Kubernetes RBAC)

#!/bin/bash
  # `./generate_kubeconfig.sh dev 10.0.X.X mypasswd 2222 > .my.kubeconfig`
  ns=default
  sa=cmp-admin

  # SSH 명령을 단 한번 호출하는 버전이 가장 안정적임
  remote_script=$(cat <<EOF
  # 1. ClusterRole 생성
  kubectl apply -f - <<'E1'
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: ${sa}-role
  rules:
  - apiGroups:
    - '*'
    resources:
    - '*'
    verbs:
    - '*'
  E1

  # 2. ServiceAccount 생성
  kubectl create sa ${sa} -n ${ns} --dry-run=client -o yaml | kubectl apply -f -

  # 3. ClusterRoleBinding 생성
  kubectl create clusterrolebinding ${sa}-rolebinding \
    --clusterrole ${sa}-role \
    --serviceaccount ${ns}:${sa} \
    --dry-run=client -o yaml | kubectl apply -f -

  # 4. Token Secret 생성
  kubectl apply -f - <<'E2'
  apiVersion: v1
  kind: Secret
  metadata:
    name: ${sa}-token
    namespace: ${ns}
    annotations:
      kubernetes.io/service-account.name: ${sa}
  type: kubernetes.io/service-account-token
  E2
  EOF
  )

  remote_script2=$(cat <<EOF
  # 5. token 값 출력
  kubectl get secret ${sa}-token -n ${ns} -o jsonpath="{.data.token}"
  EOF
  )

  printf "%s" "$remote_script" | /bin/bash -s > /dev/tty
  token_b64=$(printf "%s" "$remote_script2" | /bin/bash -s)
  printf "%s" "$token_b64" | base64 -d

• Bearer Token 사용

echo "Bearer $(kubectl get secret cmp-admin-token -o jsonpath="{.data.token}" | base64 -d)"

• SSO 인증
• ArgoCD DEX
  • Use Argo CD Dex for authentication – Argo Workflows – The workflow engine for Kubernetes