手把手教你写一个通用的helm chart
[TOC]
1. 模板介绍
首先,放上此模板链接:
https://github.com/ygqygq2/charts/tree/master/mod-chart
此chart可当作POD单image的通用模板,只需要使用sed
替换下chart名,并修改下README.md
和NOTES.txt
就可以了。下文,我通过复制此chart成example-chart
来作示范说明。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
[root@master1 mod-chart]# tree . ├── Chart.yaml # chart版本信息文件 ├── README.md # chart说明文件 ├── templates # kubernetes资源yaml模板 │ ├── configmap.yaml # configmap模板 │ ├── deployment-statefulset.yaml # deployment或statefulset模板 │ ├── _helpers.tpl # 辅助模板和 partials │ ├── ingress.yaml # ingress模板 │ ├── NOTES.txt # 部署chart后输出的帮助文档 │ ├── pvc.yaml # pvc模板 │ ├── secret.yaml # secret模板 │ ├── service-headless.yaml # service headless模板 │ └── service.yaml # service模板 └── values.yaml # 默认设置 1 directory, 12 files [root@master1 mod-chart]# helm3 lint --strict . 1 chart(s) linted, 0 chart(s) failed |
2. 新chart制作
注:
下文中文件内容我保留,只加注释。
注释中需要修改的地方[*]
标记为必选,[-]
标识为可选。
2.1 目录准备
将模板mod-chart
复制成example-chart
,并作内容替换。
1 2 3 4 5 |
rsync -avz mod-chart/ example-chart/ cd example-chart/ sed -i 's@mod-chart@example-chart@g' *.* sed -i 's@mod-chart@example-chart@g' templates/*.* |
2.2 修改Chart.yaml
vim Chart.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
apiVersion: v1 # 当前helm api版本,不需要修改 appVersion: 1.14.2 # 此处为你应用程序的版本号 [*] description: Chart for the nginx server # 介绍此chart是干嘛的,按需求修改 engine: gotpl # go模板引擎,不需要修改 [-] name: example-chart # 模板名,对应目录名 [*] version: 1.0.0 # 此chart版本号 [*] home: http://www.nginx.org # 应用程序官网 [*] icon: https://bitnami.com/assets/stacks/nginx/img/nginx-stack-220x234.png # 应用程序logo地址 [*] keywords: # 关键字列表 [*] - nginx - http - web - www - reverse proxy maintainers: # 维护人员列表 [*] - email: 29ygq@sina.com name: Chinge Yang sources: # 应用程序来源 [-] - https://github.com/bitnami/bitnami-docker-nginx |
2.3 修改values.yaml
因为values.yaml
设置涉及到yaml格式,yaml文件格式说明可以看这篇文章:
http://www.ruanyifeng.com/blog/2016/07/yaml.html
这里提几个常用的地方:
- 使用2个空格作缩进;
- 确认数字为字符类型时,使用双引号引起来;
- 为了迎合helm3的规范,空定义最好将相关符号补上:
1 2 3 4 |
string: "" list: [] map: {} |
没什么特殊要求,一般需要修改的地方有image
、service
、healthCheck
、persistentVolume.mountPaths
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 |
# Default values for mod-chart. # This is a YAML-formatted file. # Declare variables to be passed into your templates. ## Global Docker image parameters ## Please, note that this will override the image parameters, including dependencies, configured to use the global value ## Current available global Docker image parameters: imageRegistry and imagePullSecrets ## global: # 设置后覆盖后面默认的镜像仓库 imageRegistry: "" imagePullSecrets: [] # - myRegistryKeySecretName statefulset: enabled: false ## String to partially override fullname template (will maintain the release name) ## nameOverride: "" ## String to fully override fullname template ## fullnameOverride: "" ## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% . ## You can change type to <code>Recreate</code> or can uncomment <code>rollingUpdate</code> specification and adjust them to your usage. deploymentStrategy: {} # rollingUpdate: # maxSurge: 25% # maxUnavailable: 25% # type: RollingUpdate # 副本个数 replicaCount: 1 # 容器image及tag image: registry: docker.io repository: bitnami/nginx tag: latest pullPolicy: IfNotPresent # IfNotPresent: 有则不拉(减少流量和操作步骤),Always: 不管tag总拉(适合tag不变时更新) pullSecrets: [] # - private-registry-key service: type: ClusterIP # 一般不用修改 ingressPort: 8080 ports: http: # 多端口暴露时,复制一段 port: 8080 # Service port number for client-a port. protocol: TCP # Service port protocol for client-a port. ## env set ## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/ env: [] # - name: DEMO_GREETING # value: "Hello from the environment" # - name: DEMO_FAREWELL # value: "Such a sweet sorrow" ## command set startCommand: [] # - "java -Xdebug -Xnoagent -Djava.compiler=NONE" # - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n" # - "-Djava.security.egd=file:/dev/urandom" # - "-jar /test.jar" # - "-Duser.timezone=GMT+08" ## Enable configmap and add data in configmap config: enabled: false subPath: "" mountPath: /conf data: {} ############################# 示例 #################################### ## 以下示例,挂载文件至 /conf/app.conf # enabled: true # mountPath: /conf/app.conf # subPath: app.conf # 使用subPath时,上面mountPath路径写文件完整绝对路径 # data: # app.conf: |- # appname = example-chart ## 以下示例,挂载多个文件至 /conf/ 下 # enabled: true # mountPath: /conf # 不使用subPath # data: # app.conf: |- # appname = example-chart # bpp.conf: |- # bppname # ## 挂载多个文件至多个不同路径,需要相应修改 templates/deployment-statefulset.yaml ############################# 示例 #################################### ## To use an additional secret, set enable to true and add data ## 用法同上,不另作说明 secret: enabled: false mountPath: /etc/secret-volume subPath: "" readOnly: true data: {} ## liveness and readiness ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ healthCheck: enabled: true type: tcp # http/tcp port: http # 健康检查的端口名或端口 httpPath: '/' # http时必须设置 livenessInitialDelaySeconds: 10 # 初始延迟秒数 livenessPeriodSeconds: 10 # 检测周期,默认值10,最小为1 readinessInitialDelaySeconds: 10 # 初始延迟秒数 readinessPeriodSeconds: 10 # 检测周期,默认值10,最小为1 resources: {} # 容器资源设置 # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: # cpu: 100m # memory: 128Mi # requests: # cpu: 100m # memory: 128Mi ## Node labels and tolerations for pod assignment ### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector ### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature labels: {} podAnnotations: {} nodeSelector: {} tolerations: [] affinity: {} annotations: {} ## Enable persistence using Persistent Volume Claims ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ ## persistentVolume: # 是否存储持久化 enabled: false ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, azure-disk on ## Azure, standard on GKE, AWS & OpenStack) ## storageClass: "-" accessMode: ReadWriteOnce annotations: {} # helm.sh/resource-policy: keep size: 1Gi # 大小 existingClaim: {} # 使用已存在的pvc mountPaths: [] # - name: data-storage # mountPath: /config # subPath: config # 多个路径使用同一个pvc使用subPath,用法同上面config中示例说明 # - name: data-storage # mountPath: /data # subPath: data ingress: # 是否使用nginx暴露域名或端口 enabled: false annotations: {} # kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" path: / hosts: - chart-example.local tls: [] # - secretName: chart-example-tls # hosts: # - chart-example.local ## Add init containers. e.g. to be used to give specific permissions for data ## Add your own init container or uncomment and modify the given example. initContainers: [] ## Prometheus Exporter / Metrics ## metrics: enabled: false image: registry: docker.io repository: nginx/nginx-prometheus-exporter tag: 0.1.0 pullPolicy: IfNotPresent ## Optionally specify an array of imagePullSecrets. ## Secrets must be manually created in the namespace. ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ ## pullSecrets: [] # - myRegistrKeySecretName ## Metrics exporter pod Annotation and Labels podAnnotations: # prometheus.io/scrape: "true" # prometheus.io/port: "9113" ## Metrics exporter resource requests and limits ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ ## resources: {} ## Uncomment and modify this to run a command after starting the core container. ## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/ lifecycle: {} # preStop: # exec: # command: ["/bin/bash","/pre-stop.sh"] # postStart: # exec: # command: ["/bin/bash","/post-start.sh"] ## Deployment additional volumes. deployment: additionalVolumes: [] ## init containers ## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ ## Add init containers. e.g. to be used to give specific permissions for data ## Add your own init container or uncomment and modify the given example. initContainers: {} # - name: fmp-volume-permission # image: busybox # imagePullPolicy: IfNotPresent # command: ['chown','-R', '200', '/extra-data'] # volumeMounts: # - name: extra-data # mountPath: /extra-data ## Additional containers to be added to the core pod. additionalContainers: {} # - name: my-sidecar # image: nginx:latest # - name: lemonldap-ng-controller # image: lemonldapng/lemonldap-ng-controller:0.2.0 # args: # - /lemonldap-ng-controller # - --alsologtostderr # - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration # env: # - name: POD_NAME # valueFrom: # fieldRef: # fieldPath: metadata.name # - name: POD_NAMESPACE # valueFrom: # fieldRef: # fieldPath: metadata.namespace # volumeMounts: # - name: copy-portal-skins # mountPath: /srv/var/lib/lemonldap-ng/portal/skins |
2.4 修改README.md
和templates/NOTES.txt
根据 values.yaml
中的默认设置相应修改README.md
,内容使用markdown语法,这里不作详细说明。
templates/NOTES.txt
是部署chart后输出的帮助文档,里面支持go template语法。模板里已经写成了非常通用。必要情况下,适当按应用需求来修改,这样会显得部署后提示非常友好和人性化。
2.5 templates
下yaml简要说明
templates
目录下为kubernetes资源yaml文件模板,以资源名命名文件名,复杂些的可以加上资源功能或者模块名等。
templates/secret.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
{{- if .Values.secret.enabled }} # if用法。语法代码注意形成一致缩进习惯,便于阅读 apiVersion: v1 kind: Secret metadata: name: {{ template "example-chart.fullname" . }} # 使用辅助模板时,点号可用 $ 代替,特别是在range下 labels: app: {{ template "example-chart.name" . }} chart: {{ template "example-chart.chart" . }} release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" {{- if .Values.labels }} {{ toYaml .Values.labels | indent 4 }} # 使用toYaml时,不缩进,indent接空格数 {{- end }} data: {{- range $key, $value := .Values.secret.data }} # range用法 {{ $key }}: {{ $value | b64enc | quote }} # secret中value注意使用base64转换 {{- end }} {{- end }} |
templates/deployment-statefulset.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 |
{{- if .Values.statefulset.enabled }} # 判断使用deployment和statefulset资源api类型 apiVersion: apps/v1 kind: StatefulSet {{- else }} apiVersion: apps/v1 kind: Deployment {{- end }} metadata: name: {{ template "example-chart.fullname" . }} labels: app: {{ template "example-chart.name" . }} chart: {{ template "example-chart.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} {{- if .Values.labels }} # 额外的标签 {{ toYaml .Values.labels | indent 4 }} {{- end }} {{- if .Values.annotations }} # 自定义注释 annotations: {{ toYaml .Values.annotations | indent 4 }} {{- end }} spec: replicas: {{ .Values.replicaCount }} # 副本数 {{- if .Values.statefulset.enabled }} # statefulset需要定义serviceName serviceName: {{ template "example-chart.fullname" . }}-headless {{- end }} {{- if .Values.deploymentStrategy }} strategy: {{ toYaml .Values.deploymentStrategy | indent 4 }} {{- end }} selector: matchLabels: app: {{ template "example-chart.name" . }} release: {{ .Release.Name }} template: metadata: annotations: {{- if .Values.podAnnotations }} {{ toYaml .Values.podAnnotations | indent 8 }} {{- end }} {{- if .Values.metrics.podAnnotations }} {{ toYaml .Values.metrics.podAnnotations | indent 8 }} {{- end }} labels: app: {{ template "example-chart.name" . }} release: {{ .Release.Name }} spec: {{- include "example-chart.imagePullSecrets" . | indent 6 }} {{- if .Values.initContainers }} initContainers: {{ toYaml .Values.initContainers | indent 8 }} {{- end }} nodeSelector: {{ toYaml .Values.nodeSelector | indent 8 }} affinity: {{ toYaml .Values.affinity | indent 8 }} tolerations: {{ toYaml .Values.tolerations | indent 8 }} containers: {{- if .Values.metrics.enabled }} # metrics容器可根据需求修改 - name: metrics image: {{ template "metrics.image" . }} imagePullPolicy: {{ .Values.metrics.image.pullPolicy | quote }} command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status'] ports: - name: metrics containerPort: 9113 livenessProbe: httpGet: path: /metrics port: metrics initialDelaySeconds: 15 timeoutSeconds: 5 readinessProbe: httpGet: path: /metrics port: metrics initialDelaySeconds: 5 timeoutSeconds: 1 resources: {{ toYaml .Values.metrics.resources | indent 12 }} {{- end }} - name: {{ .Chart.Name }} image: {{ template "example-chart.image" . }} imagePullPolicy: {{ .Values.image.pullPolicy | quote }} {{- if .Values.lifecycle }} lifecycle: {{ toYaml .Values.lifecycle | indent 12 }} {{- end }} {{- if .Values.startCommand }} command: {{ toYaml .Values.startCommand |indent 12 }} {{- end }} env: {{ toYaml .Values.env | indent 12 }} resources: {{ toYaml .Values.resources | indent 12 }} ports: {{- range $key, $value := .Values.service.ports }} - name: {{ $key }} containerPort: {{ $value.port }} protocol: {{ $value.protocol }} {{- end }} {{- if .Values.healthCheck.enabled }} livenessProbe: {{- if eq .Values.healthCheck.type "http" }} httpGet: path: {{ .Values.healthCheck.httpPath }} port: {{ .Values.healthCheck.port }} {{- else }} tcpSocket: port: {{ .Values.healthCheck.port }} {{- end }} initialDelaySeconds: {{ .Values.healthCheck.livenessInitialDelaySeconds }} periodSeconds: {{ .Values.healthCheck.livenessPeriodSeconds }} readinessProbe: {{- if eq .Values.healthCheck.type "http" }} httpGet: path: {{ .Values.healthCheck.httpPath }} port: {{ .Values.healthCheck.port }} {{- else }} tcpSocket: port: {{ .Values.healthCheck.port }} {{- end }} initialDelaySeconds: {{ .Values.healthCheck.readinessInitialDelaySeconds }} periodSeconds: {{ .Values.healthCheck.readinessPeriodSeconds }} {{- end }} volumeMounts: # 容器挂载点 {{- if .Values.config.enabled }} - name: {{ template "example-chart.name" . }}-conf mountPath: {{ .Values.config.mountPath }} subPath: {{ .Values.config.subPath }} {{- end }} {{- if .Values.secret.enabled }} - name: {{ template "example-chart.name" . }}-secret mountPath: {{ .Values.secret.mountPath }} subPath: {{ .Values.secret.subPath }} readOnly: {{ .Values.secret.readOnly }} {{- end }} {{- if .Values.persistentVolume.mountPaths }} {{ toYaml .Values.persistentVolume.mountPaths | indent 12 }} {{- end }} {{- if .Values.additionalContainers }} {{ toYaml .Values.additionalContainers | indent 8 }} {{- end }} volumes: # volume名需要和上文volumeMounts中的名字一一对应 {{- if .Values.config.enabled }} - name: {{ template "example-chart.name" . }}-conf configMap: name: {{ template "example-chart.fullname" . }} {{- end }} {{- if .Values.secret.enabled }} - name: {{ template "example-chart.name" . }}-secret secret: secretName: {{ template "example-chart.fullname" . }} {{- end }} {{- if .Values.deployment.additionalVolumes }} {{ toYaml .Values.deployment.additionalVolumes | indent 8 }} {{- end }} {{- if not .Values.statefulset.enabled }} {{- if .Values.persistentVolume.enabled }} - name: data-storage persistentVolumeClaim: claimName: {{ .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }} {{- else }} - name: data-storage emptyDir: {} {{- end }} {{- else }} {{- if .Values.persistentVolume.enabled }} volumeClaimTemplates: - metadata: name: data-storage labels: app: {{ template "example-chart.name" . }} chart: {{ template "example-chart.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} spec: accessModes: - {{ .Values.persistentVolume.accessMode | quote }} annotations: {{- range $key, $value := $.Values.persistentVolume.annotations }} {{ $key }}: {{ $value }} {{- end }} resources: requests: storage: {{ .Values.persistentVolume.size }} {{- if .Values.persistentVolume.storageClass }} {{- if (eq "-" .Values.persistentVolume.storageClass) }} storageClassName: "" {{- else }} storageClassName: "{{ .Values.persistentVolume.storageClass }}" {{- end }} {{- end }} {{- else }} - name: data-storage emptyDir: {} {{- end }} {{- end -}} |
3. 小结
以上yaml中未作详细说明,仔细看其内容都能明白大致意思。以下是对于helm chart新手的一些建议:
- 刚接触helm chart时,多模仿stable和bianami中charts的写法,特别是
values.yaml
和templates
目录中的一些设计,差不多都已经非常统一了。这样遇到自己有相似需求,可直接使用相应的功能块,写出来的chart也显得非常专业。 - 使用chart时,最好使用
helm
命令fetch
下来,大概读一遍其chart内容,这样看多了,自然就越来越熟悉,而且出错时,也便于自己排查问题。
参考资料:
[1] https://helm.sh/
[2] https://whmzsu.github.io/helm-doc-zh-cn/
微信扫描下方的二维码阅读本文