手把手教你写一个通用的helm chart

作者: ygqygq2 分类: 系统/运维,虚拟化/容器 发布时间: 2019-07-31 23:38

[TOC]

1. 模板介绍

首先，放上此模板链接：

https://github.com/ygqygq2/charts/tree/master/mod-chart

此chart可当作POD单image的通用模板，只需要使用sed替换下chart名，并修改下README.md和NOTES.txt就可以了。下文，我通过复制此chart成example-chart来作示范说明。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@master1 mod-chart]# tree
.
├── Chart.yaml  # chart版本信息文件
├── README.md  # chart说明文件
├── templates  # kubernetes资源yaml模板
│   ├── configmap.yaml  # configmap模板
│   ├── deployment-statefulset.yaml  # deployment或statefulset模板
│   ├── _helpers.tpl  # 辅助模板和 partials
│   ├── ingress.yaml  # ingress模板
│   ├── NOTES.txt  # 部署chart后输出的帮助文档
│   ├── pvc.yaml  # pvc模板
│   ├── secret.yaml  # secret模板
│   ├── service-headless.yaml  # service headless模板
│   └── service.yaml  # service模板
└── values.yaml  # 默认设置
 
1 directory, 12 files
[root@master1 mod-chart]# helm3 lint --strict .
1 chart(s) linted, 0 chart(s) failed
 

2. 新chart制作

注：
下文中文件内容我保留，只加注释。
注释中需要修改的地方 [*] 标记为必选，[-] 标识为可选。

2.1 目录准备

将模板mod-chart复制成example-chart，并作内容替换。

1
2
3
4
5
rsync -avz mod-chart/ example-chart/
cd example-chart/
sed -i 's@mod-chart@example-chart@g' *.*
sed -i 's@mod-chart@example-chart@g' templates/*.*
 

2.2 修改Chart.yaml

vim Chart.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: v1  # 当前helm api版本，不需要修改
appVersion: 1.14.2  # 此处为你应用程序的版本号 [*]
description: Chart for the nginx server  # 介绍此chart是干嘛的，按需求修改
engine: gotpl  # go模板引擎，不需要修改 [-]
name: example-chart  # 模板名，对应目录名 [*]
version: 1.0.0  # 此chart版本号 [*]
home: http://www.nginx.org  # 应用程序官网 [*]
icon: https://bitnami.com/assets/stacks/nginx/img/nginx-stack-220x234.png  # 应用程序logo地址 [*]
keywords:  # 关键字列表 [*]
- nginx
- http
- web
- www
- reverse proxy
maintainers:  # 维护人员列表 [*]
- email: 29ygq@sina.com
  name: Chinge Yang
sources:  # 应用程序来源 [-]
- https://github.com/bitnami/bitnami-docker-nginx
 

2.3 修改values.yaml

因为values.yaml设置涉及到yaml格式，yaml文件格式说明可以看这篇文章：

http://www.ruanyifeng.com/blog/2016/07/yaml.html

这里提几个常用的地方：

  1. 使用2个空格作缩进；
  2. 确认数字为字符类型时，使用双引号引起来；
  3. 为了迎合helm3的规范，空定义最好将相关符号补上：

1
2
3
4
string: ""
list: []
map: {}
 

没什么特殊要求，一般需要修改的地方有image、service、healthCheck、persistentVolume.mountPaths

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
# Default values for mod-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
 
## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
global:  # 设置后覆盖后面默认的镜像仓库
  imageRegistry: ""
  imagePullSecrets: []
#     - myRegistryKeySecretName
 
statefulset:
  enabled: false
 
## String to partially override fullname template (will maintain the release name)
##
nameOverride: ""
 
## String to fully override fullname template
##
fullnameOverride: ""
 
## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .
## You can change type to <code>Recreate</code> or can uncomment <code>rollingUpdate</code> specification and adjust them to your usage.
deploymentStrategy: {}
  # rollingUpdate:
  #   maxSurge: 25%
  #   maxUnavailable: 25%
  # type: RollingUpdate
 
# 副本个数
replicaCount: 1
 
# 容器image及tag
image:
  registry: docker.io
  repository: bitnami/nginx
  tag: latest
  pullPolicy: IfNotPresent  # IfNotPresent: 有则不拉（减少流量和操作步骤），Always: 不管tag总拉（适合tag不变时更新）
  pullSecrets: []
  #  - private-registry-key
 
service:
  type: ClusterIP  # 一般不用修改
  ingressPort: 8080
  ports:
    http:  # 多端口暴露时，复制一段
      port: 8080  # Service port number for client-a port.
      protocol: TCP  # Service port protocol for client-a port.
 
## env set
## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
env: []
#  - name: DEMO_GREETING
#    value: "Hello from the environment"
#  - name: DEMO_FAREWELL
#    value: "Such a sweet sorrow"
 
## command set
startCommand: []
#  - "java -Xdebug -Xnoagent -Djava.compiler=NONE"
#  - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
#  - "-Djava.security.egd=file:/dev/urandom"
#  - "-jar /test.jar"
#  - "-Duser.timezone=GMT+08"
 
## Enable configmap and add data in configmap
config:
  enabled: false
  subPath: ""
  mountPath: /conf
  data: {}
 
############################# 示例 ####################################
## 以下示例，挂载文件至 /conf/app.conf
#  enabled: true
#  mountPath: /conf/app.conf  
#  subPath: app.conf    # 使用subPath时，上面mountPath路径写文件完整绝对路径
#  data:
#    app.conf: |-
#      appname = example-chart
 
## 以下示例，挂载多个文件至 /conf/ 下
#  enabled: true
#  mountPath: /conf    # 不使用subPath
#  data:
#    app.conf: |-
#      appname = example-chart
#    bpp.conf: |-
#      bppname
#
## 挂载多个文件至多个不同路径，需要相应修改 templates/deployment-statefulset.yaml
############################# 示例 ####################################
 
## To use an additional secret, set enable to true and add data
## 用法同上，不另作说明
secret:
  enabled: false
  mountPath: /etc/secret-volume
  subPath: ""
  readOnly: true
  data: {}
 
## liveness and readiness
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
healthCheck:
  enabled: true
  type: tcp  # http/tcp
  port: http  # 健康检查的端口名或端口
  httpPath: '/'  # http时必须设置
  livenessInitialDelaySeconds: 10  # 初始延迟秒数
  livenessPeriodSeconds: 10  # 检测周期，默认值10，最小为1
  readinessInitialDelaySeconds: 10  # 初始延迟秒数
  readinessPeriodSeconds: 10   # 检测周期，默认值10，最小为1
 
resources: {}
  # 容器资源设置
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi
 
## Node labels and tolerations for pod assignment
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
labels: {}
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
annotations: {}
 
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistentVolume:   # 是否存储持久化
  enabled: false
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, azure-disk on
  ##   Azure, standard on GKE, AWS & OpenStack)
  ##
  storageClass: "-"
  accessMode: ReadWriteOnce
  annotations: {}
  #   helm.sh/resource-policy: keep
  size: 1Gi  # 大小
  existingClaim: {}  # 使用已存在的pvc
  mountPaths: []
  #  - name: data-storage
  #    mountPath: /config
  #    subPath: config  # 多个路径使用同一个pvc使用subPath，用法同上面config中示例说明
  #  - name: data-storage
  #    mountPath: /data
  #    subPath: data
 
ingress:  # 是否使用nginx暴露域名或端口
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local
 
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: []
 
## Prometheus Exporter / Metrics
##
metrics:
  enabled: false
  image:
    registry: docker.io
    repository: nginx/nginx-prometheus-exporter
    tag: 0.1.0
    pullPolicy: IfNotPresent
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    pullSecrets: []
    #   - myRegistrKeySecretName
  ## Metrics exporter pod Annotation and Labels
  podAnnotations:
    # prometheus.io/scrape: "true"
    # prometheus.io/port: "9113"
    ## Metrics exporter resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
  resources: {}
 
## Uncomment and modify this to run a command after starting the core container.
## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
lifecycle: {}
  # preStop:
  #   exec:
  #     command: ["/bin/bash","/pre-stop.sh"]
  # postStart:
  #   exec:
  #     command: ["/bin/bash","/post-start.sh"]
 
## Deployment additional volumes.
deployment:
  additionalVolumes: []
 
## init containers
## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: {}
#  - name: fmp-volume-permission
#    image: busybox
#    imagePullPolicy: IfNotPresent
#    command: ['chown','-R', '200', '/extra-data']
#    volumeMounts:
#      - name: extra-data
#        mountPath: /extra-data
 
## Additional containers to be added to the core pod.
additionalContainers: {}
#  - name: my-sidecar
#    image: nginx:latest
#  - name: lemonldap-ng-controller
#    image: lemonldapng/lemonldap-ng-controller:0.2.0
#    args:
#      - /lemonldap-ng-controller
#      - --alsologtostderr
#      - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
#    env:
#      - name: POD_NAME
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.name
#      - name: POD_NAMESPACE
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.namespace
#    volumeMounts:
#    - name: copy-portal-skins
#      mountPath: /srv/var/lib/lemonldap-ng/portal/skins
 

2.4 修改README.md和templates/NOTES.txt

根据 values.yaml中的默认设置相应修改README.md，内容使用markdown语法，这里不作详细说明。
templates/NOTES.txt是部署chart后输出的帮助文档，里面支持go template语法。模板里已经写成了非常通用。必要情况下，适当按应用需求来修改，这样会显得部署后提示非常友好和人性化。

2.5 templates下yaml简要说明

templates目录下为kubernetes资源yaml文件模板，以资源名命名文件名，复杂些的可以加上资源功能或者模块名等。

templates/secret.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
{{- if .Values.secret.enabled }}  # if用法。语法代码注意形成一致缩进习惯，便于阅读
apiVersion: v1
kind: Secret
metadata:
  name: {{ template "example-chart.fullname" . }}  # 使用辅助模板时，点号可用 $ 代替，特别是在range下
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: "{{ .Release.Name }}"
    heritage: "{{ .Release.Service }}"
    {{- if .Values.labels }}
{{ toYaml .Values.labels | indent 4 }}  # 使用toYaml时，不缩进，indent接空格数
    {{- end }}
data:
{{- range $key, $value := .Values.secret.data }}  # range用法
  {{ $key }}: {{ $value | b64enc | quote }}  # secret中value注意使用base64转换
{{- end }}
{{- end }}
 

templates/deployment-statefulset.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
{{- if .Values.statefulset.enabled }}  # 判断使用deployment和statefulset资源api类型
apiVersion: apps/v1
kind: StatefulSet
{{- else }}
apiVersion: apps/v1
kind: Deployment
{{- end }}
metadata:
  name: {{ template "example-chart.fullname" . }}
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.labels }}  # 额外的标签
{{ toYaml .Values.labels | indent 4 }}
{{- end }}
{{- if .Values.annotations }}  # 自定义注释
  annotations:
{{ toYaml .Values.annotations | indent 4 }}
{{- end }}
spec:
  replicas: {{ .Values.replicaCount }}  # 副本数
  {{- if .Values.statefulset.enabled }}  # statefulset需要定义serviceName
  serviceName: {{ template "example-chart.fullname" . }}-headless
  {{- end }}
  {{- if .Values.deploymentStrategy }}
  strategy:
{{ toYaml .Values.deploymentStrategy | indent 4 }}
  {{- end }}
  selector:
    matchLabels:
      app: {{ template "example-chart.name" . }}
      release: {{ .Release.Name }}
  template:
    metadata:
      annotations:
  {{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}
{{- if .Values.metrics.podAnnotations }}
{{ toYaml .Values.metrics.podAnnotations | indent 8 }}
{{- end }}
      labels:
        app: {{ template "example-chart.name" . }}
        release: {{ .Release.Name }}
    spec:
{{- include "example-chart.imagePullSecrets" . | indent 6 }}
      {{- if .Values.initContainers }}
      initContainers:
{{ toYaml .Values.initContainers | indent 8 }}
      {{- end }}
      nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
      affinity:
{{ toYaml .Values.affinity | indent 8 }}
      tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
      containers:
{{- if .Values.metrics.enabled }}  # metrics容器可根据需求修改
        - name: metrics
          image: {{ template "metrics.image" . }}
          imagePullPolicy: {{ .Values.metrics.image.pullPolicy | quote }}
          command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status']
          ports:
          - name: metrics
            containerPort: 9113
          livenessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 15
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 5
            timeoutSeconds: 1
          resources:
{{ toYaml .Values.metrics.resources | indent 12 }}
{{- end }}
        - name: {{ .Chart.Name }}
          image: {{ template "example-chart.image" . }}
          imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
          {{- if .Values.lifecycle }}
          lifecycle:
{{ toYaml .Values.lifecycle | indent 12 }}
          {{- end }}
          {{- if .Values.startCommand }}
          command:
{{ toYaml .Values.startCommand |indent 12 }}
          {{- end }}
          env:
{{ toYaml .Values.env | indent 12 }}
          resources:
{{ toYaml .Values.resources | indent 12 }}
          ports:
            {{- range $key, $value := .Values.service.ports }}
            - name: {{ $key }}
              containerPort: {{ $value.port }}
              protocol: {{ $value.protocol }}
            {{- end }}
          {{- if .Values.healthCheck.enabled }}
          livenessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.livenessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.livenessPeriodSeconds }}
          readinessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.readinessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.readinessPeriodSeconds }}
          {{- end }}
          volumeMounts:  # 容器挂载点
            {{- if .Values.config.enabled }}
            - name: {{ template "example-chart.name" . }}-conf
              mountPath: {{ .Values.config.mountPath }}
              subPath: {{ .Values.config.subPath }}
            {{- end }}
            {{- if .Values.secret.enabled }}
            - name: {{ template "example-chart.name" . }}-secret
              mountPath: {{ .Values.secret.mountPath }}
              subPath: {{ .Values.secret.subPath }}
              readOnly: {{ .Values.secret.readOnly }}
            {{- end }}
{{- if .Values.persistentVolume.mountPaths }}
{{ toYaml .Values.persistentVolume.mountPaths | indent 12 }}
{{- end }}
        {{- if .Values.additionalContainers }}
{{ toYaml .Values.additionalContainers | indent 8 }}
        {{- end }}
      volumes:  # volume名需要和上文volumeMounts中的名字一一对应
        {{- if .Values.config.enabled }}
        - name: {{ template "example-chart.name" . }}-conf
          configMap:
            name: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.secret.enabled }}
        - name: {{ template "example-chart.name" . }}-secret
          secret:
            secretName: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.deployment.additionalVolumes }}
{{ toYaml .Values.deployment.additionalVolumes | indent 8 }}
        {{- end }}
{{- if not .Values.statefulset.enabled }}
          {{- if .Values.persistentVolume.enabled }}
        - name: data-storage
          persistentVolumeClaim:
            claimName: {{ .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }}
          {{- else }}
        - name: data-storage
          emptyDir: {}
          {{- end }}
{{- else }}
  {{- if .Values.persistentVolume.enabled }}
  volumeClaimTemplates:
  - metadata:
      name: data-storage
      labels:
        app: {{ template "example-chart.name" . }}
        chart: {{ template "example-chart.chart" . }}
        release: {{ .Release.Name }}
        heritage: {{ .Release.Service }}
    spec:
      accessModes:
        - {{ .Values.persistentVolume.accessMode | quote }}
      annotations:
      {{- range $key, $value := $.Values.persistentVolume.annotations }}
        {{ $key }}: {{ $value }}
      {{- end }}
      resources:
        requests:
          storage: {{ .Values.persistentVolume.size }}
          {{- if .Values.persistentVolume.storageClass }}
            {{- if (eq "-" .Values.persistentVolume.storageClass) }}
      storageClassName: ""
            {{- else }}
      storageClassName: "{{ .Values.persistentVolume.storageClass }}"
            {{- end }}
          {{- end }}
  {{- else }}
        - name: data-storage
          emptyDir: {}
  {{- end }}
{{- end -}}
 

3. 小结

以上yaml中未作详细说明，仔细看其内容都能明白大致意思。以下是对于helm chart新手的一些建议：

  1. 刚接触helm chart时，多模仿stable和bianami中charts的写法，特别是values.yaml和templates目录中的一些设计，差不多都已经非常统一了。这样遇到自己有相似需求，可直接使用相应的功能块，写出来的chart也显得非常专业。
  2. 使用chart时，最好使用helm命令fetch下来，大概读一遍其chart内容，这样看多了，自然就越来越熟悉，而且出错时，也便于自己排查问题。

参考资料：
[1] https://helm.sh/
[2] https://whmzsu.github.io/helm-doc-zh-cn/

如果觉得我的文章对您有用，请随意打赏。您的支持将鼓励我继续创作！

发表评论

电子邮件地址不会被公开。 必填项已用*标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据。

更多阅读
标签云
cache CDN CentOS ceph CI/CD dashboard dns docker fastcgi fastdfs gitlab GlusterFS Hash helm http ingress javascript jenkins keepalived kubeadm kubernetes Linux LVS Mycat MySQL nginx pgpool-II pipeline PostgreSQL prometheus python SaltStack squid TCP tengine windows zabbix 主从同步 代理 删除 文件系统 监控 算法 缓存 面试题