当前位置： 首页 > news >正文

k8s部署juicefs

news 2025/7/26 9:58:55
操作系统k8smysqlminiojuicefs内核
centos8.21.19.18.0.39RELEASE.2023-12-20T01-00-02Zv0.19.04.18.0-193.el8.x86_64

本文k8s较老采用老版本的juicefs，中间件也都是部署在k8s上。测试是否能成功创建动态pvc挂在到测试pod当中并查看到数据信息。一些偏理论知识就不多说了，直接开始操作

官方演示视频：​​​​​​JuiceFS CSI Driver 简介 - Office Hours 20220629_哔哩哔哩_bilibili 

官方文档：安装 | JuiceFS Document Center

 k8s部署mysql-CSDN博客

k8s部署minio集群-CSDN博客

参考以上两篇文章将中间件依赖环境先搭建起来

准备juicefs部署文件

k8s.yaml（juicefs）

# DO NOT EDIT: generated by 'kustomize build'
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-controller-sanamespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-node-sanamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-external-node-service-role
rules:
- apiGroups:- ""resources:- podsverbs:- get- list- create- update- delete- patch- watch
- apiGroups:- ""resources:- pods/logverbs:- get
- apiGroups:- ""resources:- secretsverbs:- get- create- update- delete- patch
- apiGroups:- batchresources:- jobsverbs:- get- create- update- delete- patch
- apiGroups:- ""resources:- nodes/proxyverbs:- '*'
- apiGroups:- ""resources:- persistentvolumesverbs:- get- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-external-provisioner-role
rules:
- apiGroups:- ""resources:- persistentvolumesverbs:- get- list- watch- create- delete
- apiGroups:- ""resources:- persistentvolumeclaimsverbs:- get- list- watch- update
- apiGroups:- storage.k8s.ioresources:- storageclassesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- list- watch- create- update- patch
- apiGroups:- storage.k8s.ioresources:- csinodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- secretsverbs:- get- create- update- patch- delete
- apiGroups:- ""resources:- pods- pods/logverbs:- get- list- watch- create- update- patch- delete
- apiGroups:- batchresources:- jobsverbs:- get- create- update- patch- delete
- apiGroups:- ""resources:- endpointsverbs:- get- list- watch- create- update- patch
- apiGroups:- appsresources:- daemonsetsverbs:- get- list
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- get- watch- list- delete- update- create
- apiGroups:- ""resources:- configmapsverbs:- get- watch- list- delete- update- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-node-service-binding
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: juicefs-csi-external-node-service-role
subjects:
- kind: ServiceAccountname: juicefs-csi-node-sanamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-provisioner-binding
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: juicefs-external-provisioner-role
subjects:
- kind: ServiceAccountname: juicefs-csi-controller-sanamespace: kube-system
---
apiVersion: apps/v1
kind: StatefulSet
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-controllernamespace: kube-system
spec:replicas: 1selector:matchLabels:app: juicefs-csi-controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterserviceName: juicefs-csi-controllertemplate:metadata:labels:app: juicefs-csi-controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterspec:containers:- args:- --endpoint=$(CSI_ENDPOINT)- --logtostderr- --nodeid=$(NODE_NAME)- --leader-election- --v=5env:- name: CSI_ENDPOINTvalue: unix:///var/lib/csi/sockets/pluginproxy/csi.sock- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: JUICEFS_MOUNT_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: JUICEFS_MOUNT_PATHvalue: /var/lib/juicefs/volume- name: JUICEFS_CONFIG_PATHvalue: /var/lib/juicefs/configimage: juicedata/juicefs-csi-driver:v0.19.0livenessProbe:failureThreshold: 5httpGet:path: /healthzport: healthzinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 3name: juicefs-pluginports:- containerPort: 9909name: healthzprotocol: TCPresources:limits:cpu: 1000mmemory: 1Girequests:cpu: 100mmemory: 512MisecurityContext:capabilities:add:- SYS_ADMINprivileged: truevolumeMounts:- mountPath: /var/lib/csi/sockets/pluginproxy/name: socket-dir- mountPath: /jfsmountPropagation: Bidirectionalname: jfs-dir- mountPath: /root/.juicefsmountPropagation: Bidirectionalname: jfs-root-dir- args:- --csi-address=$(ADDRESS)- --timeout=60s- --enable-leader-election- --v=5env:- name: ADDRESSvalue: /var/lib/csi/sockets/pluginproxy/csi.sockimage: quay.io/k8scsi/csi-provisioner:v1.6.0name: csi-provisionervolumeMounts:- mountPath: /var/lib/csi/sockets/pluginproxy/name: socket-dir- args:- --csi-address=$(ADDRESS)- --health-port=$(HEALTH_PORT)env:- name: ADDRESSvalue: /csi/csi.sock- name: HEALTH_PORTvalue: "9909"image: quay.io/k8scsi/livenessprobe:v1.1.0name: liveness-probevolumeMounts:- mountPath: /csiname: socket-dirpriorityClassName: system-cluster-criticalserviceAccount: juicefs-csi-controller-satolerations:- key: CriticalAddonsOnlyoperator: Existsvolumes:- emptyDir: {}name: socket-dir- hostPath:path: /var/lib/juicefs/volumetype: DirectoryOrCreatename: jfs-dir- hostPath:path: /var/lib/juicefs/configtype: DirectoryOrCreatename: jfs-root-dirvolumeClaimTemplates: []
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app.kubernetes.io/component: nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-nodenamespace: kube-system
spec:selector:matchLabels:app: juicefs-csi-nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastertemplate:metadata:labels:app: juicefs-csi-nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterspec:containers:- args:- --endpoint=$(CSI_ENDPOINT)- --logtostderr- --nodeid=$(NODE_NAME)- --v=5- --enable-manager=trueenv:- name: CSI_ENDPOINTvalue: unix:/csi/csi.sock- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: JUICEFS_MOUNT_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: HOST_IPvalueFrom:fieldRef:fieldPath: status.hostIP- name: KUBELET_PORTvalue: "10250"- name: JUICEFS_MOUNT_PATHvalue: /var/lib/juicefs/volume- name: JUICEFS_CONFIG_PATHvalue: /var/lib/juicefs/configimage: juicedata/juicefs-csi-driver:v0.19.0lifecycle:preStop:exec:command:- /bin/sh- -c- rm /csi/csi.socklivenessProbe:failureThreshold: 5httpGet:path: /healthzport: healthzinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 3name: juicefs-pluginports:- containerPort: 9909name: healthzprotocol: TCPresources:limits:cpu: 1000mmemory: 1Girequests:cpu: 100mmemory: 512MisecurityContext:privileged: truevolumeMounts:- mountPath: /var/lib/kubeletmountPropagation: Bidirectionalname: kubelet-dir- mountPath: /csiname: plugin-dir- mountPath: /devname: device-dir- mountPath: /jfsmountPropagation: Bidirectionalname: jfs-dir- mountPath: /root/.juicefsmountPropagation: Bidirectionalname: jfs-root-dir- args:- --csi-address=$(ADDRESS)- --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)- --v=5env:- name: ADDRESSvalue: /csi/csi.sock- name: DRIVER_REG_SOCK_PATHvalue: /var/lib/kubelet/csi-plugins/csi.juicefs.com/csi.sockimage: quay.io/k8scsi/csi-node-driver-registrar:v2.1.0name: node-driver-registrarvolumeMounts:- mountPath: /csiname: plugin-dir- mountPath: /registrationname: registration-dir- args:- --csi-address=$(ADDRESS)- --health-port=$(HEALTH_PORT)env:- name: ADDRESSvalue: /csi/csi.sock- name: HEALTH_PORTvalue: "9909"image: quay.io/k8scsi/livenessprobe:v1.1.0name: liveness-probevolumeMounts:- mountPath: /csiname: plugin-dirdnsPolicy: ClusterFirstWithHostNetpriorityClassName: system-node-criticalserviceAccount: juicefs-csi-node-satolerations:- key: CriticalAddonsOnlyoperator: Existsvolumes:- hostPath:path: /var/lib/kubelettype: Directoryname: kubelet-dir- hostPath:path: /var/lib/kubelet/csi-plugins/csi.juicefs.com/type: DirectoryOrCreatename: plugin-dir- hostPath:path: /var/lib/kubelet/plugins_registry/type: Directoryname: registration-dir- hostPath:path: /devtype: Directoryname: device-dir- hostPath:path: /var/lib/juicefs/volumetype: DirectoryOrCreatename: jfs-dir- hostPath:path: /var/lib/juicefs/configtype: DirectoryOrCreatename: jfs-root-dir
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: csi.juicefs.com
spec:attachRequired: falsepodInfoOnMount: false

secret.yaml（存储数据秘钥信息）

apiVersion: v1
kind: Secret
metadata:name: juicefs-secretnamespace: defaultlabels:# 增加该标签以启用认证信息校验juicefs.com/validate-secret: "true"
type: Opaque
stringData:name: data metaurl: mysql://root:PR38Ke2sP@(mysql.juicefs.svc.cluster.local:3306)/juicefs storage: minio bucket: http://minio-service.juicefs.svc.cluster.local:9000/juicefsaccess-key: adminsecret-key: 0oO1iIlLq9g#MINIO_ACCESS_KEY: admin #MINIO_SECRET_KEY: 0oO1iIlLq9g

#这里minio用来做数据存储，需要注意的是9000/juicefs这里是你的桶实现创建好，name: data会自行拼接到juicefs后面。也需要提前创建data出来

 storageclasses.yaml（根据秘钥信息创建的sc）

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: juicefs-sc
provisioner: csi.juicefs.com
parameters:csi.storage.k8s.io/provisioner-secret-name: juicefs-secretcsi.storage.k8s.io/provisioner-secret-namespace: defaultcsi.storage.k8s.io/node-publish-secret-name: juicefs-secretcsi.storage.k8s.io/node-publish-secret-namespace: default
reclaimPolicy: Retain

pvc-test.yaml（测试pvc挂载pod）

apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: juicefs-pvc#namespace: juicefs 
spec:accessModes:- ReadWriteManyresources:requests:# 从 StorageClass 中申请 10GiB 存储容量storage: 3GistorageClassName: juicefs-sc
---
apiVersion: v1
kind: Pod
metadata:name: busybox#namespace: juicefs
spec:containers:- name: busyboximage: gcr.io/google-containers/busybox:latestimagePullPolicy: IfNotPresentcommand: ["sh", "-c", "tail -f /dev/null"]tty: true  # 启用TTY以便可以连接到容器volumeMounts:  # 确保此行与 'containers' 下的其他字段对齐- mountPath: /optname: juicefs-pvmountPropagation: HostToContainerrestartPolicy: Never  # 设置为 Never 以便测试完成后不会自动重启volumes:  # 确保此行与 'containers' 对齐- name: juicefs-pvpersistentVolumeClaim:claimName: juicefs-pvc

测试步骤 

apply k8s.yaml 后会得到以下pod

当juicefs-csi相关pod成功拉起以后需要生成基于mysql与minio的secret。这个secret是用来给storageclasses使用的，让它能够调用csi去mount文件创建挂载点。

以上配置创建好以后apply pvc-test来测试动态创建pvc效果，发现会启用一个这种pod。是一个挂载点pod。

发现测试pod也成功创建了pvc也创建了并且绑定了busyboxpod

感兴趣可以看一下minio跟mysql数据存储的状态。这里就不截图演示了。

官方文档给的样例模板：创建和使用 PV | JuiceFS Document Center

#juicefs相关镜像
juicedata/juicefs-csi-driver:v0.19.0
quay.io/k8scsi/csi-node-driver-registrar:v2.1.0
quay.io/k8scsi/csi-provisioner:v1.6.0
quay.io/k8scsi/livenessprobe:v1.1.0
juicedata/mount:ce-v1.0.4

http://www.lryc.cn/news/514139.html

相关文章：

  • 【ArcGIS微课1000例】0136：制作千层饼（DEM、影像、等高线、山体阴影图层）
  • Ajax数据爬取
  • 快速上手大模型的对话生成
  • DateTimeExtensions：一个轻量C#的开源DateTime扩展方法库
  • 题解：监控屏幕调整问题
  • C语言----指针
  • 树莓派之旅-在wsl-x86-64 上进行树莓派的交叉编译
  • nature reviews genetics | 需要更多的针对不同种族的癌症基因组图谱研究，促进精准治疗和维护治疗公平权益
  • 代码随想录算法训练营day18
  • Kafka安全优化文档：漏洞修复到安全加固
  • Markdown如何添加任务列表-复选框的添加
  • 基于下垂控制的构网变换器功率控制【微电网变流器】【Simulink】
  • AI定义汽车/跨域融合/整车智能，汽车智能化2.0时代新机会来了
  • （leetcode算法题）10. 正则表达式匹配
  • SpringCloudAlibaba实战入门之Sentinel服务降级和服务熔断（十五）
  • 使用爬虫技术获取网页中的半结构化数据
  • 2025/1/1 路由期末复习作业二
  • OpenCV-Python实战（13）——图像轮廓
  • javascript变量
  • 在K8S中，如何查看kubelet组件的日志？
  • android studio android sdk下载地址
  • Fetch处理大模型流式数据请求与解析
  • FPGA自学之路：到底有多崎岖？
  • 从0到机器视觉工程师（二）:封装调用静态库和动态库
  • [极客大挑战 2019]Knife1
  • 【在Python中生成随机字符串】
  • 【three.js】场景搭建
  • Singleton: WebRTC中ThreadManager中的单例模式
  • MySQL数据库笔记——多版本并发控制MVCC
  • 【0x0037】HCI_Write_Link_Supervision_Timeout命令详解