当前位置： 首页 > news >正文

K8S cluster with multi-masters on Azure VM

news 2025/7/21 16:11:47

拓扑参考：

在这里插入图片描述

在 Azure VM 实例上部署 KubeSphere

  • 基础模板
    需要修改 IP 地址和 VM Image的可以在模板中修改。

    {"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#","contentVersion": "1.0.0.0","parameters": {"vmNamePrefix": {"defaultValue": "master-","type": "String","metadata": {"description": "The name of your VM master node."}},"vmssName": {"defaultValue": "node","type": "String","metadata": {"description": "The name of your VMSS cluster."}},"location": {"defaultValue": "[resourceGroup().location]","type": "String","metadata": {"description": "Location for all resources."}},"adminUsername": {"type": "String","metadata": {"description": "Username for the Virtual Machine."}},"adminKey": {"type": "SecureString","metadata": {"description": "SSH Key for the Virtual Machine."}},"defaultMasterCount": {"defaultValue": 3,"type": "Int","metadata": {"description": "The default instances count of master"}},"defaultNodeCount": {"defaultValue": 3,"type": "Int","metadata": {"description": "The initial node size of your VMSS cluster."}},"minNodeCount": {"defaultValue": 1,"type": "Int","metadata": {"description": "The min node size of your VMSS cluster."}},"maxNodeCount": {"defaultValue": 20,"type": "Int","metadata": {"description": "The max node size of your VMSS cluster."}},"dnsLabelPrefix": {"defaultValue": "[toLower(concat('k8s-cluster-', uniqueString(resourceGroup().id)))]","type": "String","metadata": {"description": "Unique DNS Name for the Public IP used to access the Virtual Machine."}},"vmSize": {"defaultValue": "Standard_DS2_v2","type": "String","metadata": {"description": "The size of the VM"}},"virtualNetworkName": {"defaultValue": "vNetwork","type": "String","metadata": {"description": "Name of the Virtual Network"}},"subnetName": {"defaultValue": "Subnet","type": "String","metadata": {"description": "Name of the subnet in the virtual network"}},"vmssSubnetName": {"defaultValue": "nodeSubnet","type": "String","metadata": {"description": "Name of the VMSS subnet in the virtual network"}},"publicLBName": {"defaultValue": "publicLB","type": "String","metadata": {"description": "Internal Load Balancer name"}}},"variables": {"publicIPAddressName": "[concat(parameters('publicLBName'), 'IP' )]","availabilitySetName": "masterAvSet","networkInterfaceName": "[concat(parameters('vmNamePrefix'),'Interface')]","networkSecurityGroupName": "[concat(parameters('virtualNetworkName'),'nsg-default')]","subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworkName'), parameters('subnetName'))]","vmssSubnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworkName'), parameters('vmssSubnetName'))]","osDiskType": "Standard_LRS","publicLBID": "[resourceId('Microsoft.Network/loadBalancers',parameters('publicLBName'))]"},"resources": [{"type": "Microsoft.Network/networkSecurityGroups","apiVersion": "2015-06-15","name": "[variables('networkSecurityGroupName')]","location": "[parameters('location')]","properties": {"securityRules": [{"name": "Port_SSH","properties": {"description": "SSH","protocol": "*","sourcePortRange": "*","destinationPortRange": "22","sourceAddressPrefix": "*","destinationAddressPrefix": "*","access": "Allow","priority": 100,"direction": "Inbound","sourcePortRanges": [],"destinationPortRanges": [],"sourceAddressPrefixes": [],"destinationAddressPrefixes": []}},{"name": "Port_API_Server","properties": {"description": "k8s API Server","protocol": "TCP","sourcePortRange": "*","destinationPortRange": "6443","sourceAddressPrefix": "*","destinationAddressPrefix": "*","access": "Allow","priority": 140,"direction": "Inbound","sourcePortRanges": [],"destinationPortRanges": [],"sourceAddressPrefixes": [],"destinationAddressPrefixes": []}},{"name": "Port_Http","properties": {"description": "Web","protocol": "TCP","sourcePortRange": "*","destinationPortRange": "80","sourceAddressPrefix": "*","destinationAddressPrefix": "*","access": "Allow","priority": 120,"direction": "Inbound","sourcePortRanges": [],"destinationPortRanges": [],"sourceAddressPrefixes": [],"destinationAddressPrefixes": []}},{"name": "Port_Https","properties": {"protocol": "TCP","sourcePortRange": "*","destinationPortRange": "443","sourceAddressPrefix": "*","destinationAddressPrefix": "*","access": "Allow","priority": 130,"direction": "Inbound","sourcePortRanges": [],"destinationPortRanges": [],"sourceAddressPrefixes": [],"destinationAddressPrefixes": []}}]}},{"type": "Microsoft.Network/virtualNetworks","apiVersion": "2019-11-01","name": "[parameters('virtualNetworkName')]","location": "[parameters('location')]","dependsOn": ["[resourceId('Microsoft.Network/networkSecurityGroups/', variables('networkSecurityGroupName'))]"],"properties": {"addressSpace": {"addressPrefixes": ["10.211.0.0/16"]},"subnets": [{"name": "[parameters('vmssSubnetName')]","properties": {"networkSecurityGroup": {"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"},"addressPrefix": "10.211.0.0/24"}},{"name": "[parameters('subnetName')]","properties": {"networkSecurityGroup": {"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"},"addressPrefix": "10.211.1.0/24"}}]}},{"type": "Microsoft.Network/networkInterfaces","apiVersion": "2019-11-01","name": "[concat(variables('networkInterfaceName'), copyindex())]","location": "[parameters('location')]","dependsOn": ["[resourceId('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]","[resourceId('Microsoft.Network/loadBalancers/', parameters('publicLBName'))]"],"properties": {"ipConfigurations": [{"name": "ipconfig1","properties": {"subnet": {"id": "[variables('subnetRef')]"},"loadBalancerBackendAddressPools": [{"id": "[concat(variables('publicLBID'), '/backendAddressPools/BackendPoolMaster')]"}],"loadBalancerInboundNatRules": [{"id": "[concat(variables('publicLBID'), '/inboundNatRules/lbNAT-master',copyindex())]"}],"privateIPAllocationMethod": "Dynamic"}}]},"copy": {"name": "nicLoop","count": "[parameters('defaultMasterCount')]"}},{"type": "Microsoft.Network/publicIPAddresses","apiVersion": "2019-09-01","name": "[variables('publicIPAddressName')]","location": "[parameters('location')]","sku": {"name": "Standard"},"properties": {"publicIPAllocationMethod": "Static","publicIPAddressVersion": "IPv4","dnsSettings": {"domainNameLabel": "[parameters('dnsLabelPrefix')]"},"idleTimeoutInMinutes": 10}},{"type": "Microsoft.Network/loadBalancers","apiVersion": "2018-06-01","name": "[parameters('publicLBName')]","location": "[parameters('location')]","dependsOn": ["[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]"],"sku": {"name": "Standard"},"properties": {"frontendIPConfigurations": [{"name": "LoadBalancerFrontEnd","properties": {"publicIPAddress": {"id": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]"}}}],"backendAddressPools": [{"name": "BackendPoolNode"},{"name": "BackendPoolMaster"}],"loadBalancingRules": [{"name": "HttpLBRule","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"backendAddressPool": {"id": "[concat(variables('publicLBID'),'/backendAddressPools/BackendPoolNode')]"},"protocol": "Tcp","frontendPort": 80,"backendPort": 80,"enableFloatingIP": false,"disableOutboundSnat": false,"idleTimeoutInMinutes": 5,"probe": {"id": "[concat(variables('publicLBID'),'/probes/tcpProbe')]"}}},{"name": "APILBRule","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"backendAddressPool": {"id": "[concat(variables('publicLBID'),'/backendAddressPools/BackendPoolMaster')]"},"protocol": "Tcp","frontendPort": 6443,"backendPort": 6443,"enableFloatingIP": false,"disableOutboundSnat": false,"idleTimeoutInMinutes": 5,"probe": {"id": "[concat(variables('publicLBID'),'/probes/apitcpProbe')]"}}}],"probes": [{"name": "tcpProbe","properties": {"protocol": "Tcp","port": 80,"intervalInSeconds": 5,"numberOfProbes": 2}},{"name": "apitcpProbe","properties": {"protocol": "Tcp","port": 6443,"intervalInSeconds": 5,"numberOfProbes": 2}}],"inboundNatRules": [{"name": "lbNAT-master0","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"protocol": "Tcp","frontendPort": "50200","backendPort": "22"}},{"name": "lbNAT-master1","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"protocol": "Tcp","frontendPort": "50201","backendPort": "22"}},{"name": "lbNAT-master2","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"protocol": "Tcp","frontendPort": "50202","backendPort": "22"}}],"inboundNatPools": [{"name": "lbNAT-node","properties": {"frontendIPConfiguration": {"id": "[concat(variables('publicLBID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"},"protocol": "Tcp","frontendPortRangeStart": 50100,"frontendPortRangeEnd": 50199,"backendPort": 22}}]}},{"type": "Microsoft.Compute/availabilitySets","apiVersion": "2016-04-30-preview","name": "[variables('availabilitySetName')]","location": "[parameters('location')]","properties": {"platformFaultDomainCount": 2,"platformUpdateDomainCount": 2,"managed": true}},{"type": "Microsoft.Compute/virtualMachines","apiVersion": "2019-07-01","name": "[concat(parameters('vmNamePrefix'), copyindex())]","location": "[parameters('location')]","dependsOn": ["[concat('Microsoft.Network/networkInterfaces/', variables('networkInterfaceName'), copyindex())]","[concat('Microsoft.Compute/availabilitySets/', variables('availabilitySetName'))]","[concat('Microsoft.Compute/virtualMachineScaleSets/', parameters('vmssName'))]"],"properties": {"availabilitySet": {"id": "[resourceId('Microsoft.Compute/availabilitySets',variables('availabilitySetName'))]"},"hardwareProfile": {"vmSize": "[parameters('vmSize')]"},"storageProfile": {"osDisk": {"createOption": "FromImage","managedDisk": {"storageAccountType": "[variables('osDiskType')]"}},"imageReference": {"publisher": "Canonical","offer": "0001-com-ubuntu-server-focal","sku": "20_04-lts-gen2","version": "latest"}},"networkProfile": {"networkInterfaces": [{"id": "[resourceId('Microsoft.Network/networkInterfaces',concat(variables('networkInterfaceName'),copyindex()))]"}]},"osProfile": {"computerName": "[concat(parameters('vmNamePrefix'), copyindex())]","adminUsername": "[parameters('adminUsername')]","adminPassword": "[parameters('adminKey')]","linuxConfiguration": {"disablePasswordAuthentication": true,"ssh": {"publicKeys": [{"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]","keyData": "[parameters('adminKey')]"}]}}}},"copy": {"name": "virtualMachineLoop","count": "[parameters('defaultMasterCount')]"}},{"type": "Microsoft.Compute/virtualMachineScaleSets","apiVersion": "2019-07-01","name": "[parameters('vmssName')]","location": "[parameters('location')]","dependsOn": ["[concat('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]","[concat('Microsoft.Network/loadBalancers/', parameters('publicLBName'))]"],"tags": {"cluster-autoscaler-enabled": "true","cluster-autoscaler-name": "[resourceGroup().name]","min": "[parameters('minNodeCount')]","max": "[parameters('maxNodeCount')]","poolName": "[parameters('vmssName')]"},"sku": {"name": "[parameters('vmSize')]","tier": "Standard","capacity": "[parameters('defaultNodeCount')]"},"properties": {"overprovision": false,"upgradePolicy": {"mode": "Manual"},"virtualMachineProfile": {"storageProfile": {"osDisk": {"createOption": "FromImage","caching": "ReadWrite"},"imageReference": {"publisher": "Canonical","offer": "0001-com-ubuntu-server-focal","sku": "20_04-lts-gen2","version": "latest"}},"osProfile": {"computerNamePrefix": "[parameters('vmssName')]","adminUsername": "[parameters('adminUsername')]","adminPassword": "[parameters('adminKey')]","linuxConfiguration": {"disablePasswordAuthentication": true,"ssh": {"publicKeys": [{"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]","keyData": "[parameters('adminKey')]"}]}}},"networkProfile": {"networkInterfaceConfigurations": [{"name": "[concat(parameters('vmssName'),'nic')]","properties": {"primary": true,"ipConfigurations": [{"name": "[concat('ipconfigVmss', parameters('vmssName'))]","properties": {"subnet": {"id": "[variables('vmssSubnetRef')]"},"loadBalancerBackendAddressPools": [{"id": "[concat(variables('publicLBID'), '/backendAddressPools/BackendPoolNode')]"}],"loadBalancerInboundNatPools": [{"id": "[concat(variables('publicLBID'), '/inboundNatPools/lbNAT-node')]"}]}}]}}]}}}}],"outputs": {"adminUsername": {"type": "String","value": "[parameters('adminUsername')]"},"hostname": {"type": "String","value": "[reference(variables('publicIPAddressName')).dnsSettings.fqdn]"},"sshCommand": {"type": "String","value": "[concat('ssh ', parameters('adminUsername'), '@', reference(variables('publicIPAddressName')).dnsSettings.fqdn)]"}}
}

    • 可以修改 master 和 node 的名字前缀、部署区域、数量和 VM 类型

      在这里插入图片描述

      • ssh 22 对外由 LB 配置 NAT 端口实现，如配置文件中 50200 → master-0

        • 已经包含的规则转换（不含 30880）

          服务协议规则后端端口前端端口节点池
          sshTCP入站 NAT2250200, 50201, 50202, 50100~50199主节点, 普通节点
          api 服务器TCP负载均衡64436443主节点
          ks 控制台TCP负载均衡3088030880主节点
          httpTCP负载均衡8080普通节点
          httpsTCP负载均衡443443普通节点
      • node 使用 VMSS

  • 部署 K8S cluster
    最简单的方式还是用 kk 完成，注意在 kubernetes 1.24 以后，psp 弃用，在 kk 中还有 psp 权限管理，安装的时候会报错。建议使用 1.23.10，也是 kk 现在默认的版本。

    • 证书传输 → Master-0

      scp  -i zyi.pem -P 50200 zyi.pem zyi@20.247.0.170:/home/zyi/.ssh/

    • 每一台安装 socat 等必须的软件

      ssh -i zyi.pem -p50200 zyi@20.247.0.170 'sudo apt install socat conntrack'

    • 登录到 Mater-0

      ssh -i zyi.pem -p50200 zyi@20.247.0.170

    • 下载 kk 并赋予可执行权限

    curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.10 sh -chmod +x kk

    • 创建配置文件模板

      ./kk create config --with-kubesphere v3.3.2 --with-kubernetes v1.22.12

      • KubeSphere 3.3 对应 Kubernetes 版本推荐：v1.20.x、v1.21.x、* v1.22.x、* v1.23.x 和 * v1.24.x。带星号的版本可能出现边缘节点部分功能不可用的情况。因此，如需使用边缘节点，推荐安装 v1.21.x。如果未指定 Kubernetes 版本，KubeKey 将默认安装 Kubernetes v1.23.10。有关支持的 Kubernetes 版本请参阅支持矩阵。

      • 如果在此步骤中的命令中未添加标志 -with-kubesphere，则不会部署 KubeSphere，除非您使用配置文件中的 addons 字段进行安装，或稍后使用 ./kk create cluster 时再次添加此标志。

      • 如果在未指定 KubeSphere 版本的情况下添加标志 --with kubesphere`，将安装 KubeSphere 的最新版本。

      • 修改的内容用红色标注

        apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:name: kubeCluster
spec:hosts:- {name: master-0, address: 20.210.0.156, port: 50200, internalAddress: 10.211.1.5, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}- {name: master-1, address: 20.210.0.156, port: 50201, internalAddress: 10.211.1.6, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}- {name: master-2, address: 20.210.0.156, port: 50202, internalAddress: 10.211.1.4, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}- {name: node000000, address: 20.210.0.156, port: 50100, internalAddress: 10.211.0.4, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}- {name: node000001, address: 20.210.0.156, port: 50101, internalAddress: 10.211.0.5, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}- {name: node000002, address: 20.210.0.156, port: 50102, internalAddress: 10.211.0.6, user: zyi, privateKeyPath: "~/.ssh/zyi.pem"}roleGroups:etcd:- master-0- master-1- master-2control-plane:- master-0- master-1- master-2worker:- node000000- node000001- node000002controlPlaneEndpoint:## Internal loadbalancer for apiservers# internalLoadbalancer: haproxydomain: lb.etaon.labaddress: "20.210.0.156"port: 6443kubernetes:version: v1.23.10clusterName: cluster.localautoRenewCerts: truecontainerManager: dockeretcd:type: kubekeynetwork:plugin: flannelkubePodsCIDR: 10.233.64.0/18kubeServiceCIDR: 10.233.0.0/18## multus support. https://github.com/k8snetworkplumbingwg/multus-cnimultusCNI:enabled: falseregistry:privateRegistry: ""namespaceOverride: ""registryMirrors: []insecureRegistries: []addons: []---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:name: ks-installernamespace: kubesphere-systemlabels:version: v3.3.2
spec:persistence:storageClass: ""authentication:jwtSecret: ""zone: ""local_registry: ""namespace_override: ""# dev_tag: ""etcd:monitoring: falseendpointIps: localhostport: 2379tlsEnable: truecommon:core:console:enableMultiLogin: trueport: 30880type: NodePort# apiserver:#  resources: {}# controllerManager:#  resources: {}redis:enabled: falsevolumeSize: 2Giopenldap:enabled: falsevolumeSize: 2Giminio:volumeSize: 20Gimonitoring:# type: externalendpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090GPUMonitoring:enabled: falsegpu:kinds:- resourceName: "nvidia.com/gpu"resourceType: "GPU"default: truees:# master:#   volumeSize: 4Gi#   replicas: 1#   resources: {}# data:#   volumeSize: 20Gi#   replicas: 1#   resources: {}logMaxAge: 7elkPrefix: logstashbasicAuth:enabled: falseusername: ""password: ""externalElasticsearchHost: ""externalElasticsearchPort: ""alerting:enabled: false# thanosruler:#   replicas: 1#   resources: {}auditing:enabled: false# operator:#   resources: {}# webhook:#   resources: {}devops:enabled: false# resources: {}jenkinsMemoryLim: 8GijenkinsMemoryReq: 4GijenkinsVolumeSize: 8Gievents:enabled: false# operator:#   resources: {}# exporter:#   resources: {}# ruler:#   enabled: true#   replicas: 2#   resources: {}logging:enabled: falselogsidecar:enabled: truereplicas: 2# resources: {}metrics_server:enabled: falsemonitoring:storageClass: ""node_exporter:port: 9100# resources: {}# kube_rbac_proxy:#   resources: {}# kube_state_metrics:#   resources: {}# prometheus:#   replicas: 1#   volumeSize: 20Gi#   resources: {}#   operator:#     resources: {}# alertmanager:#   replicas: 1#   resources: {}# notification_manager:#   resources: {}#   operator:#     resources: {}#   proxy:#     resources: {}gpu:nvidia_dcgm_exporter:enabled: false# resources: {}multicluster:clusterRole: nonenetwork:networkpolicy:enabled: falseippool:type: nonetopology:type: noneopenpitrix:store:enabled: falseservicemesh:enabled: falseistio:components:ingressGateways:- name: istio-ingressgatewayenabled: falsecni:enabled: falseedgeruntime:enabled: falsekubeedge:enabled: falsecloudCore:cloudHub:advertiseAddress:- ""service:cloudhubNodePort: "30000"cloudhubQuicNodePort: "30001"cloudhubHttpsNodePort: "30002"cloudstreamNodePort: "30003"tunnelNodePort: "30004"# resources: {}# hostNetWork: falseiptables-manager:enabled: truemode: "external"# resources: {}# edgeService:#   resources: {}terminal:timeout: 600

    • 安装部署

      ./kk create cluster -f config-sample.yaml
      • 安装日志
      kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

      #####################################################
###              Welcome to KubeSphere!           ###
#####################################################Console: http://10.211.1.5:30880
Account: admin
Password: P@88w0rd
NOTES：1. After you log into the console, please check themonitoring status of service components in"Cluster Management". If any service is notready, please wait patiently until all components are up and running.2. Please change the default password after login.#####################################################
https://kubesphere.io             2023-08-11 03:31:56
#####################################################

    • 在 LBer 上为 30880端口 配置规则并在 ASG 上 permit

      在这里插入图片描述

      在这里插入图片描述

    • http://hostip：30880

      在这里插入图片描述

  • 测试

    apiVersion: v1
kind: Service
metadata:name: hello-kubernetes
spec:type: NodePortports:- port: 80targetPort: 8080selector:app: hello-kubernetes
---
apiVersion: apps/v1
kind: Deployment
metadata:name: hello-kubernetes
spec:replicas: 3selector:matchLabels:app: hello-kubernetestemplate:metadata:labels:app: hello-kubernetesspec:containers:- name: hello-kubernetesimage: paulbouwer/hello-kubernetes:1.5ports:- containerPort: 8080env:- name: MESSAGEvalue: I just deployed a PodVM on the Azure VM Cluster!!

    • 配置外部访问30596 → 80

      kubectl get svc
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
hello-kubernetes   NodePort    10.233.31.158   <none>        80:30596/TCP   167m

    • 负载均衡上配置

      在这里插入图片描述

    • 访问前端 公网 IP 或 DNS 名称

      在这里插入图片描述

      在这里插入图片描述

http://www.lryc.cn/news/140037.html

相关文章：

  • 初阶c语言：趣味扫雷游戏
  • JVM——内存模型
  • java八股文面试[JVM]——元空间
  • 科技云报道：云计算下半场，公有云市场生变，私有云风景独好
  • Oracle 如何给大表添加带有默认值的字段
  • 记录Taro大坑2丢失api无法启动
  • Java-Maven-解决maven deploy时报 401 Reason Phrase Unauthorized 错误
  • 【数据结构】 栈（Stack）的应用场景
  • 人力资源小程序的设计原则与实现方法
  • 检查Javascript对象数组中是否存在对象值，如果没有向数组添加新对象
  • UG\NX二次开发 使用录制功能录制操作记录时，如何设置默认的开发语言？
  • 【业务功能篇83】微服务SpringCloud-ElasticSearch-Kibanan-docke安装-应用层实战
  • VBJSON报错：缺少：语句结束
  • Docker安装ES+kibana8.9.1
  • 12. Oracle中case when详解
  • 【电路设计】220V AC转低压DC电路概述
  • 网络地址转换NAT-动态NAT的使用范围和配置-思科EI,华为数通
  • 远程连接虚拟机中ubuntu报错：Network error：Connection refused
  • 快速排序三种思路详解！
  • 【二叉树入门指南】链式结构的实现
  • 【位运算】算法实战
  • C++构建系统
  • “深入探索JVM内部机制：理解Java虚拟机的运行原理“
  • java八股文面试[JVM]——双亲委派模型
  • NLP与大模型主题全国师资培训班落地，飞桨持续赋能AI人才培养
  • Jupyter Notebook 配置根目录
  • 算法 位运算
  • Linux 虚拟机常用命令
  • 解决抖音semi-ui的Input无法获取到onChange事件
  • 免费的png打包plist工具CppTextu，一款把若干资源图片拼接为一张大图的免费工具