当前位置: 首页 > news >正文

[Flink] Flink On Yarn(yarn-session.sh)启动错误

在Flink上启动 yarn-session.sh时出现 The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.错误。

版本说明:

Hadoop: 3.3.4

Flink:1.17.1

问题

在Flink On Yarn上启动yarn-session.sh时出现如下错误:

ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli        [] - Error while running the Flink session.org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
​	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:437) ~[flink-dist-1.17.1.jar:1.17.1]
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:608) ~[flink-dist-1.17.1.jar:1.17.1]
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:869) ~[flink-dist-1.17.1.jar:1.17.1]
​	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_231]
​	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_231]
​	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) ~[hadoop-common-3.3.4.jar:?]
​	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist-1.17.1.jar:1.17.1]
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:869) [flink-dist-1.17.1.jar:1.17.1]
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
​	at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:338) ~[flink-dist-1.17.1.jar:1.17.1]
​	at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:567) ~[flink-dist-1.17.1.jar:1.17.1]
​	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:430) ~[flink-dist-1.17.1.jar:1.17.1]... 7 more
------------------------------------------------------------The program finished with the following exception:
org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
​	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:437)
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:608)
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:869)
​	at java.security.AccessController.doPrivileged(Native Method)
​	at javax.security.auth.Subject.doAs(Subject.java:422)
​	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
​	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
​	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:869)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
​	at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:338)
​	at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:567)
​	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:430)... 7 more

原因

在yarn-site.xml文件中配置了所有可能相关的参数,重启yarn服务,执行yarn-session.sh错误依旧:

	<property><name>yarn.containers.vcores</name><value>8</value></property><property><name>yarn.nodemanager.resource.cpu-vcores</name><value>4</value></property><property><name>yarn.scheduler.maximum-allocation-vcores</name><value>2</value></property>

在看yarn cluster上的信息时突然发现Unhealth Nodes,然后查看了具体信息:
Unhealth-report
具体原因就是磁盘使用空间占比超过了90了(yarn默认为90),则认为不健康,不健康相当于这个节点不可用,由于本地只有一个节点,所以相当于整个集群不可用,于是就出现了开头的错误信息。
Unhealth-report的具体信息

解决

根据Health-report的提示,在yarn-site.xml中添加了如下参数:

	<property><name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name><value>99</value></property>

重启yarn,再查看节点状态为正常了,再执行flink的yarn-session.sh就可以正常启动了。
问题修复后的Yarn Cluster Node状态
Flink yarn-session.sh启动成功

总结

在Flink中使用yarn-session时,如果出现yarn相关的错误,可以到Yarn的WebUI上查看可能的Unhealth-report和具体的错误信息,再根据具体信息调整配置后不断调试,直到解决问题。

http://www.lryc.cn/news/92092.html

相关文章:

  • 玩转css逐帧动画,努力成为更优质的Ikun~
  • Linux Capabilities
  • 【自制C++深度学习框架】前言
  • 【高危】泛微 e-cology9 存在任意用户登录漏洞
  • 1TB文本的实时全文检索系统搭建
  • RHCA---DO477---变量实验
  • 毕业生高频常用材料线上签,高校毕业季契约锁电子签章一站式助力
  • .ini配置文件介绍与解析库使用
  • 牛客网Linux错题七
  • 牛课刷题Day5(编程题)
  • javascript基础二十五:说说你对函数式编程的理解?优缺点?
  • 常见JavaScript加密算法、JS加密算法
  • 题解2023.6.5
  • 与声音计算研究相关的挑战赛——DCASE和L3DAS
  • 实训总结-----Scrapy爬虫
  • 前端开发职业规划指南:如何做好职业规划与发展
  • 创业第一步:如何写好商业计划书
  • 【Linux驱动】字符设备驱动相关宏 / 函数介绍(module_init、register_chrdev)
  • axios解决跨域问题
  • R语言作图——热图聚类及其聚类结果输出
  • Tomcat优化
  • 我的GIT练习TWO
  • 个人器件库整理
  • javascript——内存管理
  • Qt5.15.2安卓Android项目开发环境配置
  • 第四十三章 弹跳训练2(灵识扫描)
  • 【location对象的方法,history对象,navigator--BOM】
  • 论文笔记:Normalizing Flows for Probabilistic Modeling and Inference
  • java 异常类介绍
  • shiro 550 反序列化rce