当前位置：首页 > article >正文

yarn任务筛选spark任务，判断内存/CPU使用超过限制任务

article 2025/9/15 17:03:34

yarn任务筛选spark任务，判断内存/CPU使用超过限制任务
curl -s “http://it-cdh-node01:8088/ws/v1/cluster/apps?states=RUNNING” | jq ‘.apps.app | map(select(.applicationType == “SPARK” ) | select(.allocatedMB >= 102400 or .allocatedVCores >= 50) | { id, name, allocatedMB, allocatedVCores, elapsedTime })’ >> /data03/jq/sparkjoblog

curl -s “http://it-cdh-node01:8088/ws/v1/cluster/apps?states=RUNNING” | jq ‘.apps.app | map(select(.applicationType == “SPARK” ) | select(.allocatedMB >= 102400 or .allocatedVCores >= 50) | { id, name, allocatedMB, allocatedVCores })’

curl -s “http://it-cdh-node01:8088/ws/v1/cluster/apps?states=RUNNING” | jq ‘select(.apps.app[].applicationType = “SPARK”) | .apps.app[].id’ | xargs -I {} curl -s “http://it-cdh-node01:8088/ws/v1/cluster/apps/{}” | jq ‘select(.app.allocatedMB >= 102400 or .app.allocatedVCores >= 50) | .app.id, .app.name, .app.allocatedMB, .app.allocatedVCores’

查看全文

http://www.lryc.cn/news/2379564.html