当前位置：首页 > news >正文

bazel远程构建(Remote Execution) -- Buildfarm部署中的问题

news 2025/8/11 9:51:29

问题1：server报logOverdueOperation和WARNING: removed dispatched operation shard/operations/， worker报WARNING: missing queued operation: shard/operations/等问题，详情如下：

Server Log

INFO: DispatchedMonitor: Testing shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0 because 1648017713060ms overdue (1648017713060 >= 0)
Mar 23, 2022 6:41:53 AM build.buildfarm.instance.shard.RedisShardBackplane queue
WARNING: removed dispatched operation shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0
Mar 23, 2022 6:41:53 AM build.buildfarm.instance.shard.DispatchedMonitor lambda$requeueDispatchedOperation$0
INFO: DispatchedMonitor::run: requeue(shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0) 13.2932ms
Mar 23, 2022 6:41:54 AM build.buildfarm.instance.shard.DispatchedMonitor logOverdueOperation
INFO: DispatchedMonitor: Testing shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0 because 1648017714076ms overdue (1648017714076 >= 0)
Mar 23, 2022 6:41:54 AM build.buildfarm.instance.shard.ShardInstance canOperationBeRequeued
WARNING: Operation shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0 not requeued.  Operation has been requeued too many times ( 6 > 5).
Mar 23, 2022 6:41:54 AM build.buildfarm.instance.shard.DispatchedMonitor lambda$requeueDispatchedOperation$0
INFO: DispatchedMonitor::run: requeue(shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0) 0.0108280ms

Woker Log

WARNING: missing queued operation: shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0
Mar 23, 2022 6:41:52 AM build.buildfarm.worker.InputFetcher fetchPolled
SEVERE: invalid queued operation: shard/operations/c1701659-f015-492b-abdb-ec5b190d07d0

使用了两台机器

机器1：部署redis1、server1、worker1

机器2：worker2

其中worker1默认没有任何日志，worker2报上述Woker Log。

定位步骤：

更改源码，在WARNING: missing queued operation打印的源码处加日志，判断执行情况，然后重新源码构建运行server和worker，

可以发现worker2上显示的丢失operation在worker1上运行，（测试命令执行是在机器1和机器2上执行现象一样，猜测和server同机器的worker更优先分配，减少传输时间？）

类似报错如下链接

Operation has been requeued too many times · Issue #1047 · bazelbuild/bazel-buildfarm · GitHub

备注：查询buildFarm问题，可以在如下buildFarm的issues中查询，可获取其他人类似问题时buildFarm官方人员的回答。链接如下：

Issues · bazelbuild/bazel-buildfarm · GitHub

查看全文