spark another attempt succeeded

执行spark任务发现 提示信息“ another attempt succeeded”,这是由于设置了spark.speculation为true,该参数的含义如下:

参考:

If a task appears to be taking an unusually long time to complete, Spark may launch extra duplicate copies of that task in case they can complete sooner. This is referred to as speculation or speculative execution. If one copy succeeds, the others can be killed.

See the parameters starting with spark.speculation here: https://spark.apache.org/docs/latest/configuration.html

已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 编程工作室 设计师:CSDN官方博客 返回首页