[jira] [Created] (FLINK-19548) Report correct progress to yarn resource manager

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-19548) Report correct progress to yarn resource manager

Shang Yuanchun (Jira)
fa zheng created FLINK-19548:
--------------------------------

             Summary: Report correct progress to yarn resource manager
                 Key: FLINK-19548
                 URL: https://issues.apache.org/jira/browse/FLINK-19548
             Project: Flink
          Issue Type: Improvement
          Components: Deployment / YARN
    Affects Versions: 1.11.2
            Reporter: fa zheng
             Fix For: 1.12.0


Currently, we directly report 1 to yarn resource manager. It doesn't make sense for both stream and batch job. For a stream job, it's better to report 0, and for a batch job, it's better to report a meaningful progress. A simple way is calculating  totalSubtask and currentFinishedSubTask in scheduler, and return (float)currentFinishedSubTask.get()/totalSubTask in YarnResourceManager.getProgress()



--
This message was sent by Atlassian Jira
(v8.3.4#803005)