[jira] [Created] (FLINK-16279) Per job Yarn application leak in normal execution mode.

[Shang Yuanchun (Jira)]

Wenlong Lyu created FLINK-16279:
-----------------------------------

             Summary: Per job Yarn application leak in normal execution mode.
                 Key: FLINK-16279
                 URL: https://issues.apache.org/jira/browse/FLINK-16279
             Project: Flink
          Issue Type: Bug
          Components: Runtime / Coordination
    Affects Versions: 1.10.0
            Reporter: Wenlong Lyu


I run a job in yarn per job mode using {{env.executeAsync}}, the job failed but the yarn cluster didn't be destroyed.

After some research on the code, I found that:

when running in attached mode, MiniDispatcher will neve set {{shutDownfuture}} before received a request from job client.
{code}
                if (executionMode == ClusterEntrypoint.ExecutionMode.NORMAL) {
                        // terminate the MiniDispatcher once we served the first JobResult successfully
                        jobResultFuture.thenAccept((JobResult result) -> {
                                ApplicationStatus status = result.getSerializedThrowable().isPresent() ?
                                                ApplicationStatus.FAILED : ApplicationStatus.SUCCEEDED;

                                LOG.debug("Shutting down per-job cluster because someone retrieved the job result.");
                                shutDownFuture.complete(status);
                        });
                }
{code}

However, when running in async mode(submit job by env.executeAsync), there may be no request from job client because when a user find that the job is failed from job client, he may never request the result again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)