[jira] [Created] (FLINK-4873) Add config option to specify "home directory" for YARN client resource sharing

[Shang Yuanchun (Jira)]

Gyula Fora created FLINK-4873:
---------------------------------

             Summary: Add config option to specify "home directory" for YARN client resource sharing
                 Key: FLINK-4873
                 URL: https://issues.apache.org/jira/browse/FLINK-4873
             Project: Flink
          Issue Type: Improvement
          Components: YARN Client
    Affects Versions: 1.2.0, 1.1.3
            Reporter: Gyula Fora


The YARN client currently uses FileSystem.getHomeDirectory() to store the jar files that needed to be shared on the cluster. This pretty much forces users to run HDFS or something compatible with the Hadoop FS api on the cluster.

In some production environments file systems (distributed or simply shared) are simply mounted under the same path and do not require the use of the hadoop api for convenience. If we want to run Flink on YARN in these cases we would need to be able to define the "home directory" where Flink should copy the files for sharing.

It could be something like:
yarn.resource.upload.dir in the flink-conf.yaml



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)