[jira] [Created] (FLINK-12343) Allow set hdfs.replication in FlinkYarnSessionCli

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-12343) Allow set hdfs.replication in FlinkYarnSessionCli

Shang Yuanchun (Jira)
Zhenqiu Huang created FLINK-12343:
-------------------------------------

             Summary: Allow set hdfs.replication in FlinkYarnSessionCli
                 Key: FLINK-12343
                 URL: https://issues.apache.org/jira/browse/FLINK-12343
             Project: Flink
          Issue Type: Improvement
          Components: Command Line Client, Deployment / YARN
    Affects Versions: 1.8.0, 1.7.2, 1.6.4
            Reporter: Zhenqiu Huang
            Assignee: Zhenqiu Huang


Currently, FlinkYarnSessionCli upload jars into hdfs with default 3 replications. From our production experience, we find that 3 replications will block big job (256 containers) to launch, when the HDFS is slow due to big workload for batch pipelines. Thus, we want to make the factor customizable from FlinkYarnSessionCli by adding an option.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)