[jira] [Created] (FLINK-1445) Add support to enforce local input split assignment

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-1445) Add support to enforce local input split assignment

Shang Yuanchun (Jira)
Fabian Hueske created FLINK-1445:
------------------------------------

             Summary: Add support to enforce local input split assignment
                 Key: FLINK-1445
                 URL: https://issues.apache.org/jira/browse/FLINK-1445
             Project: Flink
          Issue Type: New Feature
          Components: Java API, JobManager
    Affects Versions: 0.9
            Reporter: Fabian Hueske
            Priority: Minor


In some scenarios data sources cannot remotely read data as for example in distributed cluster setups where each machine stores data on its local FS which cannot be remotely read.

In order to enable such use cases with Flink, we need to
1) add support for enforcing local input split reading.
2) ensure that each input split can be locally read by at least one data source task which means to influence the scheduling of data source tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)