[jira] [Created] (FLINK-12735) Make shuffle environment implementation independent with IOManager

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-12735) Make shuffle environment implementation independent with IOManager

Shang Yuanchun (Jira)
zhijiang created FLINK-12735:
--------------------------------

             Summary: Make shuffle environment implementation independent with IOManager
                 Key: FLINK-12735
                 URL: https://issues.apache.org/jira/browse/FLINK-12735
             Project: Flink
          Issue Type: Sub-task
          Components: Runtime / Network
            Reporter: zhijiang
            Assignee: zhijiang


The current creation of {{NetworkEnvironment}} is relying on {{IOManager}} from {{TaskManagerServices}}. In order not to rely on external specific components for implementing shuffle environment, and let the specific implementation creates internal components if required.

The current abstract {{IOManager}} has two roles, one is for file channel management based on temp directories configuration, and the other is providing abstract methods for reading/writing files. We could further extract the file channel management as a separate internal class which could be reused for all the required components, like current {{BoundedBlockingSubpartition}}. To do so, the shuffle environment should also creates its internal channel manager to break dependency with {{IOManager}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)