[jira] [Created] (FLINK-1928) BlobManager can be a bottleneck on large deployments

[Shang Yuanchun (Jira)]

Robert Metzger created FLINK-1928:
-------------------------------------

             Summary: BlobManager can be a bottleneck on large deployments
                 Key: FLINK-1928
                 URL: https://issues.apache.org/jira/browse/FLINK-1928
             Project: Flink
          Issue Type: Improvement
          Components: BlobManager, JobManager, TaskManager
    Affects Versions: 0.9
            Reporter: Robert Metzger
            Priority: Minor


The BlobManager needed more than 2 minutes to distribute 220+ jar files to 50 TaskManagers in a user's setup.

Serving the files for all the TMs from one JM might will quickly make the network of the JM a bottleneck when distributing the files.
Potential solutions could be:
- Use a torrent-style distribution mode
- Use HDFS




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)