[jira] [Created] (FLINK-8279) Use Mesos/YARN temp directories as fallback for BlobServer/Cache temp directories

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-8279) Use Mesos/YARN temp directories as fallback for BlobServer/Cache temp directories

Shang Yuanchun (Jira)
Nico Kruber created FLINK-8279:
----------------------------------

             Summary: Use Mesos/YARN temp directories as fallback for BlobServer/Cache temp directories
                 Key: FLINK-8279
                 URL: https://issues.apache.org/jira/browse/FLINK-8279
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Coordination, Network
    Affects Versions: 1.4.0
            Reporter: Nico Kruber
            Assignee: Nico Kruber


Currently, the BLOB server and cache processes (temporarily) stash incoming files into their local file system in the directory given by the {{blob.storage.directory}} configuration property. If this property is not set or empty, it will fall back to {{java.io.tmpdir}}.

Instead, in a Mesos/YARN environment, we could use the temporary directories they assigned to the Flink job which are not only the proper folder to use, but may also offer some more space.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)