[jira] [Created] (FLINK-1232) Allow I/O spill file writers to work with futures/callbacks

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-1232) Allow I/O spill file writers to work with futures/callbacks

Shang Yuanchun (Jira)
Stephan Ewen created FLINK-1232:
-----------------------------------

             Summary: Allow I/O spill file writers to work with futures/callbacks
                 Key: FLINK-1232
                 URL: https://issues.apache.org/jira/browse/FLINK-1232
             Project: Flink
          Issue Type: Sub-task
          Components: Local Runtime
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 0.8-incubating


To simplify the implementation of the adaptive memory management, the asynchronous writers could use callbacks that are invoked when a memory has been successfully written.

In the current solution, the thread writing memory segments to disk needs to keep track of how much was spilled and release that memory later.

The callbacks can in that case release the memory directly. This is both simpler and less prone to leaks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)