[jira] [Created] (FLINK-12070) Make blocking result partitions consumable multiple times

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-12070) Make blocking result partitions consumable multiple times

Shang Yuanchun (Jira)
Till Rohrmann created FLINK-12070:
-------------------------------------

             Summary: Make blocking result partitions consumable multiple times
                 Key: FLINK-12070
                 URL: https://issues.apache.org/jira/browse/FLINK-12070
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Network
            Reporter: Till Rohrmann


In order to avoid writing produced results multiple times for multiple consumers and in order to speed up batch recoveries, we should make the blocking result partitions to be consumable multiple times. At the moment a blocking result partition will be released once the consumers has processed all data. Instead the result partition should be released once the next blocking result has been produced and all consumers of a blocking result partition have terminated. Moreover, blocking results should not hold on slot resources like network buffers or memory as it is currently the case with {{SpillableSubpartitions}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)