[jira] [Created] (FLINK-1350) Add blocking intermediate result partitions

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-1350) Add blocking intermediate result partitions

Shang Yuanchun (Jira)
Ufuk Celebi created FLINK-1350:
----------------------------------

             Summary: Add blocking intermediate result partitions
                 Key: FLINK-1350
                 URL: https://issues.apache.org/jira/browse/FLINK-1350
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime
            Reporter: Ufuk Celebi


The current state of runtime support for intermediate results (see https://github.com/apache/incubator-flink/pull/254 and FLINK-986) only supports pipelined intermediate results (with back pressure), which are consumed as they are being produced.

The next variant we need to support are blocking intermediate results (without back pressure), which are fully produced before being consumed. This is for example desirable in situations, where we currently may run into deadlocks when running pipelined.

I will start working on this on top of my pending pull request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)