[jira] [Created] (FLINK-13493) BoundedBlockingSubpartition only notifies onConsumedSubpartition when all the readers are empty

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-13493) BoundedBlockingSubpartition only notifies onConsumedSubpartition when all the readers are empty

Shang Yuanchun (Jira)
zhijiang created FLINK-13493:
--------------------------------

             Summary: BoundedBlockingSubpartition only notifies onConsumedSubpartition when all the readers are empty
                 Key: FLINK-13493
                 URL: https://issues.apache.org/jira/browse/FLINK-13493
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Network
            Reporter: zhijiang
            Assignee: zhijiang


In previous implementation, it would always notify the {{ResultPartition}} of consumed subpartition if any reader view is released. Based on the reference-counter release strategy it might cause problems if one blocking subpartition has multiple readers. That means the whole result partition might be released but there are still alive readers in some subpartitions.

Although the default release strategy for blocking partition is based on JM/scheduler notification atm. But if we switch the option to based on consumption notification it would cause problems. And from the subpartition side it should has the right behavior no matter what is the specific release strategy in upper layer.

In order to fix this bug, the {{BoundedBlockingSubpartition}} would only notify {{onConsumedSubpartition}} when all the readers are empty.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)