[jira] [Created] (FLINK-13442) Remove unnecessary notifySubpartitionConsumed method from view reader

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-13442) Remove unnecessary notifySubpartitionConsumed method from view reader

Shang Yuanchun (Jira)
zhijiang created FLINK-13442:
--------------------------------

             Summary: Remove unnecessary notifySubpartitionConsumed method from view reader
                 Key: FLINK-13442
                 URL: https://issues.apache.org/jira/browse/FLINK-13442
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Network
            Reporter: zhijiang
            Assignee: zhijiang


Currently the methods of `NetworkSequenceViewReader#notifySubpartitionConsumed` and ` NetworkSequenceViewReader#releaseAllResources` would be called meanwhile in netty stack during releasing resources.

To make this release logic simple and clean, we could remove the redundant `notifySubpartitionConsumed` from `NetworkSequenceViewReader` side, and also remove it from `ResultSubpartitionView` side. In the implementation of `ResultSubpartitionView#releaseAllResources` it would further notify the parent subpartition of consumed state via `ResultSubpartition#notifySubpartitionConsumed` which further feedback to parent `ResultPartition` layer via `onConsumedSubpartition`. Finally `ResultPartition` could decide whether to release itself or not.

E.g. for the case of `ReleaseOnConsumptionResultPartition` which is mainly used for pipelined partition, it would release partition after reference counter decreased to 0. For the case of `ResultPartition` which would be generated for blocking partition by default, it would never be released after notifying consumed. And the JM/scheduler would decide when to release partition properly.

In addition, `InputChannel#notifySubpartitionConsumed` could also be removed as a result of above.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)