[jira] [Created] (FLINK-20611) Support mini batch for window aggregate operation

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-20611) Support mini batch for window aggregate operation

Shang Yuanchun (Jira)
Suhan Mao created FLINK-20611:
---------------------------------

             Summary: Support mini batch for window aggregate operation
                 Key: FLINK-20611
                 URL: https://issues.apache.org/jira/browse/FLINK-20611
             Project: Flink
          Issue Type: New Feature
          Components: Table SQL / API
            Reporter: Suhan Mao


Currently, mini batch does not support window aggregate. In our company, many jobs run with window aggregate and suffer from large amounts of intermediate shuffle. If we can enable mini batch on window aggregate, the performance will be improved.

This issue aims to support mini batch in window aggregate operator both in SQL and Table API. Simple tumble window will be supported in the first place.

 

We still use below configurations, but they take effect only inside each single window.

{{}}

{{```}}

{{table.exec.mini-batch.enabled}}

{{}}

{{table.exec.mini-batch.allow-latency}}

{{}}

{{table.exec.mini-batch.size}}

{{}}

{{```}}{{}}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)