[jira] [Created] (FLINK-14874) add local aggregate to solve data skew for ROLLUP/CUBE case

[Shang Yuanchun (Jira)]

godfrey he created FLINK-14874:
----------------------------------

             Summary: add local aggregate to solve data skew for ROLLUP/CUBE case
                 Key: FLINK-14874
                 URL: https://issues.apache.org/jira/browse/FLINK-14874
             Project: Flink
          Issue Type: Sub-task
          Components: Table SQL / Planner
    Affects Versions: 1.9.1, 1.9.0
            Reporter: godfrey he
             Fix For: 1.10.0


Many tpc-ds queries have {{rollup}} keyword, which will be translated to multiple groups.
for example:  {{group by rollup (channel, id) }} is equivalent {{ group by (channel, id)}} +  {{ group by (channel)}} +  {{ group by () }}.
All data on empty group will be shuffled to a single node, It is a typical data skew case. If there is a local aggregate, the data size shuffled to the single node will be greatly reduced. However, currently the cost mode can't estimate the local aggregate's cost, and the plan with local aggregate may be chose even the query has {{rollup}} keyword.
we could add a rule based phase (after physical phase) to enforce local aggregate if it's input has empty group.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)