godfrey he created FLINK-12371:
----------------------------------
Summary: Add support for converting (NOT) IN/ (NOT) EXISTS to SemiJoin, and generating optimized logical plan
Key: FLINK-12371
URL:
https://issues.apache.org/jira/browse/FLINK-12371 Project: Flink
Issue Type: New Feature
Components: Table SQL / Planner
Reporter: godfrey he
Assignee: godfrey he
This issue aims to convert IN/EXISTS to semi-join, and NOT IN/NOT EXISTS to anti-join.
In Calcite, [SemiJoin|
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/SemiJoin.java] only represents semi-join, (could not represent anti-join) and requires equi join condition. Queries like `select * from left where left.a1 in (select right.a2 from right where left.b1 > right.b2)` and `select * from left where not exists (select * from right)` could not be converted to Calcite SemiJoin operator.
To solve the above problem, We need copy the {{SemiJoin}} class to Flink, and make the following changes:
1. make {{SemiJoin}} extending from {{Join}}, not from {{EquiJoin}}. (to support non-equi join condition)
2. add {{isAnti}} field attribute to represent anti-join.
Currently, there are no rules to convert (NOT) IN/ (NOT) EXISTS to SemiJoin, so we need a whole new set of rules to meet our requirement.
1. {{FlinkSubQueryRemoveRule}}, a planner rule that converts IN and EXISTS into semi-join, converts NOT IN and NOT EXISTS into anti-join
2. {{FlinkRewriteSubQueryRule}}, a planner rule that converts filter condition like: {{(select count(*) from T) > 0}} to {{exists(select * from T)}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
