Danny Chen created FLINK-17173:
----------------------------------
Summary: Supports query hint to config "IdleStateRetentionTime" per query in SQL
Key: FLINK-17173
URL:
https://issues.apache.org/jira/browse/FLINK-17173 Project: Flink
Issue Type: Improvement
Components: Table SQL / API
Affects Versions: 1.11.0
Reporter: Danny Chen
The motivation why we need this (copy from user mailing list [~qzhzm173227])
In some of the use cases our users have, they have a couple of complex join queries where the key domains key evolving - we definitely want some sort of state retention for those queries; but there are other where the key domain doesn't evolve overtime, but there isn't really a guarantee on what's the maximum gap between 2 records of the same key to appear in the stream, we don't want to accidentally invalidate the state for those keys in these streams.
Because of queries with different requirements can both exist in the pipeline, I think we have to config `IDLE_STATE_RETENTION_TIME` per operator.
Just wondering, has similar requirement not come up much for SQL users before? (being able to set table / query configuration inside SQL queries)
We are also a little bit concerned because right now since 'toRetractStream(Table, Class, QueryConfig)' is deprecated, relying on the fact that TableConfig is read during toDataStream feels like relying on an implementation details that just happens to work, and there is no guarantee that it will keep working in the future versions...
Demo syntax:
{code:sql}
CREATE TABLE `/output` AS
SELECT /*+ IDLE_STATE_RETENTION_TIME(minTime ='5m', maxTime ='11m') */ *
FROM `/input1` a
INNER JOIN `/input2` b
ON a.column_name = b.column_name;
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
