Flink Sql Join, how to clear the sql join state?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink Sql Join, how to clear the sql join state?

LakeShen
Hi community,now I am use flink sql inner join in my code,I saw the flink
document, the flink sql inner join will keep both sides of the join input
in Flink’s state forever.
As result , the hdfs files size are so big , is there any way to clear the
sql join state?
Thanks to your reply.
Reply | Threaded
Open this post in threaded view
|

Re: Flink Sql Join, how to clear the sql join state?

Benchao Li
Hi LakeShen,

Maybe "Idle State Retention Time"[1] may help in your case.

[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/query_configuration.html#idle-state-retention-time

LakeShen <[hidden email]> 于2020年1月16日周四 上午10:15写道:

> Hi community,now I am use flink sql inner join in my code,I saw the flink
> document, the flink sql inner join will keep both sides of the join input
> in Flink’s state forever.
> As result , the hdfs files size are so big , is there any way to clear the
> sql join state?
> Thanks to your reply.
>


--

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: [hidden email]; [hidden email]