Aljoscha Krettek created FLINK-5972:
---------------------------------------
Summary: Don't allow shrinking merging windows
Key: FLINK-5972
URL:
https://issues.apache.org/jira/browse/FLINK-5972 Project: Flink
Issue Type: Bug
Components: DataStream API
Affects Versions: 1.2.0, 1.1.0, 1.3.0
Reporter: Aljoscha Krettek
Priority: Blocker
Fix For: 1.3.0
A misbehaving {{MergingWindowAssigner}} can cause a merge that results in a window that is smaller than the span of all the merged windows. This, in itself is not problematic. It becomes problematic when the end timestamp of a window that was not late before merging is now earlier than the watermark (the timestamp is smaller than the watermark).
There are two choices:
- immediately process the window
- drop the window
processing the window will lead to late data downstream.
The current behaviour is to silently drop the window but that logic has a bug: we only remove the dropped window from the {{MergingWindowSet}} but we don't properly clean up state and timers that the window still (possibly) has. We should fix this bug in the process of resolving this issue.
We should either just fix the bug and still silently drop windows or add a check and throw an exception when the end timestamp falls below the watermark.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Locked
