[jira] [Created] (FLINK-19109) Split Reader eats chained periodic watermarks

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-19109) Split Reader eats chained periodic watermarks

Shang Yuanchun (Jira)
David Anderson created FLINK-19109:
--------------------------------------

             Summary: Split Reader eats chained periodic watermarks
                 Key: FLINK-19109
                 URL: https://issues.apache.org/jira/browse/FLINK-19109
             Project: Flink
          Issue Type: Bug
    Affects Versions: 1.11.1, 1.10.2, 1.11.0, 1.10.1, 1.10.0
            Reporter: David Anderson


Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in


{{        SingleOutputStreamOperator<Event> results = env
                .readTextFile(...)
                .map(...)
                .assignTimestampsAndWatermarks(bounded)
                .keyBy(...)
                .process(...);
}}

leads to the Watermarks failing to be produced. Breaking the chain, via {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using punctuated watermarks also avoids the issue.

Looking at this in the debugger reveals that timer service is being prematurely quiesced.

In many respects this is FLINK-7666 brought back to life.

The problem is not present in 1.9.3.

There's a minimal reproducible example in https://github.com/alpinegizmo/flink-question-001/tree/bug.






--
This message was sent by Atlassian Jira
(v8.3.4#803005)