[jira] [Created] (FLINK-22716) Add metrics watermarkAlignedChannelNum

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-22716) Add metrics watermarkAlignedChannelNum

Shang Yuanchun (Jira)
tartarus created FLINK-22716:
--------------------------------

             Summary: Add metrics watermarkAlignedChannelNum
                 Key: FLINK-22716
                 URL: https://issues.apache.org/jira/browse/FLINK-22716
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Metrics
            Reporter: tartarus


WatermarkStrategy#withIdleness will introduce IDLE state;
When the downstream Watermark is aligned, the subtask Watermark will be ignored.
When the status of the subtask changes from IDLE to ACTIVE, the watermark of the subtask is not immediately considered when the downstream Watermark is aligned, until its Watermark is greater than the lastOutputWatermark of the Operator.
For details, please refer to class StatusWatermarkValve.
If the downstream is a window operator and delay is not allowed, the data may be drop.
We should add some metrics to identify the current Watermark alignment, considering the data of multiple channels.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)