Piotr Nowojski created FLINK-19537:
--------------------------------------
Summary: Processed in-flight bytes metric is not accurate
Key: FLINK-19537
URL:
https://issues.apache.org/jira/browse/FLINK-19537 Project: Flink
Issue Type: Improvement
Components: Runtime / Metrics, Runtime / Network
Affects Versions: 1.12.0
Reporter: Piotr Nowojski
Processed in-flight bytes as introduced in FLINK-18662 is not entirely accurate, as it's ignoring the buffer/bytes accumulated in the record deserializers. If buffer is processed here, it doesn't mean it was fully processed (so we can over estimate the amount of processed bytes). On the other hand some records/bytes might be processed without polling anything from this {{CheckpointedInputGate}} (underestimating the amount of processed bytes). All in all this should have been calculated on the {{StreamTaskNetworkInput}} level, where we have an access to the records deserializers. However the current is on average accurate and it might be just good enough (at least for the time being).
Also this metric is currently ignoring chained source inputs to the multiple input stream task.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
