Till Rohrmann created FLINK-15924:
-------------------------------------
Summary: Detect and log blocking main thread operations
Key: FLINK-15924
URL:
https://issues.apache.org/jira/browse/FLINK-15924 Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: Till Rohrmann
Fix For: 1.11.0
When using the {{RpcEndpoint}} it is important that all operations which run on the main thread are never blocking. We have seen in the past that it is quite hard to always catch blocking operations in reviews and sometimes these changes caused instabilities in Flink. Once this happens it is not trivial to find the culprit which is responsible for the blocking operation.
One way to make debugging easier is to add a monitor which detects and logs if a {{RpcEndpoint}} operation takes longer than {{n}} seconds for example. Depending on the overhead of this monitor one could even think about enabling it only via a special configuration (e.g. debug mode).
A proper class to introduce this monitor could be the {{AkkRpcActor}} which is responsible for executing main thread operations. Whenever we schedule an operation, we could start a timeout which if triggered and the operation has not been completed will log a warning.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
