[jira] [Created] (FLINK-3346) Replace Akka death watch with own heartbeats

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-3346) Replace Akka death watch with own heartbeats

Shang Yuanchun (Jira)
Till Rohrmann created FLINK-3346:
------------------------------------

             Summary: Replace Akka death watch with own heartbeats
                 Key: FLINK-3346
                 URL: https://issues.apache.org/jira/browse/FLINK-3346
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime
    Affects Versions: 1.0.0
            Reporter: Till Rohrmann
            Priority: Minor


Currently, we're using Akka's death watch to detect failed instances (e.g. the JM watches the TM and vice versa). Whenever a death watch is triggered, the corresponding {{ActorSystem}} is quarantined. This will prevent the quarantined {{ActorSystem}} from reconnecting to the detecting system, in case that it didn't die. This can happen once in a while given a sub-optimal death watch configuration.

In order to prevent this from happening, we could implement our own heartbeats which are periodically sent from the {{TaskManagers}} to the {{JobManager}} and vice versa, for example. Not receiving a certain number of heartbeats could then mean that the {{TaskManager}} is marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)