[jira] [Created] (FLINK-14149) Introduce ZooKeeperLeaderElectionServiceNG

[Shang Yuanchun (Jira)]

tison created FLINK-14149:
-----------------------------

             Summary: Introduce ZooKeeperLeaderElectionServiceNG
                 Key: FLINK-14149
                 URL: https://issues.apache.org/jira/browse/FLINK-14149
             Project: Flink
          Issue Type: Sub-task
          Components: Runtime / Coordination
            Reporter: tison
            Assignee: tison


Subsequent to the discussion in FLINK-10333, we reach a consensus that refactor ZK based storage with a transaction store mechanism. The overall design can be found in the design document linked below.

This subtask is aimed at introducing the prerequisite to adopt transaction store, i.e., a new leader election service for ZK scenario. The necessity is that we have to retrieve the corresponding latch path per contender following the algorithm describe in FLINK-10333.

Here is the (descriptive) details about the implementation.

We adopt the optimized version of [this recipe|https://zookeeper.apache.org/doc/current/recipes.html#sc_leaderElection][1]. Code details can be found in [this branch|https://github.com/TisonKun/flink/tree/election-service] and the state machine can be found in the design document attached. Here is only the most important two differences from the former implementation:

(1) *Leader election is an one-shot service.*

Specifically, we only create one latch for a specific contender. We tolerate {{SUSPENDED}} a.k.a. {{CONNECTIONLOSS}} so that the only situation we lost leadership is session expired, which infers the ephemeral latch znode is deleted. We don't re-participant as contender so after {{revokeLeadership}} a contender will never be granted any more. This is not a problem but we can do further refactor in contender side for better behavior.

(2) *Leader info znode is {{PERSISTENT}}.*

It is because we now regard create/setData to leader info znode a leader-only operation and thus do it in a transaction. If we keep using ephemeral znode it is hard to test. Because we share ZK client so the ephemeral znode is not deleted so that we should deal with complex znode stat that transaction cannot simply deal with. And since znode is {{PERSISTENT}} we introduce a {{concealLeaderInfo}} method called back on contender stop to clean up.

Another topic is about interface. Back to the big picture of FLINK-10333 we eventually use a transaction store for persisting job graph and checkpoint and so on. So there will be a {{getLeaderStore}} method added on {{LeaderElectionServices}}. Because we don't use it at all it is an open question that whether we add the method to the interface in this subtask. And if so, whether we implement it for other election services implementation.

{{concealLeaderInfo}} is another method appeared in the document that aimed at clean up leader info node on stop. So the same problem as {{getLeaderStore}}.

**For what we gain**

1. Basics for the overall goal under FLINK-10333
2. Leader info node must be modified by the current leader. Thus we can reduce a lot of concurrency handling logic in currently ZLES, including using {{NodeCache}} as well as dealing with complex stat of ephemeral leader info node.

[1] For other implementation, I start [a thread|https://lists.apache.org/x/thread.html/594b66ecb1d60b560a5c4c08ed1b2a67bc29143cb4e8d368da8c39b2@%3Cuser.zookeeper.apache.org%3E] in ZK and Curator to discuss. Anyway, it will be implementation details only, and interfaces and semantics should not be affected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)