Re: MODERATE for [e-mail]

[Henry Saputra]

Hi,

Looks like you have not subscribe to Apache Flink dev@ mailing list.

Please subscribe by following instruction here to continue with the
discussions and receive followups:
https://flink.apache.org/community.html#mailing-lists


Thanks,

Henry Saputra
On behalf of Apache Flink PMC

>
>
> ---------- Forwarded message ----------
> From: Teng Fei Liao <[hidden email]>
> To: [hidden email]
> Cc:
> Bcc:
> Date: Tue, 2 Jun 2020 03:04:38 -0400
> Subject: Common HA setups
>
> Hi Flink devs!
>
> After reading through documentation and reading posts others have made
> online for their setups, it seems like there’s similarities in achieving HA
> with just a single job manager. For example, the yarn setup
> <https://apollo.palantircloud.com/aries/logDetails/v2/%7B%22columns%22%3A%5B%7B%22propertyKey%22%3A%22time%22%7D%2C%7B%22propertyKey%22%3A%22traceId%22%7D%2C%7B%22propertyKey%22%3A%22level%22%7D%2C%7B%22propertyKey%22%3A%22message%22%7D%2C%7B%22propertyKey%22%3A%22params%22%7D%2C%7B%22propertyKey%22%3A%22origin%22%7D%2C%7B%22propertyKey%22%3A%22stacktrace%22%7D%2C%7B%22propertyKey%22%3A%22params%22%2C%22path%22%3A%22clusterId%22%7D%5D%2C%22queryStrings%22%3A%5B%22origin%3A%5C%22com.palantir.flink.runtime.ha.FoundryHaServicesFactory%5C%22%22%2C%22((params.key%3AclusterId)%20AND%20(params.value%3A%5C%226a5e3805%5C%5C-aa20%5C%5C-4bd9%5C%5C-969d%5C%5C-08f0acee67ec%5C%22))%22%5D%2C%22startTime%22%3A%222020-06-01T03%3A04%3A05.989%22%2C%22serviceLocators%22%3A%5B%7B%22environmentId%22%3A%22oregano-rubix-staging-lowtrust%22%2C%22logType%22%3A%22SERVICE_LOG%22%2C%22blueGreenGroupId%22%3A%22production%7Cflink-job-manager%22%7D%5D%7D>
> specifies only a single job manager is necessary and will be restarted on
> failures. This Kubernetes post
> <https://jobs.zalando.com/en/tech/blog/running-apache-flink-on-kubernetes/?gh_src=22377bdd1us>
> has a similar single job manager setup. It has a fill-in for zookeeper but
> I think abstractly, the two have these same features in common:
>
> 1. Persistent storage (the high-availability.storageDir config value)
>
> 2. Low latency job manager restart times.
>
> For our setup, we're actually experimenting with a variation of the
> kubernetes set up that removes zookeeper altogether by implementing a file
> based HighAvailabilityServices and trivial leader election services. Given
> the relative simplicity of the setup and code, I was wondering how
> recommended and supported this variant is. Potentially, this could be made
> available by default to help other users simplify their setups. Curious
> what your thoughts are.
>
> Thanks,
>
> Teng.
>