Re: [DISCUSS] Support configure remote flink user jar In k8s per-job mode

[Yang Wang]

Hi Ouyang,

If you are focused on kubernetes standalone per-job cluster, i think you
could use the initContainer
to download user jars from remote storage and then copy them to flink
classpath. When the
JobManager and TaskManger container launched, they could use the jars
locally.

If you want to get a more common case, Peter Huang has started another
discussion about this,
"Delayed Job Graph Generation in Yarn Per Job Cluster"[1]. I think you
could join in and try to get a
more common solution, both for Yarn and Kubernetes per-job.

[1].
https://docs.google.com/document/d/1aAwVjdZByA-0CHbgv16Me-vjaaDMCfhX7TzVVTuifYM/edit?ts=5df05a6f#heading=h.be92q3uiam4t


Best,
Yang

ouywl <[hidden email]> 于2019年12月11日周三 下午2:26写道：

> Dear all,
>    This discuss is about flink support a remote user jar. Now as run job
> on k8s,  use doc
> https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html,
> It need build user jar in image, because It use ‘
> ClassPathJobGraphRetriever
> ’ Load  the jobgraph. But It is a Poor user experience. I
> have two suggestions.
>       1. Can we develop the function get jar from remote, and retriever it
> to JobGraph. It can also used on yarn or mesos,not only k8s
>       2. Also It can resolve by user ‘initcontainers’ to download jar
> from remote, But It only can use in k8s .
>  @Yang Wang  What do think about this？ It can add a jira for this？
> ouywl
> [hidden email]
>
> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=ouywl&uid=ouywl%40139.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsma8dc7719018ba2517da7111b3db5a170.jpg&items=%5B%22ouywl%40139.com%22%5D>
> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制
>
>