Submitting jobs via maven coordinates?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Submitting jobs via maven coordinates?

Ron Crocker
Internally we use artifactory to store our jars. I have a team that would like to submit their Flink jobs via their maven coordinates, resulting in pull the specified jar from the artifactory repository.

Has anyone else seen a need for this? Has it already been accomplished?

Ron

Ron Crocker
Principal Engineer & Architect
( ( •)) New Relic
[hidden email]
M: +1 630 363 8835

Reply | Threaded
Open this post in threaded view
|

Re: Submitting jobs via maven coordinates?

Till Rohrmann
Hi Ron,

as far as I know, nobody has really accomplished that. It is an interesting
idea and I would suggest to file a JIRA for it. Would it be enough to add
support for arbitrary URLs (flink script) when submitting a job to Flink?
Alternatively I think a wrapper script could do the trick but I'm not too
familiar with artifactory.

Cheers,
Till

On Tue, Nov 14, 2017 at 7:46 AM, Ron Crocker <[hidden email]> wrote:

> Internally we use artifactory to store our jars. I have a team that would
> like to submit their Flink jobs via their maven coordinates, resulting in
> pull the specified jar from the artifactory repository.
>
> Has anyone else seen a need for this? Has it already been accomplished?
>
> Ron
> —
> Ron Crocker
> Principal Engineer & Architect
> ( ( •)) New Relic
> [hidden email]
> M: +1 630 363 8835
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Submitting jobs via maven coordinates?

Jörn Franke
In reply to this post by Ron Crocker
I have seen no official script but what you describe should be easily down with a build tool such as Gradle. The advantage would be that a build tool is already tested and you do not have to maintain scripts for downloading etc.

> On 14. Nov 2017, at 07:46, Ron Crocker <[hidden email]> wrote:
>
> Internally we use artifactory to store our jars. I have a team that would like to submit their Flink jobs via their maven coordinates, resulting in pull the specified jar from the artifactory repository.
>
> Has anyone else seen a need for this? Has it already been accomplished?
>
> Ron
> —
> Ron Crocker
> Principal Engineer & Architect
> ( ( •)) New Relic
> [hidden email]
> M: +1 630 363 8835
>
Reply | Threaded
Open this post in threaded view
|

Re: Submitting jobs via maven coordinates?

Ron Crocker
I'll file the Jira and I might take a try at it as well.

Ron
On Tue, Nov 14, 2017 at 1:03 AM Jörn Franke <[hidden email]> wrote:

> I have seen no official script but what you describe should be easily down
> with a build tool such as Gradle. The advantage would be that a build tool
> is already tested and you do not have to maintain scripts for downloading
> etc.
>
> > On 14. Nov 2017, at 07:46, Ron Crocker <[hidden email]> wrote:
> >
> > Internally we use artifactory to store our jars. I have a team that
> would like to submit their Flink jobs via their maven coordinates,
> resulting in pull the specified jar from the artifactory repository.
> >
> > Has anyone else seen a need for this? Has it already been accomplished?
> >
> > Ron
> > —
> > Ron Crocker
> > Principal Engineer & Architect
> > ( ( •)) New Relic
> > [hidden email]
> > M: +1 630 363 8835
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Submitting jobs via maven coordinates?

Ron Crocker
https://issues.apache.org/jira/browse/FLINK-8074 <https://issues.apache.org/jira/browse/FLINK-8074>

Ron

Ron Crocker
Principal Engineer & Architect
( ( •)) New Relic
[hidden email]
M: +1 630 363 8835

> On Nov 14, 2017, at 9:00 AM, Ron Crocker <[hidden email]> wrote:
>
> I'll file the Jira and I might take a try at it as well.
>
> Ron
> On Tue, Nov 14, 2017 at 1:03 AM Jörn Franke <[hidden email] <mailto:[hidden email]>> wrote:
> I have seen no official script but what you describe should be easily down with a build tool such as Gradle. The advantage would be that a build tool is already tested and you do not have to maintain scripts for downloading etc.
>
> > On 14. Nov 2017, at 07:46, Ron Crocker <[hidden email] <mailto:[hidden email]>> wrote:
> >
> > Internally we use artifactory to store our jars. I have a team that would like to submit their Flink jobs via their maven coordinates, resulting in pull the specified jar from the artifactory repository.
> >
> > Has anyone else seen a need for this? Has it already been accomplished?
> >
> > Ron
> > —
> > Ron Crocker
> > Principal Engineer & Architect
> > ( ( •)) New Relic
> > [hidden email] <mailto:[hidden email]>
> > M: +1 630 363 8835
> >