ClassLoader issue when submitting Flink Streaming programs through the web cliend

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ClassLoader issue when submitting Flink Streaming programs through the web cliend

Gyula Fóra-2
Hey,

While trying to add support for plan visualisation I also realised that
streaming programs cannot be run through the flink web client.

It seems to be a ClassLoader issue, which is interesting because we use the
classloader set in the environment to deserialize user defined objects.
This works for submitting jobs through the command line client though. I
dont see why it should be different when you submit something through the
commandline or the web interface.

Thanks,
Gyula
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader issue when submitting Flink Streaming programs through the web cliend

Robert Metzger
Didn't we have a similar issue before the 0.7.0-incubating release as well?

I thought I've tested submitting a streaming program with the web frontend
for the 0.8 release and it worked.

On Thu, Jan 22, 2015 at 2:31 PM, Gyula Fóra <[hidden email]> wrote:

> Hey,
>
> While trying to add support for plan visualisation I also realised that
> streaming programs cannot be run through the flink web client.
>
> It seems to be a ClassLoader issue, which is interesting because we use the
> classloader set in the environment to deserialize user defined objects.
> This works for submitting jobs through the command line client though. I
> dont see why it should be different when you submit something through the
> commandline or the web interface.
>
> Thanks,
> Gyula
>
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader issue when submitting Flink Streaming programs through the web cliend

Gyula Fóra-2
Actually I might have found the problem. I was unaware of the
OptimizerPlanEnvironment at the time I implemented the
StreamExecutionEnvironment.

So currently if the getExecutionEnvironment method of the
StreamExecutionEnvironment does not detect a ContextEnvironment (with
instanceOf check) it creates a minicluster for local execution. I basically
need to make sure it also detect this OptimizerPlanEnvironment which is
created when you submit from the webclient.

I hope this is the issue, I will try to fix it tomorrow.

https://issues.apache.org/jira/browse/FLINK-1434

On Thu, Jan 22, 2015 at 3:32 PM, Robert Metzger <[hidden email]> wrote:

> Didn't we have a similar issue before the 0.7.0-incubating release as well?
>
> I thought I've tested submitting a streaming program with the web frontend
> for the 0.8 release and it worked.
>
> On Thu, Jan 22, 2015 at 2:31 PM, Gyula Fóra <[hidden email]> wrote:
>
> > Hey,
> >
> > While trying to add support for plan visualisation I also realised that
> > streaming programs cannot be run through the flink web client.
> >
> > It seems to be a ClassLoader issue, which is interesting because we use
> the
> > classloader set in the environment to deserialize user defined objects.
> > This works for submitting jobs through the command line client though. I
> > dont see why it should be different when you submit something through the
> > commandline or the web interface.
> >
> > Thanks,
> > Gyula
> >
>