Exception during job submission in RemoteEnvironment

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Exception during job submission in RemoteEnvironment

Andreas Kunft
Hello,

I want to execute a Job from Java on an already running instance of
Flink. I connect via the RemoteEnvironment. Everything goes fine, but
then I get an exception from akka telling me the payload size is exceeded:

15-07-16 17:47:27 [ERROR] Transient association error (association
remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent
to Actor[akka.tcp://flink@127.0.0.1:6123/user/jobmanager#697056301]: max
allowed size 10485760 bytes, actual size of encoded class
org.apache.flink.runtime.messages.JobManagerMessages$SubmitJob was
28318303 bytes.

I tried to increase the akka.framesize configuration value but then
realized the executePlanWithJars method in the RemoteExecutor creates a
client with a new configuration.

Am I doing something wrong here or how can I provide my own
configuration to the Client?

Best
Andreas Kunft


Reply | Threaded
Open this post in threaded view
|

Re: Exception during job submission in RemoteEnvironment

Till Rohrmann
Hi Andreas,

for me this looks like an error. The problem is that the Client does not
respect the global configuration file when you execute your program from
Java/IDE. There is also no way to specify a configuration file for the
RemoteEnvironment, unlike for the LocalEnvironment. I think this should be
added.

Can you file an issue for that?

Cheers,
Till


On Thu, Jul 16, 2015 at 6:01 PM, Andreas Kunft <[hidden email]>
wrote:

> Hello,
>
> I want to execute a Job from Java on an already running instance of Flink.
> I connect via the RemoteEnvironment. Everything goes fine, but then I get
> an exception from akka telling me the payload size is exceeded:
>
> 15-07-16 17:47:27 [ERROR] Transient association error (association remains
> live)
> akka.remote.OversizedPayloadException: Discarding oversized payload sent
> to Actor[akka.tcp://flink@127.0.0.1:6123/user/jobmanager#697056301]: max
> allowed size 10485760 bytes, actual size of encoded class
> org.apache.flink.runtime.messages.JobManagerMessages$SubmitJob was 28318303
> bytes.
>
> I tried to increase the akka.framesize configuration value but then
> realized the executePlanWithJars method in the RemoteExecutor creates a
> client with a new configuration.
>
> Am I doing something wrong here or how can I provide my own configuration
> to the Client?
>
> Best
> Andreas Kunft
>
>
>