[jira] [Commented] (FLINK-934) Job and TaskManager startup scripts always set JVM heap size

[Shang Yuanchun (Jira)]

    [ https://issues.apache.org/jira/browse/FLINK-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031071#comment-14031071 ]

ASF GitHub Bot commented on FLINK-934:
--------------------------------------

GitHub user rmetzger opened a pull request:

    https://github.com/apache/incubator-flink/pull/18

    Do not limit the client's heap space (See FLINK-934)

    Similar to  FLINK-934 and https://github.com/apache/incubator-flink/pull/14 we should not limit the heap space of the job submission client to 512MB.
   
    A user actually had the following exception while using Flink:
    ```
    java.lang.OutOfMemoryError: Java heap space
    at eu.stratosphere.runtime.io.serialization.DataOutputSerializer.resize(DataOutputSerializer.java:243)
    at eu.stratosphere.runtime.io.serialization.DataOutputSerializer.write(DataOutputSerializer.java:87)
    at eu.stratosphere.nephele.jobgraph.JobGraph.writeRequiredJarFiles(JobGraph.java:706)
    at eu.stratosphere.nephele.jobgraph.JobGraph.write(JobGraph.java:641)
    at eu.stratosphere.nephele.ipc.RPC$Invocation.write(RPC.java:153)
    at eu.stratosphere.nephele.ipc.Client$Connection.sendParam(Client.java:469)
    at eu.stratosphere.nephele.ipc.Client.call(Client.java:689)
    at eu.stratosphere.nephele.ipc.RPC$Invoker.invoke(RPC.java:250)
    at com.sun.proxy.$Proxy1.submitJob(Unknown Source)
    at eu.stratosphere.nephele.client.JobClient.submitJobAndWait(JobClient.java:258)
    at eu.stratosphere.client.program.Client.run(Client.java:284)
    at eu.stratosphere.client.program.Client.run(Client.java:268)
    at eu.stratosphere.client.program.Client.run(Client.java:262)
    at eu.stratosphere.client.program.ContextEnvironment.execute(ContextEnvironment.java:50)
    at eu.stratosphere.example.java.wordcount.WordCount.main(WordCount.java:79)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
    at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
    at eu.stratosphere.client.program.Client.run(Client.java:220)
    at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
    at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
    at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
    at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
    ```
   
    I verified that the change resolves the problem.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rmetzger/incubator-flink stratosphere_client_fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-flink/pull/18.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #18
   
----
commit 8e1bff41a21d75f5e7dc78fbc5c30835a3802781
Author: Robert Metzger <[hidden email]>
Date:   2014-06-13T19:40:34Z

    Do not limit the client's heap space (See FLINK-934) [ci skip]

----

> Job and TaskManager startup scripts always set JVM heap size
> ------------------------------------------------------------
>
>                 Key: FLINK-934
>                 URL: https://issues.apache.org/jira/browse/FLINK-934
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
>             Fix For: 0.6-incubating
>
>
> The default distribution config {{stratosphere-conf.yaml}} contains keys {{jobmanager.heap.mb}} and {{taskmanager.heap.mb}} for the heap sizes of the JobManager and TaskManager JVMs. If they are not set, {{config.sh}} has default values for them.
> We _always_ use one of these values and set the {{Xms}} and {{Xmx}} properties. After a short discussion with [~rmetzger] and [~StephanEwen] we actually expected that they should not be set if not found in {{stratosphere-conf.yaml}}. This would result in a fall back to the respective JVM default (imho 1/4th of the physical memory since Java 6).

--
This message was sent by Atlassian JIRA
(v6.2#6252)