Posted by
Shang Yuanchun (Jira) on
Jun 12, 2014; 7:21pm
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/jira-Commented-FLINK-934-Job-and-TaskManager-startup-scripts-always-set-JVM-heap-size-tp133.html
[
https://issues.apache.org/jira/browse/FLINK-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029645#comment-14029645 ]
ASF GitHub Bot commented on FLINK-934:
--------------------------------------
GitHub user uce opened a pull request:
https://github.com/apache/incubator-flink/pull/14 [FLINK-934] Remove default values for JVM heap sizes in startup scripts
This is a fix for [FLINK-934](
https://issues.apache.org/jira/browse/FLINK-934). As a follow up we should have the heap config keys only as suggested entries (commented out) in [FLINK-898](
https://issues.apache.org/jira/browse/FLINK-898) aka #6.
With `jobmanager.heap.mb` and `taskmanager.heap.mb` not set, 16 GB of main memory and Java 7.
```bash
$ ./bin/start-local.sh
Starting job manager
$ jps
9601 JobManager
$ jinfo -flag MaxHeapSize 9601
-XX:MaxHeapSize=4294967296
```
JobManager with 4294967296/2^20 = 4096 MB heap size.
---
```bash
$ bin/start-cluster.sh
Starting job manager
Starting task manager on host batman
$ jps
10049 TaskManager
9966 JobManager
$ jinfo -flag MaxHeapSize 9966
-XX:MaxHeapSize=4294967296
$jinfo -flag MaxHeapSize 10049
-XX:MaxHeapSize=4294967296
```
Both JobManager and TaskManager with 4294967296/2^20 = 4096 MB heap size each.
---
Setting `jobmanager.heap.mb` and `taskmanager.heap.mb` to 256 and 512 MB respectively.
```bash
$ bin/start-local.sh
Starting job manager
$ jps
10542 JobManager
$ jinfo -flag MaxHeapSize 10542
-XX:MaxHeapSize=805306368
```
JobManager with 805306368/2^20 = 768 MB (256 for the JobManager and 512 MB for the TaskManager).
---
```bash
$ bin/start-cluster.sh
Starting job manager
Starting task manager on host batman
$ jps
10833 JobManager
10918 TaskManager
$ jinfo -flag MaxHeapSize 10833
-XX:MaxHeapSize=268435456
$jinfo -flag MaxHeapSize 10918
-XX:MaxHeapSize=536870912
```
JobManager with 268435456/2^20 = 256 MB and TaskManager with 536870912/2^20 = 512 MB.
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/uce/incubator-flink FLINK-934
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/14.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14
----
commit f239909c57d16ad109767f9342f18b314fd412e4
Author: uce <
[hidden email]>
Date: 2014-06-12T19:08:06Z
[FLINK-934] Remove default values for JobManager and TaskManager JVM heap sizes in startup scripts
----
> Job and TaskManager startup scripts always set JVM heap size
> ------------------------------------------------------------
>
> Key: FLINK-934
> URL:
https://issues.apache.org/jira/browse/FLINK-934> Project: Flink
> Issue Type: Improvement
> Reporter: Ufuk Celebi
>
> The default distribution config {{stratosphere-conf.yaml}} contains keys {{jobmanager.heap.mb}} and {{taskmanager.heap.mb}} for the heap sizes of the JobManager and TaskManager JVMs. If they are not set, {{config.sh}} has default values for them.
> We _always_ use one of these values and set the {{Xms}} and {{Xmx}} properties. After a short discussion with [~rmetzger] and [~StephanEwen] we actually expected that they should not be set if not found in {{stratosphere-conf.yaml}}. This would result in a fall back to the respective JVM default (imho 1/4th of the physical memory since Java 6).
--
This message was sent by Atlassian JIRA
(v6.2#6252)