Andrey Zagrebin created FLINK-16406:
---------------------------------------
Summary: Increase default value for JVM Metaspace to minimise its OutOfMemoryError
Key: FLINK-16406
URL:
https://issues.apache.org/jira/browse/FLINK-16406 Project: Flink
Issue Type: Improvement
Components: Runtime / Configuration, Runtime / Task
Affects Versions: 1.10.0
Reporter: Andrey Zagrebin
Assignee: Andrey Zagrebin
Fix For: 1.10.1, 1.11.0
With FLIP-49 ([FLINK-13980|
https://issues.apache.org/jira/browse/FLINK-13980]), we introduced a limit for JVM Metaspace ('taskmanager.memory.jvm-metaspace.size') when TM JVM process is started. It caused '_OutOfMemoryError: Metaspace_' for some use cases after upgrading to the latest 1.10 version. In some cases, a real class loading leak has been discovered, like in [FLINK-16142|
https://issues.apache.org/jira/browse/FLINK-16142]. Some users had to increase the default value to accommodate for their use cases (mostly from 96Mb to 256Mb).
While this limit was introduced to properly plan Flink resources, especially for container environment, and to detect class loading leaks, the user experience should be as smooth as possible. One way is provide good documentation for this change ([FLINK-16278|
https://issues.apache.org/jira/browse/FLINK-16278]).
Another question is the sanity of the default value. It is still arguable what the default value should be (currently 96Mb). In general, the size depends on the use case (job user code, how many jobs are deployed in the cluster etc).
This issue tries to tackle this problem by firstly increasing it to 256Mb. We also want to poll which Metaspace setting resolved the _OutOfMemoryError_. Please, if you encountered this problem, report here any relevant specifics of your job and your Metaspace size if there was no class loading leak.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
