Yun Gao created FLINK-13752:
-------------------------------
Summary: TaskDeploymentDescriptor cannot be recycled by GC due to referenced by an anonymous function
Key: FLINK-13752
URL:
https://issues.apache.org/jira/browse/FLINK-13752 Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.9.0
Reporter: Yun Gao
When comparing the 1.8 and 1.9.0-rc2 on a test streaming job, we found that the performance on 1.9.0-rc2 is much lower than that of 1.8. By comparing the two versions, we found that the count of Full GC on 1.9.0-rc2 is much more than that on 1.8.
A further analysis found that the difference is due to in TaskExecutor#setupResultPartitionBookkeeping, the anonymous function in taskTermimationWithResourceCleanFuture has referenced the TaskDeploymentDescriptor, since this function will be kept till the task is terminated, TaskDeploymentDescriptor will also be kept referenced in the closure and cannot be recycled by GC. In this job, TaskDeploymentDescriptor of some tasks are as large as 10M, and the total heap is about 113M, thus the kept TaskDeploymentDescriptors will cause relatively large impact on GC and performance.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
Locked
