Maximilian Michels created FLINK-1843:
-----------------------------------------
Summary: Job History gets cleared too fast
Key: FLINK-1843
URL:
https://issues.apache.org/jira/browse/FLINK-1843 Project: Flink
Issue Type: Bug
Components: JobManager
Affects Versions: 0.9
Reporter: Maximilian Michels
Fix For: 0.9
As per FLINK-1442, the JobManager stores the archived ExecutionGraph behind a SoftReference. At least for local setups, this mechanism doesn't seem to work properly. There are two issues:
- The history gets cleared too fast
- The history gets cleared in a non-sequential fashion, i.e. arbitrary old ExecutionGraph are discarded
To solve these problems we might
- Store the least recent ExecutionGraph behind a SoftReference
- Store the most recent ExecutionGraphs without a SoftReference
That way, we can save memory but have the latest history available to the user. We might introduce a configuration variable where the user can specify the number of ExecutionGraphs that should be held in memory. The remaining can be stored behind a SoftReference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)