Stephan Ewen created FLINK-1325:
-----------------------------------
Summary: Add a closure cleaner for Java
Key: FLINK-1325
URL:
https://issues.apache.org/jira/browse/FLINK-1325 Project: Flink
Issue Type: Improvement
Components: Java API
Reporter: Stephan Ewen
The Java API could really need a simple closure cleaner.
All functions that are implemented as anonymous subclasses hold a reference to the enclosing class, unless they are implemented as part of a static method.
That reference (called {{this$0}}) causes serialization to fail, as it draws non serializable classes into the function, even in cases where the function makes no access to the enclosing data.
It is possible to manually set this reference to {{null}}, using reflection, or using a debugger. Then the serialization succeeds.
I suggest to add a closure cleaner that uses an ASM visitor over the function's code to see if there is any access to the {{this$0}} field. In case there is non, the field should be set to {{null}}.
The problem can be reproduced with the simple program below:
{code}
public class Test {
public void runProgram() throws Exception {
ExecutionEnvironment env = ExecutionEnvironment .getExecutionEnvironment();
env.generateSequence(1, 10)
.map(new MapFunction<Long, Long>() {
public Long map(Long value) {
return value * 2;
}
})
.print();
env.execute();
}
public static void main(String[] args) throws Exception {
new Test().runProgram();
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
