Jingsong Lee created FLINK-19916:
------------------------------------
Summary: Hadoop3 ShutdownHookManager visit closed ClassLoader
Key: FLINK-19916
URL:
https://issues.apache.org/jira/browse/FLINK-19916 Project: Flink
Issue Type: Bug
Components: Connectors / Hadoop Compatibility
Environment: {code:java}
Exception in thread "Thread-10" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:161)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:179)
at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2780)
at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3036)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2995)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2968)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1200)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1812)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1789)
at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)
{code}
This is because Hadoop 3 starts asynchronous threads to execute some shutdown hooks.
These hooks are run after the job is executed, as a result, the classloader has been released, but in hooks, configuration still holds the released classloader, so it will fail to throw an exception in this asynchronous thread.
Now it doesn't affect our function, it just prints the exception stack on the console.
Reporter: Jingsong Lee
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
