Re: Looks like a ClassLoader bug

Posted by Timo Walther on Jun 18, 2014; 12:20pm
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/Looks-like-a-ClassLoader-bug-tp336p410.html

Meanwhile I don't think that it is a bug, because the getStubWrapper()
method in TaskConfig uses the classloader. userCodeClass already
contains the loaded class, so no need for the classloader:

@Override
     public T getUserCodeObject(Class<? super T> superClass, ClassLoader
cl) {
         return InstantiationUtil.instantiate(userCodeClass, superClass);
     }

Anyways, I'm still having the problem, that my InputFormat does not find
the user class HCatInputFormat. How can I access the user code class
loader from within an input format? Or do I have to serialize my class
within the InputFormat in my custom writeObject() method? Here is my
exception, maybe someone could give me a hint:

eu.stratosphere.client.program.ProgramInvocationException: The program
execution failed:
eu.stratosphere.nephele.executiongraph.GraphConversionException:
java.lang.RuntimeException: Unable to instantiate the hadoop input format
     at
eu.stratosphere.hadoopcompatibility.mapreduce.HadoopInputFormat.readObject(HadoopInputFormat.java:315)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:622)
     at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1001)
     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1892)
     at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1797)
     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1349)
     at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1914)
     at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1797)
     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1349)
     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at
eu.stratosphere.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:239)
     at
eu.stratosphere.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:227)
     at
eu.stratosphere.pact.runtime.task.util.TaskConfig.getStubWrapper(TaskConfig.java:262)
     at
eu.stratosphere.pact.runtime.task.RegularPactTask.instantiateUserCode(RegularPactTask.java:1487)
     at
eu.stratosphere.pact.runtime.task.DataSourceTask.initInputFormat(DataSourceTask.java:313)
     at
eu.stratosphere.pact.runtime.task.DataSourceTask.registerInputOutput(DataSourceTask.java:100)
     at
eu.stratosphere.nephele.execution.RuntimeEnvironment.<init>(RuntimeEnvironment.java:192)
     at
eu.stratosphere.nephele.executiongraph.ExecutionGroupVertex.<init>(ExecutionGroupVertex.java:229)
     at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.createVertex(ExecutionGraph.java:493)
     at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.constructExecutionGraph(ExecutionGraph.java:275)
     at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.<init>(ExecutionGraph.java:174)
     at
eu.stratosphere.nephele.jobmanager.JobManager.submitJob(JobManager.java:499)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:622)
     at eu.stratosphere.nephele.ipc.RPC$Server.call(RPC.java:417)
     at eu.stratosphere.nephele.ipc.Server$Handler.run(Server.java:951)

Caused by: java.lang.ClassNotFoundException:
org.apache.hive.hcatalog.mapreduce.HCatInputFormat
     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
     at java.security.AccessController.doPrivileged(Native Method)
     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
     at java.lang.Class.forName0(Native Method)
     at java.lang.Class.forName(Class.java:270)
     at
eu.stratosphere.hadoopcompatibility.mapreduce.HadoopInputFormat.readObject(HadoopInputFormat.java:313)
     ... 31 more


On 17.06.2014 16:42, Stephan Ewen wrote:
> Yep, this looks like a bug, I agree.
>