[jira] [Created] (FLINK-14876) Putting xercesImpl related classes into alwaysParentFirstLoaderPatterns

[Shang Yuanchun (Jira)]

liupengcheng created FLINK-14876:
------------------------------------

             Summary: Putting xercesImpl related classes into alwaysParentFirstLoaderPatterns
                 Key: FLINK-14876
                 URL: https://issues.apache.org/jira/browse/FLINK-14876
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Configuration
    Affects Versions: 1.9.0
         Environment: hadoop2.6

 
            Reporter: liupengcheng


As mentioned in the [9683|[https://github.com/apache/flink/pull/9683]]  , There is a issue when running jobs when including flink-shaded-hadoop-2 package while changing the `hadoop.version` to our cluster hadoop version.

I reproducing this case, the following exception was reported:
{code:java}
javax.xml.parsers.FactoryConfigurationError: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be createdjavax.xml.parsers.FactoryConfigurationError: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:311) at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:267) at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:120) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2412) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2375) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2285) at org.apache.hadoop.conf.Configuration.get(Configuration.java:892) at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:2010) at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:449) at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:186) at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:167) at org.apache.flink.hadoopcompatibility.scala.HadoopInputs$.readHadoopFile(HadoopInputs.scala:127) at com.github.ehiggs.spark.terasort.FlinkTeraSort$.main(FlinkTeraSort.scala:76) at com.github.ehiggs.spark.terasort.FlinkTeraSort.main(FlinkTeraSort.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:586) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:448) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746) at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273) at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205) at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1009) at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1082) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886) at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1082)Caused by: java.lang.RuntimeException: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:308) ... 30 moreCaused by: java.util.ServiceConfigurationError: javax.xml.parsers.DocumentBuilderFactory: Provider org.apache.xerces.jaxp.DocumentBuilderFactoryImpl not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:239) at java.util.ServiceLoader.access$300(ServiceLoader.java:185) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.xml.parsers.FactoryFinder$1.run(FactoryFinder.java:294) at java.security.AccessController.doPrivileged(Native Method) at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:289) ... 30 more
{code}
I enabled `-vebose:class` for client, and find out that the `DocumentBuilderFactoryImpl` class was loaded twice. The first time it was loaded is because the flink app starts and initialize the configuration and print something, the second time it was loaded is because the calling of `HadoopInputs.readHadoopFile` in the user code.
{code:java}
[Loaded javax.xml.parsers.DocumentBuilderFactory from /opt/soft/openjdk8u202-b08/jre/lib/rt.jar]
[Loaded org.apache.xerces.jaxp.DocumentBuilderFactoryImpl from file:/home/liupengcheng/git/infra-client/bin/packages/common-infra_client-pack-zjyprc-hadoop/bin/packages/zjyprc-hadoop-flink1.9-hadoop-pack-2.6.0-mdh2.6.0.4/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar]

[Loaded javax.xml.parsers.DocumentBuilderFactory from file:/home/liupengcheng/git/infra-client/bin/packages/common-infra_client-pack-zjyprc-hadoop/bin/packages/flink-1.9.0-mdh1.9.0.0-SNAPSHOT/../benchmark-test-1.1-SNAPSHOT-shaded.jar]
[Loaded org.apache.xerces.jaxp.DocumentBuilderFactoryImpl from file:/home/liupengcheng/git/infra-client/bin/packages/common-infra_client-pack-zjyprc-hadoop/bin/packages/flink-1.9.0-mdh1.9.0.0-SNAPSHOT/../benchmark-test-1.1-SNAPSHOT-shaded.jar]
{code}
{code:java}
val dataSet = env.createInput(HadoopInputs.readHadoopFile(
  new TeraInputFormat, classOf[Array[Byte]], classOf[Array[Byte]], inputFile))
  .partitionCustom(new FlinkTeraSortPartitioner(new TeraSortPartitioner(partitions)), 0)
  .sortPartition(0, Order.ASCENDING)
{code}
when calling `HadoopInputs.readhadoopFile`, it will instantiate `jobConf` and call it's `get` method, thus will finally causing the loading of `DocumentBuilderFactoryImpl`.

After loading `DocumentBuilderFactoryImpl`, it will try to check whether it's subtype of `DocumentBuilderFactory`. Here, it' will report error due to that the `DocumentBuilderFacotory` is loaded from `HADOOP_CLASSPATH`, not the user code.
{code:java}
public static DocumentBuilderFactory newInstance() {
    return FactoryFinder.find(
            /* The default property name according to the JAXP spec */
            DocumentBuilderFactory.class, // "javax.xml.parsers.DocumentBuilderFactory"
            /* The fallback implementation class name */
            "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");
}
{code}
I found that  the `xercesImpl` package was introduced by the `flink-shaded-hadoop-2`, because it shaded in `xercesImpl` from hadoop-hdfs deps.

 

Moreover, this `xcercesImpl` is included in hadoop-hdfs for all 2.6 ~ 3.0 versions, I think we can add it to the `alwaysParentFirstPatterns` to avoid this problem. 

cc [~aljoscha]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)