Hey,
I'm trying out 1.2 because I need the new Kafka 0.10 connector. I compiled my job using the SNAPSHOT releases of flink 1.2 and the job works fine in local mode. The problem is that deploying the job to our EMR/YARN cluster fails with a ----- java.lang.VerifyError: Cannot inherit from final class at java.lang.ClassLoader.defineClass1(Native Method) ----- The full log is available here: https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017 The job is from my local machine to our EMR cluster. My local machine runs a Flink 1.2 SNAPSHOT. Is this some kind of version conflict? The dependencies for the job are (sbt project): ---- resolvers += "Relayr Repo" at "https://s3-eu-west-1.amazonaws.com/relayr-maven" resolvers += "Apache Snapshots" at "https://repository.apache.org/content/groups/snapshots/" val flinkDependencies = Seq( "org.apache.flink" %% "flink-scala" % flinkVersion % "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), "org.apache.flink" %% "flink-connector-cassandra" % flinkVersion exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), "org.json" % "json" % "20160212", "org.clapper" %% "grizzled-slf4j" % "1.3.0", "ch.qos.logback" % "logback-classic" % "1.1.7" % "provided", "com.datadoghq" % "java-dogstatsd-client" % "2.3", "io.dropwizard.metrics" % "metrics-core" % "3.1.2", "org.scalatest" %% "scalatest" % "2.2.6" % "test" ) ---- How can I debug this further? Thanks, Dominik |
This looks at a first glance like you are mixing up some versions of
different modules. Either some Scala version mix, or Flink dependencies from different versions. Can you double check that? On Thu, Dec 15, 2016 at 5:10 PM, Dominik Bruhn <[hidden email]> wrote: > Hey, > I'm trying out 1.2 because I need the new Kafka 0.10 connector. > > I compiled my job using the SNAPSHOT releases of flink 1.2 and the job > works fine in local mode. > > The problem is that deploying the job to our EMR/YARN cluster fails with a > > ----- > java.lang.VerifyError: Cannot inherit from final class > at java.lang.ClassLoader.defineClass1(Native Method) > ----- > The full log is available here: > https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017 > > > The job is from my local machine to our EMR cluster. My local machine runs > a Flink 1.2 SNAPSHOT. > > Is this some kind of version conflict? > > The dependencies for the job are (sbt project): > ---- > resolvers += "Relayr Repo" at "https://s3-eu-west-1.amazonaw > s.com/relayr-maven" > resolvers += "Apache Snapshots" at "https://repository.apache.org > /content/groups/snapshots/" > > val flinkDependencies = Seq( > "org.apache.flink" %% "flink-scala" % flinkVersion % "provided" > exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), > "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % > "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), > > "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion > exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), > "org.apache.flink" %% "flink-connector-cassandra" % flinkVersion > exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), > > "org.json" % "json" % "20160212", > "org.clapper" %% "grizzled-slf4j" % "1.3.0", > "ch.qos.logback" % "logback-classic" % "1.1.7" % "provided", > "com.datadoghq" % "java-dogstatsd-client" % "2.3", > "io.dropwizard.metrics" % "metrics-core" % "3.1.2", > > "org.scalatest" %% "scalatest" % "2.2.6" % "test" > ) > ---- > > > How can I debug this further? > > Thanks, > Dominik > |
Tracked down the issue at least to some extend:
We are running into a JAR-Hell because the job-jar uses the typesafe config package in version 1.3, but flink enforces 1.1. We are trying to migrate away from typesafe for our configs. Thanks, Dominik On 16.12.2016 10:47, Stephan Ewen wrote: > This looks at a first glance like you are mixing up some versions of > different modules. Either some Scala version mix, or Flink dependencies > from different versions. > > Can you double check that? > > On Thu, Dec 15, 2016 at 5:10 PM, Dominik Bruhn <[hidden email]> wrote: > >> Hey, >> I'm trying out 1.2 because I need the new Kafka 0.10 connector. >> >> I compiled my job using the SNAPSHOT releases of flink 1.2 and the job >> works fine in local mode. >> >> The problem is that deploying the job to our EMR/YARN cluster fails with a >> >> ----- >> java.lang.VerifyError: Cannot inherit from final class >> at java.lang.ClassLoader.defineClass1(Native Method) >> ----- >> The full log is available here: >> https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017 >> >> >> The job is from my local machine to our EMR cluster. My local machine runs >> a Flink 1.2 SNAPSHOT. >> >> Is this some kind of version conflict? >> >> The dependencies for the job are (sbt project): >> ---- >> resolvers += "Relayr Repo" at "https://s3-eu-west-1.amazonaw >> s.com/relayr-maven" >> resolvers += "Apache Snapshots" at "https://repository.apache.org >> /content/groups/snapshots/" >> >> val flinkDependencies = Seq( >> "org.apache.flink" %% "flink-scala" % flinkVersion % "provided" >> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), >> "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % >> "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), >> >> "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion >> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), >> "org.apache.flink" %% "flink-connector-cassandra" % flinkVersion >> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"), >> >> "org.json" % "json" % "20160212", >> "org.clapper" %% "grizzled-slf4j" % "1.3.0", >> "ch.qos.logback" % "logback-classic" % "1.1.7" % "provided", >> "com.datadoghq" % "java-dogstatsd-client" % "2.3", >> "io.dropwizard.metrics" % "metrics-core" % "3.1.2", >> >> "org.scalatest" %% "scalatest" % "2.2.6" % "test" >> ) >> ---- >> >> >> How can I debug this further? >> >> Thanks, >> Dominik >> > |
Free forum by Nabble | Edit this page |