maven dependency problem when building stream job

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

maven dependency problem when building stream job

Martin Neumann-2
Hi,

I have a weird problem. Yesterday I had to clean my local maven cache for a
different application.
Since afterwards one of my Flink streaming jobs does not compile anymore. I
didn't change any code just made maven pull all dependencies again.

I'm totally stomped by this, please help me!

Here is the error I get from maven:

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for io.netty:netty:3.7.0.Final paths
to dependency are:
+-com.spotify:logproc-flink:0.0.0-SNAPSHOT
  +-org.apache.flink:flink-java:0.10.1
    +-org.apache.flink:flink-shaded-hadoop2:0.10.1
      +-org.apache.zookeeper:zookeeper:3.4.6
        +-io.netty:netty:3.7.0.Final
and
+-com.spotify:logproc-flink:0.0.0-SNAPSHOT
  +-org.apache.flink:flink-streaming-java:0.10.1
    +-org.apache.flink:flink-runtime:0.10.1
      +-com.typesafe.akka:akka-remote_2.10:2.3.7
        +-io.netty:netty:3.8.0.Final
]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce) on
project logproc-flink: Some Enforcer rules have failed. Look above for
specific messages explaining why the rule failed. -> [Help 1]
Reply | Threaded
Open this post in threaded view
|

Re: maven dependency problem when building stream job

Robert Metzger
Hi Martin,

is the logproc-flink project using the maven-enforcer-plugin to enforce a
minimum version of netty?
We recently downgraded (a minor version) of netty because of an issue.
Maybe that's the issue.

Can you check the enforcer rules of your project?

On Wed, Jan 20, 2016 at 1:48 PM, Martin Neumann <[hidden email]> wrote:

> Hi,
>
> I have a weird problem. Yesterday I had to clean my local maven cache for a
> different application.
> Since afterwards one of my Flink streaming jobs does not compile anymore. I
> didn't change any code just made maven pull all dependencies again.
>
> I'm totally stomped by this, please help me!
>
> Here is the error I get from maven:
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
> failed with message:
> Failed while enforcing RequireUpperBoundDeps. The error(s) are [
> Require upper bound dependencies error for io.netty:netty:3.7.0.Final paths
> to dependency are:
> +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
>   +-org.apache.flink:flink-java:0.10.1
>     +-org.apache.flink:flink-shaded-hadoop2:0.10.1
>       +-org.apache.zookeeper:zookeeper:3.4.6
>         +-io.netty:netty:3.7.0.Final
> and
> +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
>   +-org.apache.flink:flink-streaming-java:0.10.1
>     +-org.apache.flink:flink-runtime:0.10.1
>       +-com.typesafe.akka:akka-remote_2.10:2.3.7
>         +-io.netty:netty:3.8.0.Final
> ]
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce) on
> project logproc-flink: Some Enforcer rules have failed. Look above for
> specific messages explaining why the rule failed. -> [Help 1]
>
Reply | Threaded
Open this post in threaded view
|

Re: maven dependency problem when building stream job

Martin Neumann-2
Its using a maven-enforcer-plugin (see config below). The project itself
does not use netty.
I just discovered that I still compile fine from within IntelliJ I just
cannot built it externally with maven (which I need to package the jar to
deploy on Yarn).

cheers Martin

<plugin>
  <artifactId>maven-enforcer-plugin</artifactId>
  <version>1.4.1</version>
  <executions>
    <execution>
      <id>enforce</id>
      <configuration>
        <rules>
          <requireUpperBoundDeps />
        </rules>
        <fail>true</fail>
      </configuration>
      <goals>
        <goal>enforce</goal>
      </goals>
    </execution>
  </executions>

</plugin>




On Wed, Jan 20, 2016 at 2:04 PM, Robert Metzger <[hidden email]> wrote:

> Hi Martin,
>
> is the logproc-flink project using the maven-enforcer-plugin to enforce a
> minimum version of netty?
> We recently downgraded (a minor version) of netty because of an issue.
> Maybe that's the issue.
>
> Can you check the enforcer rules of your project?
>
> On Wed, Jan 20, 2016 at 1:48 PM, Martin Neumann <[hidden email]> wrote:
>
> > Hi,
> >
> > I have a weird problem. Yesterday I had to clean my local maven cache
> for a
> > different application.
> > Since afterwards one of my Flink streaming jobs does not compile
> anymore. I
> > didn't change any code just made maven pull all dependencies again.
> >
> > I'm totally stomped by this, please help me!
> >
> > Here is the error I get from maven:
> >
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
> > failed with message:
> > Failed while enforcing RequireUpperBoundDeps. The error(s) are [
> > Require upper bound dependencies error for io.netty:netty:3.7.0.Final
> paths
> > to dependency are:
> > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> >   +-org.apache.flink:flink-java:0.10.1
> >     +-org.apache.flink:flink-shaded-hadoop2:0.10.1
> >       +-org.apache.zookeeper:zookeeper:3.4.6
> >         +-io.netty:netty:3.7.0.Final
> > and
> > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> >   +-org.apache.flink:flink-streaming-java:0.10.1
> >     +-org.apache.flink:flink-runtime:0.10.1
> >       +-com.typesafe.akka:akka-remote_2.10:2.3.7
> >         +-io.netty:netty:3.8.0.Final
> > ]
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce) on
> > project logproc-flink: Some Enforcer rules have failed. Look above for
> > specific messages explaining why the rule failed. -> [Help 1]
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: maven dependency problem when building stream job

Robert Metzger
Hi Martin.

can you try to exclude the netty dependency from your Flink dependencies?
Another approach would be to disable the check, or add an exception to it ;)

Why did you add the check in the first place?


On Wed, Jan 20, 2016 at 2:13 PM, Martin Neumann <[hidden email]> wrote:

> Its using a maven-enforcer-plugin (see config below). The project itself
> does not use netty.
> I just discovered that I still compile fine from within IntelliJ I just
> cannot built it externally with maven (which I need to package the jar to
> deploy on Yarn).
>
> cheers Martin
>
> <plugin>
>   <artifactId>maven-enforcer-plugin</artifactId>
>   <version>1.4.1</version>
>   <executions>
>     <execution>
>       <id>enforce</id>
>       <configuration>
>         <rules>
>           <requireUpperBoundDeps />
>         </rules>
>         <fail>true</fail>
>       </configuration>
>       <goals>
>         <goal>enforce</goal>
>       </goals>
>     </execution>
>   </executions>
>
> </plugin>
>
>
>
>
> On Wed, Jan 20, 2016 at 2:04 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > Hi Martin,
> >
> > is the logproc-flink project using the maven-enforcer-plugin to enforce a
> > minimum version of netty?
> > We recently downgraded (a minor version) of netty because of an issue.
> > Maybe that's the issue.
> >
> > Can you check the enforcer rules of your project?
> >
> > On Wed, Jan 20, 2016 at 1:48 PM, Martin Neumann <[hidden email]>
> wrote:
> >
> > > Hi,
> > >
> > > I have a weird problem. Yesterday I had to clean my local maven cache
> > for a
> > > different application.
> > > Since afterwards one of my Flink streaming jobs does not compile
> > anymore. I
> > > didn't change any code just made maven pull all dependencies again.
> > >
> > > I'm totally stomped by this, please help me!
> > >
> > > Here is the error I get from maven:
> > >
> > > [WARNING] Rule 0:
> org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
> > > failed with message:
> > > Failed while enforcing RequireUpperBoundDeps. The error(s) are [
> > > Require upper bound dependencies error for io.netty:netty:3.7.0.Final
> > paths
> > > to dependency are:
> > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > >   +-org.apache.flink:flink-java:0.10.1
> > >     +-org.apache.flink:flink-shaded-hadoop2:0.10.1
> > >       +-org.apache.zookeeper:zookeeper:3.4.6
> > >         +-io.netty:netty:3.7.0.Final
> > > and
> > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > >   +-org.apache.flink:flink-streaming-java:0.10.1
> > >     +-org.apache.flink:flink-runtime:0.10.1
> > >       +-com.typesafe.akka:akka-remote_2.10:2.3.7
> > >         +-io.netty:netty:3.8.0.Final
> > > ]
> > > [ERROR] Failed to execute goal
> > > org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce)
> on
> > > project logproc-flink: Some Enforcer rules have failed. Look above for
> > > specific messages explaining why the rule failed. -> [Help 1]
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: maven dependency problem when building stream job

Martin Neumann-2
Its a fork of a different project, so I didn't add it myself. I disabled it
for now and so far it works, I will try to find out why it was there in the
first place and if necessary find a workaround.

thanks for the help
Martin

On Wed, Jan 20, 2016 at 2:32 PM, Robert Metzger <[hidden email]> wrote:

> Hi Martin.
>
> can you try to exclude the netty dependency from your Flink dependencies?
> Another approach would be to disable the check, or add an exception to it
> ;)
>
> Why did you add the check in the first place?
>
>
> On Wed, Jan 20, 2016 at 2:13 PM, Martin Neumann <[hidden email]> wrote:
>
> > Its using a maven-enforcer-plugin (see config below). The project itself
> > does not use netty.
> > I just discovered that I still compile fine from within IntelliJ I just
> > cannot built it externally with maven (which I need to package the jar to
> > deploy on Yarn).
> >
> > cheers Martin
> >
> > <plugin>
> >   <artifactId>maven-enforcer-plugin</artifactId>
> >   <version>1.4.1</version>
> >   <executions>
> >     <execution>
> >       <id>enforce</id>
> >       <configuration>
> >         <rules>
> >           <requireUpperBoundDeps />
> >         </rules>
> >         <fail>true</fail>
> >       </configuration>
> >       <goals>
> >         <goal>enforce</goal>
> >       </goals>
> >     </execution>
> >   </executions>
> >
> > </plugin>
> >
> >
> >
> >
> > On Wed, Jan 20, 2016 at 2:04 PM, Robert Metzger <[hidden email]>
> > wrote:
> >
> > > Hi Martin,
> > >
> > > is the logproc-flink project using the maven-enforcer-plugin to
> enforce a
> > > minimum version of netty?
> > > We recently downgraded (a minor version) of netty because of an issue.
> > > Maybe that's the issue.
> > >
> > > Can you check the enforcer rules of your project?
> > >
> > > On Wed, Jan 20, 2016 at 1:48 PM, Martin Neumann <[hidden email]>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I have a weird problem. Yesterday I had to clean my local maven cache
> > > for a
> > > > different application.
> > > > Since afterwards one of my Flink streaming jobs does not compile
> > > anymore. I
> > > > didn't change any code just made maven pull all dependencies again.
> > > >
> > > > I'm totally stomped by this, please help me!
> > > >
> > > > Here is the error I get from maven:
> > > >
> > > > [WARNING] Rule 0:
> > org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
> > > > failed with message:
> > > > Failed while enforcing RequireUpperBoundDeps. The error(s) are [
> > > > Require upper bound dependencies error for io.netty:netty:3.7.0.Final
> > > paths
> > > > to dependency are:
> > > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > > >   +-org.apache.flink:flink-java:0.10.1
> > > >     +-org.apache.flink:flink-shaded-hadoop2:0.10.1
> > > >       +-org.apache.zookeeper:zookeeper:3.4.6
> > > >         +-io.netty:netty:3.7.0.Final
> > > > and
> > > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > > >   +-org.apache.flink:flink-streaming-java:0.10.1
> > > >     +-org.apache.flink:flink-runtime:0.10.1
> > > >       +-com.typesafe.akka:akka-remote_2.10:2.3.7
> > > >         +-io.netty:netty:3.8.0.Final
> > > > ]
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce
> (enforce)
> > on
> > > > project logproc-flink: Some Enforcer rules have failed. Look above
> for
> > > > specific messages explaining why the rule failed. -> [Help 1]
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: maven dependency problem when building stream job

Stephan Ewen
Hi Martin!

It looks like your project added a rule to not allow a higher netty.io
version. Any specific reason that the rule was added? Would be good to
understand that, because the rule stands in conflict with Flink's
dependencies.

Flink internally has multiple uses of netty: Akka remote, Flink network
stack, ZooKeeper. It has had those dependencies for quite a while, so I
guess it was not a recent change in Flink, but probably a change in
the "logproc-flink"
pom.xml file.

Stephan


On Wed, Jan 20, 2016 at 2:43 PM, Martin Neumann <[hidden email]> wrote:

> Its a fork of a different project, so I didn't add it myself. I disabled it
> for now and so far it works, I will try to find out why it was there in the
> first place and if necessary find a workaround.
>
> thanks for the help
> Martin
>
> On Wed, Jan 20, 2016 at 2:32 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > Hi Martin.
> >
> > can you try to exclude the netty dependency from your Flink dependencies?
> > Another approach would be to disable the check, or add an exception to it
> > ;)
> >
> > Why did you add the check in the first place?
> >
> >
> > On Wed, Jan 20, 2016 at 2:13 PM, Martin Neumann <[hidden email]>
> wrote:
> >
> > > Its using a maven-enforcer-plugin (see config below). The project
> itself
> > > does not use netty.
> > > I just discovered that I still compile fine from within IntelliJ I just
> > > cannot built it externally with maven (which I need to package the jar
> to
> > > deploy on Yarn).
> > >
> > > cheers Martin
> > >
> > > <plugin>
> > >   <artifactId>maven-enforcer-plugin</artifactId>
> > >   <version>1.4.1</version>
> > >   <executions>
> > >     <execution>
> > >       <id>enforce</id>
> > >       <configuration>
> > >         <rules>
> > >           <requireUpperBoundDeps />
> > >         </rules>
> > >         <fail>true</fail>
> > >       </configuration>
> > >       <goals>
> > >         <goal>enforce</goal>
> > >       </goals>
> > >     </execution>
> > >   </executions>
> > >
> > > </plugin>
> > >
> > >
> > >
> > >
> > > On Wed, Jan 20, 2016 at 2:04 PM, Robert Metzger <[hidden email]>
> > > wrote:
> > >
> > > > Hi Martin,
> > > >
> > > > is the logproc-flink project using the maven-enforcer-plugin to
> > enforce a
> > > > minimum version of netty?
> > > > We recently downgraded (a minor version) of netty because of an
> issue.
> > > > Maybe that's the issue.
> > > >
> > > > Can you check the enforcer rules of your project?
> > > >
> > > > On Wed, Jan 20, 2016 at 1:48 PM, Martin Neumann <[hidden email]>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I have a weird problem. Yesterday I had to clean my local maven
> cache
> > > > for a
> > > > > different application.
> > > > > Since afterwards one of my Flink streaming jobs does not compile
> > > > anymore. I
> > > > > didn't change any code just made maven pull all dependencies again.
> > > > >
> > > > > I'm totally stomped by this, please help me!
> > > > >
> > > > > Here is the error I get from maven:
> > > > >
> > > > > [WARNING] Rule 0:
> > > org.apache.maven.plugins.enforcer.RequireUpperBoundDeps
> > > > > failed with message:
> > > > > Failed while enforcing RequireUpperBoundDeps. The error(s) are [
> > > > > Require upper bound dependencies error for
> io.netty:netty:3.7.0.Final
> > > > paths
> > > > > to dependency are:
> > > > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > > > >   +-org.apache.flink:flink-java:0.10.1
> > > > >     +-org.apache.flink:flink-shaded-hadoop2:0.10.1
> > > > >       +-org.apache.zookeeper:zookeeper:3.4.6
> > > > >         +-io.netty:netty:3.7.0.Final
> > > > > and
> > > > > +-com.spotify:logproc-flink:0.0.0-SNAPSHOT
> > > > >   +-org.apache.flink:flink-streaming-java:0.10.1
> > > > >     +-org.apache.flink:flink-runtime:0.10.1
> > > > >       +-com.typesafe.akka:akka-remote_2.10:2.3.7
> > > > >         +-io.netty:netty:3.8.0.Final
> > > > > ]
> > > > > [ERROR] Failed to execute goal
> > > > > org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce
> > (enforce)
> > > on
> > > > > project logproc-flink: Some Enforcer rules have failed. Look above
> > for
> > > > > specific messages explaining why the rule failed. -> [Help 1]
> > > > >
> > > >
> > >
> >
>