Hi!
I was working on FLINK-1522 <https://issues.apache.org/jira/browse/FLINK-1522> and FLINK-1576 <https://issues.apache.org/jira/browse/FLINK-1576> and started converting Gelly library examples to work with file inputs. I added a test for LabelPropagationExample and got failures in Travis when hadoop.profile was 1 (Travis build <https://travis-ci.org/balidani/flink/builds/51440170>). The error was the following: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project flink-gelly: Compilation failure: Compilation failure: [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[21,30] package com.google.common.base does not exist [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[22,28] package com.google.common.io does not exist [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[80,53] cannot find symbol Then I looked at flink-tests and saw that there was a `profiles` part in the POM that adds the dependency to guava for hadoop profile 1. I added this to the POM of Gelly and it seems to fix the issue. (Travis build <https://travis-ci.org/balidani/flink/builds/51496196>) (There is a failure, but the log says it's a timeout) Is this the correct way to solve this error? Thanks, Daniel |
Hey Daniel,
welcome to the shady world of shading maven dependencies (and Hadoop not shading its super old guava version ;) ). I'll have a look at your branch. I think Aljoscha added the guava dependency for the test scope of the hadoop1 profile. He made this change when he moved the HadoopInputFormat support into the "flink-java" package. It seems that his tests also failed due to some dependency conflict with hadoop. Maybe can explain the details of the issue. In general, we use these maven build profiles ( http://maven.apache.org/guides/introduction/introduction-to-profiles.html) to control which modules of flink we are building and which dependencies we are adding. For example in the hadoop2 build, we add the "flink-yarn" and "flink-yarn-tests" modules. Also, the names of the hadoop dependencies have different names for hd1 and hd2. I'll get back to you once I've took a closer look into the issue. Robert On Fri, Feb 20, 2015 at 1:05 PM, Dániel Bali <[hidden email]> wrote: > Hi! > > I was working on FLINK-1522 > <https://issues.apache.org/jira/browse/FLINK-1522> and FLINK-1576 > <https://issues.apache.org/jira/browse/FLINK-1576> and started converting > Gelly library examples to work with file inputs. I added a test for > LabelPropagationExample and got failures in Travis when hadoop.profile was > 1 (Travis build <https://travis-ci.org/balidani/flink/builds/51440170>). > > The error was the following: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile > (default-testCompile) on project flink-gelly: Compilation failure: > Compilation failure: > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[21,30] > package com.google.common.base does not exist > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[22,28] > package com.google.common.io does not exist > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[80,53] > cannot find symbol > > Then I looked at flink-tests and saw that there was a `profiles` part in > the POM that adds the dependency to guava for hadoop profile 1. I added > this to the POM of Gelly and it seems to fix the issue. (Travis build > <https://travis-ci.org/balidani/flink/builds/51496196>) (There is a > failure, but the log says it's a timeout) > > Is this the correct way to solve this error? > > Thanks, > > Daniel > |
In reply to this post by Dániel Bali
Did you set the scope to provided? As in:
<dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>${guava.version}</version> <scope>provided</scope> </dependency> Then this should be a good solution, yes. On Fri, Feb 20, 2015 at 1:05 PM, Dániel Bali <[hidden email]> wrote: > Hi! > > I was working on FLINK-1522 > <https://issues.apache.org/jira/browse/FLINK-1522> and FLINK-1576 > <https://issues.apache.org/jira/browse/FLINK-1576> and started converting > Gelly library examples to work with file inputs. I added a test for > LabelPropagationExample and got failures in Travis when hadoop.profile was > 1 (Travis build <https://travis-ci.org/balidani/flink/builds/51440170>). > > The error was the following: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile > (default-testCompile) on project flink-gelly: Compilation failure: > Compilation failure: > [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[21,30] > package com.google.common.base does not exist > [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[22,28] > package com.google.common.io does not exist > [ERROR] /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[80,53] > cannot find symbol > > Then I looked at flink-tests and saw that there was a `profiles` part in > the POM that adds the dependency to guava for hadoop profile 1. I added > this to the POM of Gelly and it seems to fix the issue. (Travis build > <https://travis-ci.org/balidani/flink/builds/51496196>) (There is a > failure, but the log says it's a timeout) > > Is this the correct way to solve this error? > > Thanks, > > Daniel |
Thanks for the answers!
I didn't set the scope to provided, but now I will make that change. Regards, Daniel 2015-02-20 14:27 GMT+01:00 Aljoscha Krettek <[hidden email]>: > Did you set the scope to provided? As in: > > <dependency> > <groupId>com.google.guava</groupId> > <artifactId>guava</artifactId> > <version>${guava.version}</version> > <scope>provided</scope> > </dependency> > > Then this should be a good solution, yes. > > On Fri, Feb 20, 2015 at 1:05 PM, Dániel Bali <[hidden email]> > wrote: > > Hi! > > > > I was working on FLINK-1522 > > <https://issues.apache.org/jira/browse/FLINK-1522> and FLINK-1576 > > <https://issues.apache.org/jira/browse/FLINK-1576> and started > converting > > Gelly library examples to work with file inputs. I added a test for > > LabelPropagationExample and got failures in Travis when hadoop.profile > was > > 1 (Travis build <https://travis-ci.org/balidani/flink/builds/51440170>). > > > > The error was the following: > > > > [ERROR] Failed to execute goal > > org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile > > (default-testCompile) on project flink-gelly: Compilation failure: > > Compilation failure: > > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[21,30] > > package com.google.common.base does not exist > > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[22,28] > > package com.google.common.io does not exist > > [ERROR] > /home/travis/build/balidani/flink/flink-staging/flink-gelly/src/test/java/org/apache/flink/graph/test/LabelPropagationExampleITCase.java:[80,53] > > cannot find symbol > > > > Then I looked at flink-tests and saw that there was a `profiles` part in > > the POM that adds the dependency to guava for hadoop profile 1. I added > > this to the POM of Gelly and it seems to fix the issue. (Travis build > > <https://travis-ci.org/balidani/flink/builds/51496196>) (There is a > > failure, but the log says it's a timeout) > > > > Is this the correct way to solve this error? > > > > Thanks, > > > > Daniel > |
Free forum by Nabble | Edit this page |