[jira] [Created] (FLINK-7597) broken flink-connectors-kinesis setup in Intellij that potentially results from improper pom.xml

[Shang Yuanchun (Jira)]

Bowen Li created FLINK-7597:
-------------------------------

             Summary: broken flink-connectors-kinesis setup in Intellij that potentially results from improper pom.xml
                 Key: FLINK-7597
                 URL: https://issues.apache.org/jira/browse/FLINK-7597
             Project: Flink
          Issue Type: Bug
          Components: Kinesis Connector
    Affects Versions: 1.3.2, 1.4.0
            Reporter: Bowen Li
            Assignee: Bowen Li


I use Intellij to develop flink and flink-connectors-kinesis. I imported the whole flink src code into Intellij, and Intellij treats flink-connectors-kinesis as a module. The project structure in intellij looks like this: https://imgur.com/a/uK3Fd

Here's the problem: The {{flink-connectors-kinesis}} module always complains about not being able to find dependencies like amazon-kinesis-producer, amazon-kinesis-client, flink-streaming-java_2.11, etc. Seems like Intellij cannot properly parse {{/flink-connectors-kinesis/pom.xml}}. And Intellij always suggest I add those dependencies to {{flink-connectors/pom.xml}}. In short, {{flink-connectors-kinesis}} won't compile in my Intellij until I added those dependencies to {{flink-connectors/pom.xml}}.

My {{flink-connectors/pom.xml}} file ends up like this all the time:

{code:java}
C02SD32LG8WP:flink-connectors Bowen$ git diff
diff --git a/flink-connectors/pom.xml b/flink-connectors/pom.xml
index bc3f82f..2b001f5 100644
--- a/flink-connectors/pom.xml
+++ b/flink-connectors/pom.xml
@@ -71,6 +71,16 @@ under the License.
                        <artifactId>jsr305</artifactId>
                        <scope>provided</scope>
                </dependency>
+               <dependency>
+                       <groupId>com.amazonaws</groupId>
+                       <artifactId>amazon-kinesis-producer</artifactId>
+                       <version>0.12.5</version>
+               </dependency>
+               <dependency>
+                       <groupId>com.amazonaws</groupId>
+                       <artifactId>amazon-kinesis-client</artifactId>
+                       <version>1.8.1</version>
+               </dependency>
+               <dependency>
+                       <groupId>org.apache.flink</groupId>
+                       <artifactId>flink-streaming-java_2.11</artifactId>
+                       <version>1.4-SNAPSHOT</version>
+               </dependency>
        </dependencies>

        <!-- See main pom.xml for explanation of profiles -->
{code}

FYI, building flink-connectors-kinesis from command line always works.

[~tzulitai] Do you use Intellij? If so, how do you properly set up the flink-connectors-kinesis project in Intellij to be able to retrieve dependencies?




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[Stephan Ewen]

Hi!

Kinesis is not built by default, because it has a license that does not
allow us to build it by default :-(
(The Amazon Software License).

Can you go to the maven profiles in IntelliJ and activate the profile to
include maven? That should help!

Best,
Stephan


On Thu, Sep 7, 2017 at 8:30 AM, Bowen Li (JIRA) <[hidden email]> wrote:

> Bowen Li created FLINK-7597:
> -------------------------------
>
>              Summary: broken flink-connectors-kinesis setup in Intellij
> that potentially results from improper pom.xml
>                  Key: FLINK-7597
>                  URL: https://issues.apache.org/jira/browse/FLINK-7597
>              Project: Flink
>           Issue Type: Bug
>           Components: Kinesis Connector
>     Affects Versions: 1.3.2, 1.4.0
>             Reporter: Bowen Li
>             Assignee: Bowen Li
>
>
> I use Intellij to develop flink and flink-connectors-kinesis. I imported
> the whole flink src code into Intellij, and Intellij treats
> flink-connectors-kinesis as a module. The project structure in intellij
> looks like this: https://imgur.com/a/uK3Fd
>
> Here's the problem: The {{flink-connectors-kinesis}} module always
> complains about not being able to find dependencies like
> amazon-kinesis-producer, amazon-kinesis-client, flink-streaming-java_2.11,
> etc. Seems like Intellij cannot properly parse {{/flink-connectors-kinesis/pom.xml}}.
> And Intellij always suggest I add those dependencies to
> {{flink-connectors/pom.xml}}. In short, {{flink-connectors-kinesis}} won't
> compile in my Intellij until I added those dependencies to
> {{flink-connectors/pom.xml}}.
>
> My {{flink-connectors/pom.xml}} file ends up like this all the time:
>
> {code:java}
> C02SD32LG8WP:flink-connectors Bowen$ git diff
> diff --git a/flink-connectors/pom.xml b/flink-connectors/pom.xml
> index bc3f82f..2b001f5 100644
> --- a/flink-connectors/pom.xml
> +++ b/flink-connectors/pom.xml
> @@ -71,6 +71,16 @@ under the License.
>                         <artifactId>jsr305</artifactId>
>                         <scope>provided</scope>
>                 </dependency>
> +               <dependency>
> +                       <groupId>com.amazonaws</groupId>
> +                       <artifactId>amazon-kinesis-producer</artifactId>
> +                       <version>0.12.5</version>
> +               </dependency>
> +               <dependency>
> +                       <groupId>com.amazonaws</groupId>
> +                       <artifactId>amazon-kinesis-client</artifactId>
> +                       <version>1.8.1</version>
> +               </dependency>
> +               <dependency>
> +                       <groupId>org.apache.flink</groupId>
> +                       <artifactId>flink-streaming-java_2.11</artifactId>
> +                       <version>1.4-SNAPSHOT</version>
> +               </dependency>
>         </dependencies>
>
>         <!-- See main pom.xml for explanation of profiles -->
> {code}
>
> FYI, building flink-connectors-kinesis from command line always works.
>
> [~tzulitai] Do you use Intellij? If so, how do you properly set up the
> flink-connectors-kinesis project in Intellij to be able to retrieve
> dependencies?
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.4.14#64029)
>

[Bowen Li]

Hi Stephan,

Importing that module separately after importing the whole flink project
works for me. Thanks, Stephan! I've closed that ticket.

Bowen


On Thu, Sep 7, 2017 at 11:24 AM, Stephan Ewen <[hidden email]> wrote:

> Hi!
>
> Kinesis is not built by default, because it has a license that does not
> allow us to build it by default :-(
> (The Amazon Software License).
>
> Can you go to the maven profiles in IntelliJ and activate the profile to
> include maven? That should help!
>
> Best,
> Stephan
>
>
> On Thu, Sep 7, 2017 at 8:30 AM, Bowen Li (JIRA) <[hidden email]> wrote:
>
> > Bowen Li created FLINK-7597:
> > -------------------------------
> >
> >              Summary: broken flink-connectors-kinesis setup in Intellij
> > that potentially results from improper pom.xml
> >                  Key: FLINK-7597
> >                  URL: https://issues.apache.org/jira/browse/FLINK-7597
> >              Project: Flink
> >           Issue Type: Bug
> >           Components: Kinesis Connector
> >     Affects Versions: 1.3.2, 1.4.0
> >             Reporter: Bowen Li
> >             Assignee: Bowen Li
> >
> >
> > I use Intellij to develop flink and flink-connectors-kinesis. I imported
> > the whole flink src code into Intellij, and Intellij treats
> > flink-connectors-kinesis as a module. The project structure in intellij
> > looks like this: https://imgur.com/a/uK3Fd
> >
> > Here's the problem: The {{flink-connectors-kinesis}} module always
> > complains about not being able to find dependencies like
> > amazon-kinesis-producer, amazon-kinesis-client,
> flink-streaming-java_2.11,
> > etc. Seems like Intellij cannot properly parse
> {{/flink-connectors-kinesis/pom.xml}}.
> > And Intellij always suggest I add those dependencies to
> > {{flink-connectors/pom.xml}}. In short, {{flink-connectors-kinesis}}
> won't
> > compile in my Intellij until I added those dependencies to
> > {{flink-connectors/pom.xml}}.
> >
> > My {{flink-connectors/pom.xml}} file ends up like this all the time:
> >
> > {code:java}
> > C02SD32LG8WP:flink-connectors Bowen$ git diff
> > diff --git a/flink-connectors/pom.xml b/flink-connectors/pom.xml
> > index bc3f82f..2b001f5 100644
> > --- a/flink-connectors/pom.xml
> > +++ b/flink-connectors/pom.xml
> > @@ -71,6 +71,16 @@ under the License.
> >                         <artifactId>jsr305</artifactId>
> >                         <scope>provided</scope>
> >                 </dependency>
> > +               <dependency>
> > +                       <groupId>com.amazonaws</groupId>
> > +                       <artifactId>amazon-kinesis-producer</artifactId>
> > +                       <version>0.12.5</version>
> > +               </dependency>
> > +               <dependency>
> > +                       <groupId>com.amazonaws</groupId>
> > +                       <artifactId>amazon-kinesis-client</artifactId>
> > +                       <version>1.8.1</version>
> > +               </dependency>
> > +               <dependency>
> > +                       <groupId>org.apache.flink</groupId>
> > +                       <artifactId>flink-streaming-
> java_2.11</artifactId>
> > +                       <version>1.4-SNAPSHOT</version>
> > +               </dependency>
> >         </dependencies>
> >
> >         <!-- See main pom.xml for explanation of profiles -->
> > {code}
> >
> > FYI, building flink-connectors-kinesis from command line always works.
> >
> > [~tzulitai] Do you use Intellij? If so, how do you properly set up the
> > flink-connectors-kinesis project in Intellij to be able to retrieve
> > dependencies?
> >
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.4.14#64029)
> >
>