Build error: package does not exist

[Hynek Noll]

Hi everyone,
I'm trying to start the development in Flink and have followed the
instructions on the web to start an IntelliJ project.
I have cloned the latest Flink repo and upon Build, it gives me:

*Error:(21, 40) java: package org.apache.flink.sql.parser.impl does not
exist*

I've set up the project to use JDK 8 and later tried switching to Maven
3.1.1 (from the default 3.6.2), but no success.

This package seems to be missing on GitHub:
https://github.com/apache/flink/tree/master/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser


Although a JavaDoc of it exists:
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/sql/parser/impl/package-summary.html


Am I missing something?

Best regards,
Hynek

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

[yanjun qiu]

Hi Hynek,
I think you should run maven build first, execute mvn clean install -DskipTests. Because the Flink SQL parser is used apache calcite framework to generate the sql parser source code.

Regards,
Bruce

> 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
>
> package seems to be missing on GitHub:

[Jark Wu-2]

Hi Hynek,

Bruce is right, you should build Flink source code first before developing
by `mvn clean package -DskipTests` in the root directory of Flink.
This may take 10 minutes or more depends on your machine.

Best,
Jark

On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]> wrote:

> Hi Hynek,
> I think you should run maven build first, execute mvn clean install
> -DskipTests. Because the Flink SQL parser is used apache calcite framework
> to generate the sql parser source code.
>
> Regards,
> Bruce
>
> > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
> >
> > package seems to be missing on GitHub:
>
>

[Hynek Noll]

Hi Bruce and Jark,
Thank you for the tip, but I already did the similar by clicking "Generate
Sources and Update Folders". I tried the suggested command(s), but without
success unfortunately.
Executing  `mvn clean install -DskipTests` resulted in an error: "Too many
files with unapproved license: 2 See RAT report ...". (In the report it
states the two files are:
flink-core/src/test/resources/abstractID-with-toString-field
    flink-core/src/test/resources/abstractID-with-toString-field-set
) While `mvn clean package -DskipTests` actually runs for 30+ minutes (much
longer that the first command but maybe that stops early because of the
error) and finishes fine but I have the same problems afterwards.

I've tried switching to Maven 3.1.1 (from 3.6.1).

Now the one thing that resolved the above stated missing package was
switching to Scala 2.11.12 SDK (instead of 2.12)!

The steps I've been taking were (within IntelliJ) Invalidate caches &
restart (alternatively Exit and delete the .idea folder) -> Open IntelliJ
again -> Maven Reimport -> Maven Generate Sources and Update Folders ->
Build Project. That results in further package(s) missing:


*Error:(21, 53) java: package org.apache.flink.kinesis.shaded.com.amazonaws
does not exist*
Maybe it now has to do with just the dependency shading?

Best regards,
Hynek

ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]> napsal:

> Hi Hynek,
>
> Bruce is right, you should build Flink source code first before developing
> by `mvn clean package -DskipTests` in the root directory of Flink.
> This may take 10 minutes or more depends on your machine.
>
> Best,
> Jark
>
> On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]> wrote:
>
> > Hi Hynek,
> > I think you should run maven build first, execute mvn clean install
> > -DskipTests. Because the Flink SQL parser is used apache calcite
> framework
> > to generate the sql parser source code.
> >
> > Regards,
> > Bruce
> >
> > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
> > >
> > > package seems to be missing on GitHub:
> >
> >
>

[Pritam Sadhukhan]

Hi Hynek,

please run mvn clean install -DskipTests -Drat.skip=true.

It should build properly but takes time.

Regards

On Mon, Oct 28, 2019, 10:06 PM Hynek Noll <[hidden email]> wrote:

> Hi Bruce and Jark,
> Thank you for the tip, but I already did the similar by clicking "Generate
> Sources and Update Folders". I tried the suggested command(s), but without
> success unfortunately.
> Executing  `mvn clean install -DskipTests` resulted in an error: "Too many
> files with unapproved license: 2 See RAT report ...". (In the report it
> states the two files are:
> flink-core/src/test/resources/abstractID-with-toString-field
>     flink-core/src/test/resources/abstractID-with-toString-field-set
> ) While `mvn clean package -DskipTests` actually runs for 30+ minutes (much
> longer that the first command but maybe that stops early because of the
> error) and finishes fine but I have the same problems afterwards.
>
> I've tried switching to Maven 3.1.1 (from 3.6.1).
>
> Now the one thing that resolved the above stated missing package was
> switching to Scala 2.11.12 SDK (instead of 2.12)!
>
> The steps I've been taking were (within IntelliJ) Invalidate caches &
> restart (alternatively Exit and delete the .idea folder) -> Open IntelliJ
> again -> Maven Reimport -> Maven Generate Sources and Update Folders ->
> Build Project. That results in further package(s) missing:
>
>
> *Error:(21, 53) java: package org.apache.flink.kinesis.shaded.com.amazonaws
> does not exist*
> Maybe it now has to do with just the dependency shading?
>
> Best regards,
> Hynek
>
> ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]> napsal:
>
> > Hi Hynek,
> >
> > Bruce is right, you should build Flink source code first before
> developing
> > by `mvn clean package -DskipTests` in the root directory of Flink.
> > This may take 10 minutes or more depends on your machine.
> >
> > Best,
> > Jark
> >
> > On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]> wrote:
> >
> > > Hi Hynek,
> > > I think you should run maven build first, execute mvn clean install
> > > -DskipTests. Because the Flink SQL parser is used apache calcite
> > framework
> > > to generate the sql parser source code.
> > >
> > > Regards,
> > > Bruce
> > >
> > > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
> > > >
> > > > package seems to be missing on GitHub:
> > >
> > >
> >
>

[Hynek Noll]

Dear Pritam,
I've tried that as well, specifically I ran:
`mvn clean install -DskipTests -Drat.skip=true -DskipITs -Pinclude-kinesis
-Daws.kinesis-kpl.version=0.12.6`
But the result is still the same. During the build, the packages that I
suppose should be generated by Maven based on Amazon Kinesis are missing.

Best regards,
Hynek

po 28. 10. 2019 v 19:57 odesílatel Pritam Sadhukhan <
[hidden email]> napsal:

> Hi Hynek,
>
> please run mvn clean install -DskipTests -Drat.skip=true.
>
> It should build properly but takes time.
>
> Regards
>
> On Mon, Oct 28, 2019, 10:06 PM Hynek Noll <[hidden email]> wrote:
>
> > Hi Bruce and Jark,
> > Thank you for the tip, but I already did the similar by clicking
> "Generate
> > Sources and Update Folders". I tried the suggested command(s), but
> without
> > success unfortunately.
> > Executing  `mvn clean install -DskipTests` resulted in an error: "Too
> many
> > files with unapproved license: 2 See RAT report ...". (In the report it
> > states the two files are:
> > flink-core/src/test/resources/abstractID-with-toString-field
> >     flink-core/src/test/resources/abstractID-with-toString-field-set
> > ) While `mvn clean package -DskipTests` actually runs for 30+ minutes
> (much
> > longer that the first command but maybe that stops early because of the
> > error) and finishes fine but I have the same problems afterwards.
> >
> > I've tried switching to Maven 3.1.1 (from 3.6.1).
> >
> > Now the one thing that resolved the above stated missing package was
> > switching to Scala 2.11.12 SDK (instead of 2.12)!
> >
> > The steps I've been taking were (within IntelliJ) Invalidate caches &
> > restart (alternatively Exit and delete the .idea folder) -> Open IntelliJ
> > again -> Maven Reimport -> Maven Generate Sources and Update Folders ->
> > Build Project. That results in further package(s) missing:
> >
> >
> > *Error:(21, 53) java: package
> org.apache.flink.kinesis.shaded.com.amazonaws
> > does not exist*
> > Maybe it now has to do with just the dependency shading?
> >
> > Best regards,
> > Hynek
> >
> > ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]> napsal:
> >
> > > Hi Hynek,
> > >
> > > Bruce is right, you should build Flink source code first before
> > developing
> > > by `mvn clean package -DskipTests` in the root directory of Flink.
> > > This may take 10 minutes or more depends on your machine.
> > >
> > > Best,
> > > Jark
> > >
> > > On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]>
> wrote:
> > >
> > > > Hi Hynek,
> > > > I think you should run maven build first, execute mvn clean install
> > > > -DskipTests. Because the Flink SQL parser is used apache calcite
> > > framework
> > > > to generate the sql parser source code.
> > > >
> > > > Regards,
> > > > Bruce
> > > >
> > > > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
> > > > >
> > > > > package seems to be missing on GitHub:
> > > >
> > > >
> > >
> >
>

[Hynek Noll-2]

And I've tried just
`mvn clean install -DskipTests -Drat.skip=true -DskipITs`
as well. It takes around half an hour, so I'm not too keen to try all the
possibilities.

I guess there might be some other SDKs/libraries that I'm missing and Maven
won't tell me? Or just some random incompatibility?

Thanks for any tips,
Hynek

po 28. 10. 2019 v 20:24 odesílatel Hynek Noll <[hidden email]> napsal:

> Dear Pritam,
> I've tried that as well, specifically I ran:
> `mvn clean install -DskipTests -Drat.skip=true -DskipITs -Pinclude-kinesis
> -Daws.kinesis-kpl.version=0.12.6`
> But the result is still the same. During the build, the packages that I
> suppose should be generated by Maven based on Amazon Kinesis are missing.
>
> Best regards,
> Hynek
>
> po 28. 10. 2019 v 19:57 odesílatel Pritam Sadhukhan <
> [hidden email]> napsal:
>
>> Hi Hynek,
>>
>> please run mvn clean install -DskipTests -Drat.skip=true.
>>
>> It should build properly but takes time.
>>
>> Regards
>>
>> On Mon, Oct 28, 2019, 10:06 PM Hynek Noll <[hidden email]> wrote:
>>
>> > Hi Bruce and Jark,
>> > Thank you for the tip, but I already did the similar by clicking
>> "Generate
>> > Sources and Update Folders". I tried the suggested command(s), but
>> without
>> > success unfortunately.
>> > Executing  `mvn clean install -DskipTests` resulted in an error: "Too
>> many
>> > files with unapproved license: 2 See RAT report ...". (In the report it
>> > states the two files are:
>> > flink-core/src/test/resources/abstractID-with-toString-field
>> >     flink-core/src/test/resources/abstractID-with-toString-field-set
>> > ) While `mvn clean package -DskipTests` actually runs for 30+ minutes
>> (much
>> > longer that the first command but maybe that stops early because of the
>> > error) and finishes fine but I have the same problems afterwards.
>> >
>> > I've tried switching to Maven 3.1.1 (from 3.6.1).
>> >
>> > Now the one thing that resolved the above stated missing package was
>> > switching to Scala 2.11.12 SDK (instead of 2.12)!
>> >
>> > The steps I've been taking were (within IntelliJ) Invalidate caches &
>> > restart (alternatively Exit and delete the .idea folder) -> Open
>> IntelliJ
>> > again -> Maven Reimport -> Maven Generate Sources and Update Folders ->
>> > Build Project. That results in further package(s) missing:
>> >
>> >
>> > *Error:(21, 53) java: package
>> org.apache.flink.kinesis.shaded.com.amazonaws
>> > does not exist*
>> > Maybe it now has to do with just the dependency shading?
>> >
>> > Best regards,
>> > Hynek
>> >
>> > ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]> napsal:
>> >
>> > > Hi Hynek,
>> > >
>> > > Bruce is right, you should build Flink source code first before
>> > developing
>> > > by `mvn clean package -DskipTests` in the root directory of Flink.
>> > > This may take 10 minutes or more depends on your machine.
>> > >
>> > > Best,
>> > > Jark
>> > >
>> > > On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]>
>> wrote:
>> > >
>> > > > Hi Hynek,
>> > > > I think you should run maven build first, execute mvn clean install
>> > > > -DskipTests. Because the Flink SQL parser is used apache calcite
>> > > framework
>> > > > to generate the sql parser source code.
>> > > >
>> > > > Regards,
>> > > > Bruce
>> > > >
>> > > > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
>> > > > >
>> > > > > package seems to be missing on GitHub:
>> > > >
>> > > >
>> > >
>> >
>>
>

[Hynek Noll]

And I've tried just
`mvn clean install -DskipTests -Drat.skip=true -DskipITs`
as well. It takes around half an hour, so I'm not too keen to try all the
possibilities.

I guess there might be some other SDKs/libraries that I'm missing and Maven
won't tell me? Or just some random incompatibility?

Thanks for any tips,
Hynek

po 28. 10. 2019 v 20:28 odesílatel Hynek Noll <[hidden email]> napsal:

> And I've tried just
> `mvn clean install -DskipTests -Drat.skip=true -DskipITs`
> as well. It takes around half an hour, so I'm not too keen to try all the
> possibilities.
>
> I guess there might be some other SDKs/libraries that I'm missing and
> Maven won't tell me? Or just some random incompatibility?
>
> Thanks for any tips,
> Hynek
>
> po 28. 10. 2019 v 20:24 odesílatel Hynek Noll <[hidden email]>
> napsal:
>
>> Dear Pritam,
>> I've tried that as well, specifically I ran:
>> `mvn clean install -DskipTests -Drat.skip=true -DskipITs
>> -Pinclude-kinesis -Daws.kinesis-kpl.version=0.12.6`
>> But the result is still the same. During the build, the packages that I
>> suppose should be generated by Maven based on Amazon Kinesis are missing.
>>
>> Best regards,
>> Hynek
>>
>> po 28. 10. 2019 v 19:57 odesílatel Pritam Sadhukhan <
>> [hidden email]> napsal:
>>
>>> Hi Hynek,
>>>
>>> please run mvn clean install -DskipTests -Drat.skip=true.
>>>
>>> It should build properly but takes time.
>>>
>>> Regards
>>>
>>> On Mon, Oct 28, 2019, 10:06 PM Hynek Noll <[hidden email]> wrote:
>>>
>>> > Hi Bruce and Jark,
>>> > Thank you for the tip, but I already did the similar by clicking
>>> "Generate
>>> > Sources and Update Folders". I tried the suggested command(s), but
>>> without
>>> > success unfortunately.
>>> > Executing  `mvn clean install -DskipTests` resulted in an error: "Too
>>> many
>>> > files with unapproved license: 2 See RAT report ...". (In the report it
>>> > states the two files are:
>>> > flink-core/src/test/resources/abstractID-with-toString-field
>>> >     flink-core/src/test/resources/abstractID-with-toString-field-set
>>> > ) While `mvn clean package -DskipTests` actually runs for 30+ minutes
>>> (much
>>> > longer that the first command but maybe that stops early because of the
>>> > error) and finishes fine but I have the same problems afterwards.
>>> >
>>> > I've tried switching to Maven 3.1.1 (from 3.6.1).
>>> >
>>> > Now the one thing that resolved the above stated missing package was
>>> > switching to Scala 2.11.12 SDK (instead of 2.12)!
>>> >
>>> > The steps I've been taking were (within IntelliJ) Invalidate caches &
>>> > restart (alternatively Exit and delete the .idea folder) -> Open
>>> IntelliJ
>>> > again -> Maven Reimport -> Maven Generate Sources and Update Folders ->
>>> > Build Project. That results in further package(s) missing:
>>> >
>>> >
>>> > *Error:(21, 53) java: package
>>> org.apache.flink.kinesis.shaded.com.amazonaws
>>> > does not exist*
>>> > Maybe it now has to do with just the dependency shading?
>>> >
>>> > Best regards,
>>> > Hynek
>>> >
>>> > ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]> napsal:
>>> >
>>> > > Hi Hynek,
>>> > >
>>> > > Bruce is right, you should build Flink source code first before
>>> > developing
>>> > > by `mvn clean package -DskipTests` in the root directory of Flink.
>>> > > This may take 10 minutes or more depends on your machine.
>>> > >
>>> > > Best,
>>> > > Jark
>>> > >
>>> > > On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]>
>>> wrote:
>>> > >
>>> > > > Hi Hynek,
>>> > > > I think you should run maven build first, execute mvn clean install
>>> > > > -DskipTests. Because the Flink SQL parser is used apache calcite
>>> > > framework
>>> > > > to generate the sql parser source code.
>>> > > >
>>> > > > Regards,
>>> > > > Bruce
>>> > > >
>>> > > > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
>>> > > > >
>>> > > > > package seems to be missing on GitHub:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
>>

[Pritam Sadhukhan]

Hi Hynek,

I used `mvn clean install -DskipTests -Drat.skip=true` on the root
directory.
I have build release 1.9 branch and took me around 1hr 23 minutes to
complete with no errors.

Regards,
Pritam.

On Tue, 29 Oct 2019 at 01:02, Hynek Noll <[hidden email]> wrote:

> And I've tried just
> `mvn clean install -DskipTests -Drat.skip=true -DskipITs`
> as well. It takes around half an hour, so I'm not too keen to try all the
> possibilities.
>
> I guess there might be some other SDKs/libraries that I'm missing and Maven
> won't tell me? Or just some random incompatibility?
>
> Thanks for any tips,
> Hynek
>
> po 28. 10. 2019 v 20:28 odesílatel Hynek Noll <[hidden email]> napsal:
>
> > And I've tried just
> > `mvn clean install -DskipTests -Drat.skip=true -DskipITs`
> > as well. It takes around half an hour, so I'm not too keen to try all the
> > possibilities.
> >
> > I guess there might be some other SDKs/libraries that I'm missing and
> > Maven won't tell me? Or just some random incompatibility?
> >
> > Thanks for any tips,
> > Hynek
> >
> > po 28. 10. 2019 v 20:24 odesílatel Hynek Noll <[hidden email]>
> > napsal:
> >
> >> Dear Pritam,
> >> I've tried that as well, specifically I ran:
> >> `mvn clean install -DskipTests -Drat.skip=true -DskipITs
> >> -Pinclude-kinesis -Daws.kinesis-kpl.version=0.12.6`
> >> But the result is still the same. During the build, the packages that I
> >> suppose should be generated by Maven based on Amazon Kinesis are
> missing.
> >>
> >> Best regards,
> >> Hynek
> >>
> >> po 28. 10. 2019 v 19:57 odesílatel Pritam Sadhukhan <
> >> [hidden email]> napsal:
> >>
> >>> Hi Hynek,
> >>>
> >>> please run mvn clean install -DskipTests -Drat.skip=true.
> >>>
> >>> It should build properly but takes time.
> >>>
> >>> Regards
> >>>
> >>> On Mon, Oct 28, 2019, 10:06 PM Hynek Noll <[hidden email]>
> wrote:
> >>>
> >>> > Hi Bruce and Jark,
> >>> > Thank you for the tip, but I already did the similar by clicking
> >>> "Generate
> >>> > Sources and Update Folders". I tried the suggested command(s), but
> >>> without
> >>> > success unfortunately.
> >>> > Executing  `mvn clean install -DskipTests` resulted in an error: "Too
> >>> many
> >>> > files with unapproved license: 2 See RAT report ...". (In the report
> it
> >>> > states the two files are:
> >>> > flink-core/src/test/resources/abstractID-with-toString-field
> >>> >     flink-core/src/test/resources/abstractID-with-toString-field-set
> >>> > ) While `mvn clean package -DskipTests` actually runs for 30+ minutes
> >>> (much
> >>> > longer that the first command but maybe that stops early because of
> the
> >>> > error) and finishes fine but I have the same problems afterwards.
> >>> >
> >>> > I've tried switching to Maven 3.1.1 (from 3.6.1).
> >>> >
> >>> > Now the one thing that resolved the above stated missing package was
> >>> > switching to Scala 2.11.12 SDK (instead of 2.12)!
> >>> >
> >>> > The steps I've been taking were (within IntelliJ) Invalidate caches &
> >>> > restart (alternatively Exit and delete the .idea folder) -> Open
> >>> IntelliJ
> >>> > again -> Maven Reimport -> Maven Generate Sources and Update Folders
> ->
> >>> > Build Project. That results in further package(s) missing:
> >>> >
> >>> >
> >>> > *Error:(21, 53) java: package
> >>> org.apache.flink.kinesis.shaded.com.amazonaws
> >>> > does not exist*
> >>> > Maybe it now has to do with just the dependency shading?
> >>> >
> >>> > Best regards,
> >>> > Hynek
> >>> >
> >>> > ne 27. 10. 2019 v 15:02 odesílatel Jark Wu <[hidden email]>
> napsal:
> >>> >
> >>> > > Hi Hynek,
> >>> > >
> >>> > > Bruce is right, you should build Flink source code first before
> >>> > developing
> >>> > > by `mvn clean package -DskipTests` in the root directory of Flink.
> >>> > > This may take 10 minutes or more depends on your machine.
> >>> > >
> >>> > > Best,
> >>> > > Jark
> >>> > >
> >>> > > On Sun, 27 Oct 2019 at 20:46, yanjun qiu <[hidden email]>
> >>> wrote:
> >>> > >
> >>> > > > Hi Hynek,
> >>> > > > I think you should run maven build first, execute mvn clean
> install
> >>> > > > -DskipTests. Because the Flink SQL parser is used apache calcite
> >>> > > framework
> >>> > > > to generate the sql parser source code.
> >>> > > >
> >>> > > > Regards,
> >>> > > > Bruce
> >>> > > >
> >>> > > > > 在 2019年10月27日，上午12:09，Hynek Noll <[hidden email]> 写道：
> >>> > > > >
> >>> > > > > package seems to be missing on GitHub:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
>