flink-connector-hive uses third-party repositories

[Visser, M.J.H. (Martijn)]

Hi all,

While trying to get the latest Flink master compiled within a corporate environment, I've noticed that the flink-connector-hive keeps failing. It appears that it uses dependencies which are not located in Maven Central, but on third-party repositories like Conjar. Examples are the org.pentaho pentaho-aggdesigner-algorithm and the javax.jms:jms:jar:1.1

With a little bit of digging, I can probably get it work within the corporate environment, but it might be interesting to reduce the dependency on third-party maven repositories. Calcite did something similar https://issues.apache.org/jira/browse/CALCITE-605 https://jira.apache.org/jira/browse/CALCITE-1474

Best regards,

Martijn


-----------------------------------------------------------------
ATTENTION:
The information in this e-mail is confidential and only meant for the intended recipient. If you are not the intended recipient, don't use or disclose it in any way. Please let the sender know and delete the message immediately.
-----------------------------------------------------------------

[Till Rohrmann]

Thanks a lot for raising this issue Martijn. I've pulled in Rui who works
on the flink-connector-hive module and Kurt.

@Kurt and @Rui, do you think it's doable to decrease the dependency on
third party repositories? In any case, I've create a JIRA issue for it [1].

[1] https://issues.apache.org/jira/browse/FLINK-13475

Cheers,
Till

On Mon, Jul 29, 2019 at 11:26 AM Visser, M.J.H. (Martijn)
<[hidden email]> wrote:

> Hi all,
>
> While trying to get the latest Flink master compiled within a corporate
> environment, I've noticed that the flink-connector-hive keeps failing. It
> appears that it uses dependencies which are not located in Maven Central,
> but on third-party repositories like Conjar. Examples are the org.pentaho
> pentaho-aggdesigner-algorithm and the javax.jms:jms:jar:1.1
>
> With a little bit of digging, I can probably get it work within the
> corporate environment, but it might be interesting to reduce the dependency
> on third-party maven repositories. Calcite did something similar
> https://issues.apache.org/jira/browse/CALCITE-605
> https://jira.apache.org/jira/browse/CALCITE-1474
>
> Best regards,
>
> Martijn
>
>
> -----------------------------------------------------------------
> ATTENTION:
> The information in this e-mail is confidential and only meant for the
> intended recipient. If you are not the intended recipient, don't use or
> disclose it in any way. Please let the sender know and delete the message
> immediately.
> -----------------------------------------------------------------
>

[Terry Wang]

Hi Martijn:
        I looked through this problem and found javax.jms:jms:jar:1.1 can be founded in  https://mvnrepository.com/artifact/javax.jms/jms <https://mvnrepository.com/artifact/javax.jms/jms> .
javax.jms:jms:jar:1.1 is imported by HiveRunner with test scope while another org.pentaho:pentaho-aggdesigner-algorithm artifacted is not imported. I upload a dependency tree file in the attachments.
Martijn Visser <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=MartijnVisser> can you provide an environment reproducing compile problem?

I think we can discuss under https://issues.apache.org/jira/browse/FLINK-13475 <https://issues.apache.org/jira/browse/FLINK-13475>.

Cheers,
Terry Wang

> 在 2019年7月29日，下午9:09，Till Rohrmann <[hidden email]> 写道：
>
> Thanks a lot for raising this issue Martijn. I've pulled in Rui who works
> on the flink-connector-hive module and Kurt.
>
> @Kurt and @Rui, do you think it's doable to decrease the dependency on
> third party repositories? In any case, I've create a JIRA issue for it [1].
>
> [1] https://issues.apache.org/jira/browse/FLINK-13475
>
> Cheers,
> Till
>
> On Mon, Jul 29, 2019 at 11:26 AM Visser, M.J.H. (Martijn)
> <[hidden email]> wrote:
>
>> Hi all,
>>
>> While trying to get the latest Flink master compiled within a corporate
>> environment, I've noticed that the flink-connector-hive keeps failing. It
>> appears that it uses dependencies which are not located in Maven Central,
>> but on third-party repositories like Conjar. Examples are the org.pentaho
>> pentaho-aggdesigner-algorithm and the javax.jms:jms:jar:1.1
>>
>> With a little bit of digging, I can probably get it work within the
>> corporate environment, but it might be interesting to reduce the dependency
>> on third-party maven repositories. Calcite did something similar
>> https://issues.apache.org/jira/browse/CALCITE-605
>> https://jira.apache.org/jira/browse/CALCITE-1474
>>
>> Best regards,
>>
>> Martijn
>>
>>
>> -----------------------------------------------------------------
>> ATTENTION:
>> The information in this e-mail is confidential and only meant for the
>> intended recipient. If you are not the intended recipient, don't use or
>> disclose it in any way. Please let the sender know and delete the message
>> immediately.
>> -----------------------------------------------------------------
>>