Introduce flink-connector-hive-xx modules

[Jingsong Lee]

Hi all,

I'd like to propose introduce flink-connector-hive-xx modules.

We have documented the dependencies detailed information[2]. But still has
some inconvenient:
- Too many versions, users need to pick one version from 8 versions.
- Too many versions, It's not friendly to our developers either, because
there's a problem/exception, we need to look at eight different versions of
hive client code, which are often various.
- Too many jars, for example, users need to download 4+ jars for Hive 1.x
from various places.

We have discussed in [1] and [2], but unfortunately, we can not achieve an
agreement.

For improving this, I'd like to introduce few flink-connector-hive-xx
modules in flink-connectors, module contains all the dependencies related
to hive. And only support lower hive metastore versions:
- "flink-connector-hive-1.2" to support hive 1.0.0 - 1.2.2
- "flink-connector-hive-2.0" to support hive 2.0.0 - 2.0.1
- "flink-connector-hive-2.2" to support hive 2.1.0 - 2.2.0
- "flink-connector-hive-2.3" to support hive 2.3.0 - 2.3.6
- "flink-connector-hive-3.1" to support hive 3.0.0 - 3.1.2

Users can choose one and download to flink/lib. It includes all hive things.

I try to use a single module to deploy multiple versions, but I can not
find a suitable way, because different modules require different versions
and different dependencies.

What do you think?

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-have-separate-Flink-distributions-with-built-in-Hive-dependencies-td35918.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-109-Improve-Hive-dependencies-out-of-box-experience-td38290.html

Best,
Jingsong Lee

[Jingsong Lee]

Sorry to forget the title [DISCUSS].
Close this thread.

Best,
Jingsong Lee


On Wed, Mar 4, 2020 at 4:57 PM Jingsong Lee <[hidden email]> wrote:

> Hi all,
>
> I'd like to propose introduce flink-connector-hive-xx modules.
>
> We have documented the dependencies detailed information[2]. But still has
> some inconvenient:
> - Too many versions, users need to pick one version from 8 versions.
> - Too many versions, It's not friendly to our developers either, because
> there's a problem/exception, we need to look at eight different versions of
> hive client code, which are often various.
> - Too many jars, for example, users need to download 4+ jars for Hive 1.x
> from various places.
>
> We have discussed in [1] and [2], but unfortunately, we can not achieve an
> agreement.
>
> For improving this, I'd like to introduce few flink-connector-hive-xx
> modules in flink-connectors, module contains all the dependencies related
> to hive. And only support lower hive metastore versions:
> - "flink-connector-hive-1.2" to support hive 1.0.0 - 1.2.2
> - "flink-connector-hive-2.0" to support hive 2.0.0 - 2.0.1
> - "flink-connector-hive-2.2" to support hive 2.1.0 - 2.2.0
> - "flink-connector-hive-2.3" to support hive 2.3.0 - 2.3.6
> - "flink-connector-hive-3.1" to support hive 3.0.0 - 3.1.2
>
> Users can choose one and download to flink/lib. It includes all hive
> things.
>
> I try to use a single module to deploy multiple versions, but I can not
> find a suitable way, because different modules require different versions
> and different dependencies.
>
> What do you think?
>
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-have-separate-Flink-distributions-with-built-in-Hive-dependencies-td35918.html
> [2]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-109-Improve-Hive-dependencies-out-of-box-experience-td38290.html
>
> Best,
> Jingsong Lee
>

--
Best, Jingsong Lee