[DISCUSS] Change default planner to blink planner in 1.11

[Kurt Young]

Hi Dev and User,

Blink planner for Table API & SQL is introduced in Flink 1.9 and already be
the default planner for
SQL client in Flink 1.10. And since we already decided not introducing any
new features to the
original Flink planner, it already lacked of so many great features that
the community has been working on, such as brand new type system, more DDL
support and more planner capabilities.
During this time, we've also received lots of great feedback from users who
were trying to use blink
planner, both positive and negative (like bugs). This is a good sign, it at
least shows more and more
users are starting to try out.

So I want to start this discussion more formally to talk about
replacing the default planner to blink.
Specifically, I want to gather feedbacks from dev and user about whether
blink planner already
cover the original planner's capabilities, what kind of issues you've ran
into when try out blink
planner and then make you fallback to original one. Since there is still a
month to go when feature
freeze, there's still enough time for community to further enhance blink
planner for this purpose.

Let me know what you think, especially if you want to report or complain
about something. Thanks
in advance.

Best,
Kurt

[Benchao Li]

Hi Kurt,

It's excited to hear that the community aims to make Blink Planner default
in 1.11.

We have been using blink planner since 1.9 for streaming processing, it
works very well,
and covers many use cases in our company.
So +1 to make it default in 1.11 from our side.

Kurt Young <[hidden email]> 于2020年4月1日周三 上午9:15写道：

> Hi Dev and User,
>
> Blink planner for Table API & SQL is introduced in Flink 1.9 and already
> be the default planner for
> SQL client in Flink 1.10. And since we already decided not introducing any
> new features to the
> original Flink planner, it already lacked of so many great features that
> the community has been working on, such as brand new type system, more DDL
> support and more planner capabilities.
> During this time, we've also received lots of great feedback from users
> who were trying to use blink
> planner, both positive and negative (like bugs). This is a good sign, it
> at least shows more and more
> users are starting to try out.
>
> So I want to start this discussion more formally to talk about
> replacing the default planner to blink.
> Specifically, I want to gather feedbacks from dev and user about whether
> blink planner already
> cover the original planner's capabilities, what kind of issues you've ran
> into when try out blink
> planner and then make you fallback to original one. Since there is still a
> month to go when feature
> freeze, there's still enough time for community to further enhance blink
> planner for this purpose.
>
> Let me know what you think, especially if you want to report or complain
> about something. Thanks
> in advance.
>
> Best,
> Kurt
>

--

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:[phone]
Email: [hidden email]; [hidden email]

[Jark Wu-2]

+1 to make blink planner as default planner.

We should give blink planner more exposure to encourage users trying out
new features and lead users to migrate to blink planner.

Glad to see blink planner is used in production since 1.9! @Benchao

Best,
Jark

On Wed, 1 Apr 2020 at 11:31, Benchao Li <[hidden email]> wrote:

> Hi Kurt,
>
> It's excited to hear that the community aims to make Blink Planner default
> in 1.11.
>
> We have been using blink planner since 1.9 for streaming processing, it
> works very well,
> and covers many use cases in our company.
> So +1 to make it default in 1.11 from our side.
>
> Kurt Young <[hidden email]> 于2020年4月1日周三 上午9:15写道：
>
>> Hi Dev and User,
>>
>> Blink planner for Table API & SQL is introduced in Flink 1.9 and already
>> be the default planner for
>> SQL client in Flink 1.10. And since we already decided not introducing
>> any new features to the
>> original Flink planner, it already lacked of so many great features that
>> the community has been working on, such as brand new type system, more DDL
>> support and more planner capabilities.
>> During this time, we've also received lots of great feedback from users
>> who were trying to use blink
>> planner, both positive and negative (like bugs). This is a good sign, it
>> at least shows more and more
>> users are starting to try out.
>>
>> So I want to start this discussion more formally to talk about
>> replacing the default planner to blink.
>> Specifically, I want to gather feedbacks from dev and user about whether
>> blink planner already
>> cover the original planner's capabilities, what kind of issues you've ran
>> into when try out blink
>> planner and then make you fallback to original one. Since there is still
>> a month to go when feature
>> freeze, there's still enough time for community to further enhance blink
>> planner for this purpose.
>>
>> Let me know what you think, especially if you want to report or complain
>> about something. Thanks
>> in advance.
>>
>> Best,
>> Kurt
>>
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:[phone]
> Email: [hidden email]; [hidden email]
>
>

[Jingsong Li]

+1

In 1.10, we have set default planner for SQL Client to Blink planner[1].
Looks good.

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Set-default-planner-for-SQL-Client-to-Blink-planner-in-1-10-release-td36379.html

Best,
Jingsong Lee

On Wed, Apr 1, 2020 at 11:39 AM Jark Wu <[hidden email]> wrote:

> +1 to make blink planner as default planner.
>
> We should give blink planner more exposure to encourage users trying out
> new features and lead users to migrate to blink planner.
>
> Glad to see blink planner is used in production since 1.9! @Benchao
>
> Best,
> Jark
>
> On Wed, 1 Apr 2020 at 11:31, Benchao Li <[hidden email]> wrote:
>
> > Hi Kurt,
> >
> > It's excited to hear that the community aims to make Blink Planner
> default
> > in 1.11.
> >
> > We have been using blink planner since 1.9 for streaming processing, it
> > works very well,
> > and covers many use cases in our company.
> > So +1 to make it default in 1.11 from our side.
> >
> > Kurt Young <[hidden email]> 于2020年4月1日周三 上午9:15写道：
> >
> >> Hi Dev and User,
> >>
> >> Blink planner for Table API & SQL is introduced in Flink 1.9 and already
> >> be the default planner for
> >> SQL client in Flink 1.10. And since we already decided not introducing
> >> any new features to the
> >> original Flink planner, it already lacked of so many great features that
> >> the community has been working on, such as brand new type system, more
> DDL
> >> support and more planner capabilities.
> >> During this time, we've also received lots of great feedback from users
> >> who were trying to use blink
> >> planner, both positive and negative (like bugs). This is a good sign, it
> >> at least shows more and more
> >> users are starting to try out.
> >>
> >> So I want to start this discussion more formally to talk about
> >> replacing the default planner to blink.
> >> Specifically, I want to gather feedbacks from dev and user about whether
> >> blink planner already
> >> cover the original planner's capabilities, what kind of issues you've
> ran
> >> into when try out blink
> >> planner and then make you fallback to original one. Since there is still
> >> a month to go when feature
> >> freeze, there's still enough time for community to further enhance blink
> >> planner for this purpose.
> >>
> >> Let me know what you think, especially if you want to report or complain
> >> about something. Thanks
> >> in advance.
> >>
> >> Best,
> >> Kurt
> >>
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:[phone]
> > Email: [hidden email]; [hidden email]
> >
> >
>

--
Best, Jingsong Lee

[Timo Walther-2]

Hi Kurt,

+1 for making the Blink planner the default in 1.11.

One prerequisite that I see would be to resolve the remaining @Ignore
test cases in the code base.

One example that I recently spotted was here:

https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/test/scala/org/apache/flink/table/planner/plan/stream/table/TemporalTableJoinTest.scala

We should find a solution for those.

Apart from that the feature set is much bigger in 1.11 than in the
legacy planner and the APIs have settle after FLIP-65 and FLIP-95, which
make the Blink planner ready to be the default.

Regards,
Timo




On 01.04.20 03:15, Kurt Young wrote:

> Hi Dev and User,
>
> Blink planner for Table API & SQL is introduced in Flink 1.9 and already be
> the default planner for
> SQL client in Flink 1.10. And since we already decided not introducing any
> new features to the
> original Flink planner, it already lacked of so many great features that
> the community has been working on, such as brand new type system, more DDL
> support and more planner capabilities.
> During this time, we've also received lots of great feedback from users who
> were trying to use blink
> planner, both positive and negative (like bugs). This is a good sign, it at
> least shows more and more
> users are starting to try out.
>
> So I want to start this discussion more formally to talk about
> replacing the default planner to blink.
> Specifically, I want to gather feedbacks from dev and user about whether
> blink planner already
> cover the original planner's capabilities, what kind of issues you've ran
> into when try out blink
> planner and then make you fallback to original one. Since there is still a
> month to go when feature
> freeze, there's still enough time for community to further enhance blink
> planner for this purpose.
>
> Let me know what you think, especially if you want to report or complain
> about something. Thanks
> in advance.
>
> Best,
> Kurt
>

[Aljoscha Krettek-2]

+1 to making Blink the default planner, we definitely don't want to
maintain two planners for much longer.

Best,
Aljoscha

[Leonard Xu]

+1 to make the Blink planner be the planner in 1.11.

Blink planner has richer features than legacy planner and stable enough from now.

Best,
Leonard Xu

> 在 2020年4月1日，19:51，Aljoscha Krettek <[hidden email]> 写道：
>
> +1 to making Blink the default planner, we definitely don't want to maintain two planners for much longer.
>
> Best,
> Aljoscha

[Danny Chan]

+1 to use blink-planner as the default. It’s really awful to upgrade Calcite for both planners each time. I believe user would also want a default planner that is more powerful.

Best,
Danny Chan
在 2020年4月1日 +0800 PM10:26，Leonard Xu <[hidden email]>，写道：

> +1 to make the Blink planner be the planner in 1.11.
>
> Blink planner has richer features than legacy planner and stable enough from now.
>
> Best,
> Leonard Xu
>
> > 在 2020年4月1日，19:51，Aljoscha Krettek <[hidden email]> 写道：
> >
> > +1 to making Blink the default planner, we definitely don't want to maintain two planners for much longer.
> >
> > Best,
> > Aljoscha
>

[zoudan]

+1 to make the blink planner be the planner in 1.11.
We have already user blink planner as default planner in our production environment, and it works well.

Best,
Dan Zou

> 在 2020年4月1日，22:39，Danny Chan <[hidden email]> 写道：
>
> +1 to use blink-planner as the default. It’s really awful to upgrade Calcite for both planners each time. I believe user would also want a default planner that is more powerful.
>
> Best,
> Danny Chan
> 在 2020年4月1日 +0800 PM10:26，Leonard Xu <[hidden email]>，写道：
>> +1 to make the Blink planner be the planner in 1.11.
>>
>> Blink planner has richer features than legacy planner and stable enough from now.
>>
>> Best,
>> Leonard Xu
>>
>>> 在 2020年4月1日，19:51，Aljoscha Krettek <[hidden email]> 写道：
>>>
>>> +1 to making Blink the default planner, we definitely don't want to maintain two planners for much longer.
>>>
>>> Best,
>>> Aljoscha
>>

[Zhenghua Gao]

+1 to make blink planner as the default. It's more powerful and has richer
features.

*Best Regards,*
*Zhenghua Gao*


On Thu, Apr 2, 2020 at 11:12 AM zoudan <[hidden email]> wrote:

>
> +1 to make the blink planner be the planner in 1.11.
> We have already user blink planner as default planner in our production
> environment, and it works well.
>
> Best,
> Dan Zou
>
>
> > 在 2020年4月1日，22:39，Danny Chan <[hidden email]> 写道：
> >
> > +1 to use blink-planner as the default. It’s really awful to upgrade
> Calcite for both planners each time. I believe user would also want a
> default planner that is more powerful.
> >
> > Best,
> > Danny Chan
> > 在 2020年4月1日 +0800 PM10:26，Leonard Xu <[hidden email]>，写道：
> >> +1 to make the Blink planner be the planner in 1.11.
> >>
> >> Blink planner has richer features than legacy planner and stable enough
> from now.
> >>
> >> Best,
> >> Leonard Xu
> >>
> >>> 在 2020年4月1日，19:51，Aljoscha Krettek <[hidden email]> 写道：
> >>>
> >>> +1 to making Blink the default planner, we definitely don't want to
> maintain two planners for much longer.
> >>>
> >>> Best,
> >>> Aljoscha
> >>
>
>

[Kurt Young]

Thanks all for your feedback!

I've created FLINK-16934 [1] to track this effort.

@Timo I created a sub-task to this issue to track ignored test cases.

Any further suggestions or feedbacks are still appreciated.

[1] https://issues.apache.org/jira/browse/FLINK-16934

Best,
Kurt


On Thu, Apr 2, 2020 at 11:37 AM Zhenghua Gao <[hidden email]> wrote:

> +1 to make blink planner as the default. It's more powerful and has richer
> features.
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Thu, Apr 2, 2020 at 11:12 AM zoudan <[hidden email]> wrote:
>
> >
> > +1 to make the blink planner be the planner in 1.11.
> > We have already user blink planner as default planner in our production
> > environment, and it works well.
> >
> > Best,
> > Dan Zou
> >
> >
> > > 在 2020年4月1日，22:39，Danny Chan <[hidden email]> 写道：
> > >
> > > +1 to use blink-planner as the default. It’s really awful to upgrade
> > Calcite for both planners each time. I believe user would also want a
> > default planner that is more powerful.
> > >
> > > Best,
> > > Danny Chan
> > > 在 2020年4月1日 +0800 PM10:26，Leonard Xu <[hidden email]>，写道：
> > >> +1 to make the Blink planner be the planner in 1.11.
> > >>
> > >> Blink planner has richer features than legacy planner and stable
> enough
> > from now.
> > >>
> > >> Best,
> > >> Leonard Xu
> > >>
> > >>> 在 2020年4月1日，19:51，Aljoscha Krettek <[hidden email]> 写道：
> > >>>
> > >>> +1 to making Blink the default planner, we definitely don't want to
> > maintain two planners for much longer.
> > >>>
> > >>> Best,
> > >>> Aljoscha
> > >>
> >
> >
>