[DISCUSS] Make SQL docs Blink only

[Seth Wiesman-4]

Hi Everyone,

I've been spending a lot of time recently working on the SQL documentation
and I'm finding it very difficult to explain semantics as the two table
planners continue to diverge. As Blink has been the default planner for
some time, and 1.12 now offers bounded data stream support, how does the
community feel about making the documentation "blink only"?

We would update the documentation to assume users are always using the
Blink planner. As the legacy planner still exists we would create a
dedicated legacy planner page for users that have not migrated for whatever
reason - likely dataset interop. On this page, we would clearly list the
features that are not supported by the legacy planner and any semantics
that differ from the Blink planner.

Seth

[Timo Walther-2]

Hi Seth,

this is a very good idea. We might not be able to remove the legacy
planner immediately but at least we can make the docs easier for current
and future users of the Blink planner.

Making the SQL docs Blink-only with a dedicated legacy planner page
sounds good to me.

Regards,
Timo
On 08.12.20 16:36, Seth Wiesman wrote:

> Hi Everyone,
>
> I've been spending a lot of time recently working on the SQL documentation
> and I'm finding it very difficult to explain semantics as the two table
> planners continue to diverge. As Blink has been the default planner for
> some time, and 1.12 now offers bounded data stream support, how does the
> community feel about making the documentation "blink only"?
>
> We would update the documentation to assume users are always using the
> Blink planner. As the legacy planner still exists we would create a
> dedicated legacy planner page for users that have not migrated for whatever
> reason - likely dataset interop. On this page, we would clearly list the
> features that are not supported by the legacy planner and any semantics
> that differ from the Blink planner.
>
> Seth
>

[David Anderson-4]

I agree -- I think separating out the legacy planner info should make
things clearer for everyone, and then some day we can simply drop it. Plus,
doing it now will make it easier to make improvements to the docs going
forward.

David

On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]> wrote:

> Hi Seth,
>
> this is a very good idea. We might not be able to remove the legacy
> planner immediately but at least we can make the docs easier for current
> and future users of the Blink planner.
>
> Making the SQL docs Blink-only with a dedicated legacy planner page
> sounds good to me.
>
> Regards,
> Timo
> On 08.12.20 16:36, Seth Wiesman wrote:
> > Hi Everyone,
> >
> > I've been spending a lot of time recently working on the SQL
> documentation
> > and I'm finding it very difficult to explain semantics as the two table
> > planners continue to diverge. As Blink has been the default planner for
> > some time, and 1.12 now offers bounded data stream support, how does the
> > community feel about making the documentation "blink only"?
> >
> > We would update the documentation to assume users are always using the
> > Blink planner. As the legacy planner still exists we would create a
> > dedicated legacy planner page for users that have not migrated for
> whatever
> > reason - likely dataset interop. On this page, we would clearly list the
> > features that are not supported by the legacy planner and any semantics
> > that differ from the Blink planner.
> >
> > Seth
> >
>
>

[Aljoscha Krettek-2]

+1

Yes, please!

On 08.12.20 16:52, David Anderson wrote:

> I agree -- I think separating out the legacy planner info should make
> things clearer for everyone, and then some day we can simply drop it. Plus,
> doing it now will make it easier to make improvements to the docs going
> forward.
>
> David
>
> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]> wrote:
>
>> Hi Seth,
>>
>> this is a very good idea. We might not be able to remove the legacy
>> planner immediately but at least we can make the docs easier for current
>> and future users of the Blink planner.
>>
>> Making the SQL docs Blink-only with a dedicated legacy planner page
>> sounds good to me.
>>
>> Regards,
>> Timo
>> On 08.12.20 16:36, Seth Wiesman wrote:
>>> Hi Everyone,
>>>
>>> I've been spending a lot of time recently working on the SQL
>> documentation
>>> and I'm finding it very difficult to explain semantics as the two table
>>> planners continue to diverge. As Blink has been the default planner for
>>> some time, and 1.12 now offers bounded data stream support, how does the
>>> community feel about making the documentation "blink only"?
>>>
>>> We would update the documentation to assume users are always using the
>>> Blink planner. As the legacy planner still exists we would create a
>>> dedicated legacy planner page for users that have not migrated for
>> whatever
>>> reason - likely dataset interop. On this page, we would clearly list the
>>> features that are not supported by the legacy planner and any semantics
>>> that differ from the Blink planner.
>>>
>>> Seth
>>>
>>
>>
>

[Marta Paes Moreira]

+1, this is confusing (esp. for new users) and also creates more and more
"annotation clutter" as new features are added.

On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]> wrote:

> +1
>
> Yes, please!
>
> On 08.12.20 16:52, David Anderson wrote:
> > I agree -- I think separating out the legacy planner info should make
> > things clearer for everyone, and then some day we can simply drop it.
> Plus,
> > doing it now will make it easier to make improvements to the docs going
> > forward.
> >
> > David
> >
> > On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]> wrote:
> >
> >> Hi Seth,
> >>
> >> this is a very good idea. We might not be able to remove the legacy
> >> planner immediately but at least we can make the docs easier for current
> >> and future users of the Blink planner.
> >>
> >> Making the SQL docs Blink-only with a dedicated legacy planner page
> >> sounds good to me.
> >>
> >> Regards,
> >> Timo
> >> On 08.12.20 16:36, Seth Wiesman wrote:
> >>> Hi Everyone,
> >>>
> >>> I've been spending a lot of time recently working on the SQL
> >> documentation
> >>> and I'm finding it very difficult to explain semantics as the two table
> >>> planners continue to diverge. As Blink has been the default planner for
> >>> some time, and 1.12 now offers bounded data stream support, how does
> the
> >>> community feel about making the documentation "blink only"?
> >>>
> >>> We would update the documentation to assume users are always using the
> >>> Blink planner. As the legacy planner still exists we would create a
> >>> dedicated legacy planner page for users that have not migrated for
> >> whatever
> >>> reason - likely dataset interop. On this page, we would clearly list
> the
> >>> features that are not supported by the legacy planner and any semantics
> >>> that differ from the Blink planner.
> >>>
> >>> Seth
> >>>
> >>
> >>
> >
>
>

[Arvid Heise-3]

+1, add a small info box about legacy planner and point to 1.11 doc
(nothing should have changed)

Ideally, "legacy" and "blink" does not appear anywhere in the doc (except
for that info box)

On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <[hidden email]>
wrote:

> +1, this is confusing (esp. for new users) and also creates more and more
> "annotation clutter" as new features are added.
>
> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]>
> wrote:
>
> > +1
> >
> > Yes, please!
> >
> > On 08.12.20 16:52, David Anderson wrote:
> > > I agree -- I think separating out the legacy planner info should make
> > > things clearer for everyone, and then some day we can simply drop it.
> > Plus,
> > > doing it now will make it easier to make improvements to the docs going
> > > forward.
> > >
> > > David
> > >
> > > On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
> wrote:
> > >
> > >> Hi Seth,
> > >>
> > >> this is a very good idea. We might not be able to remove the legacy
> > >> planner immediately but at least we can make the docs easier for
> current
> > >> and future users of the Blink planner.
> > >>
> > >> Making the SQL docs Blink-only with a dedicated legacy planner page
> > >> sounds good to me.
> > >>
> > >> Regards,
> > >> Timo
> > >> On 08.12.20 16:36, Seth Wiesman wrote:
> > >>> Hi Everyone,
> > >>>
> > >>> I've been spending a lot of time recently working on the SQL
> > >> documentation
> > >>> and I'm finding it very difficult to explain semantics as the two
> table
> > >>> planners continue to diverge. As Blink has been the default planner
> for
> > >>> some time, and 1.12 now offers bounded data stream support, how does
> > the
> > >>> community feel about making the documentation "blink only"?
> > >>>
> > >>> We would update the documentation to assume users are always using
> the
> > >>> Blink planner. As the legacy planner still exists we would create a
> > >>> dedicated legacy planner page for users that have not migrated for
> > >> whatever
> > >>> reason - likely dataset interop. On this page, we would clearly list
> > the
> > >>> features that are not supported by the legacy planner and any
> semantics
> > >>> that differ from the Blink planner.
> > >>>
> > >>> Seth
> > >>>
> > >>
> > >>
> > >
> >
> >
>

--

Arvid Heise | Senior Java Developer

<https://www.ververica.com/>

Follow us @VervericaData

--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng

[Leonard Xu]

+1

Very good proposal, blink planner has been the default planner since Flink 1.11.0, and many new features are not supported in legacy planner.  

Best,
Leonard

> 在 2020年12月9日，05:15，Arvid Heise <[hidden email]> 写道：
>
> +1, add a small info box about legacy planner and point to 1.11 doc
> (nothing should have changed)
>
> Ideally, "legacy" and "blink" does not appear anywhere in the doc (except
> for that info box)
>
> On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <[hidden email]>
> wrote:
>
>> +1, this is confusing (esp. for new users) and also creates more and more
>> "annotation clutter" as new features are added.
>>
>> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>>> +1
>>>
>>> Yes, please!
>>>
>>> On 08.12.20 16:52, David Anderson wrote:
>>>> I agree -- I think separating out the legacy planner info should make
>>>> things clearer for everyone, and then some day we can simply drop it.
>>> Plus,
>>>> doing it now will make it easier to make improvements to the docs going
>>>> forward.
>>>>
>>>> David
>>>>
>>>> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
>> wrote:
>>>>
>>>>> Hi Seth,
>>>>>
>>>>> this is a very good idea. We might not be able to remove the legacy
>>>>> planner immediately but at least we can make the docs easier for
>> current
>>>>> and future users of the Blink planner.
>>>>>
>>>>> Making the SQL docs Blink-only with a dedicated legacy planner page
>>>>> sounds good to me.
>>>>>
>>>>> Regards,
>>>>> Timo
>>>>> On 08.12.20 16:36, Seth Wiesman wrote:
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I've been spending a lot of time recently working on the SQL
>>>>> documentation
>>>>>> and I'm finding it very difficult to explain semantics as the two
>> table
>>>>>> planners continue to diverge. As Blink has been the default planner
>> for
>>>>>> some time, and 1.12 now offers bounded data stream support, how does
>>> the
>>>>>> community feel about making the documentation "blink only"?
>>>>>>
>>>>>> We would update the documentation to assume users are always using
>> the
>>>>>> Blink planner. As the legacy planner still exists we would create a
>>>>>> dedicated legacy planner page for users that have not migrated for
>>>>> whatever
>>>>>> reason - likely dataset interop. On this page, we would clearly list
>>> the
>>>>>> features that are not supported by the legacy planner and any
>> semantics
>>>>>> that differ from the Blink planner.
>>>>>>
>>>>>> Seth
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
> --
>
> Arvid Heise | Senior Java Developer
>
> <https://www.ververica.com/>
>
> Follow us @VervericaData
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Toni) Cheng

[Xingbo Huang]

+1

This is a very good proposal.In release-1.12, many newly added features are
only supported on the blink planner. For example, the newly added features
of PyFlnk in FLIP-137[1] and FLIP-139[2] are only available on the blink
planner.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-137%3A+Support+Pandas+UDAF+in+PyFlink
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-139%3A+General+Python+User-Defined+Aggregate+Function+Support+on+Table+API

Best,
Xingbo

Leonard Xu <[hidden email]> 于2020年12月9日周三 上午9:46写道：

> +1
>
> Very good proposal, blink planner has been the default planner since Flink
> 1.11.0, and many new features are not supported in legacy planner.
>
> Best,
> Leonard
>
> > 在 2020年12月9日，05:15，Arvid Heise <[hidden email]> 写道：
> >
> > +1, add a small info box about legacy planner and point to 1.11 doc
> > (nothing should have changed)
> >
> > Ideally, "legacy" and "blink" does not appear anywhere in the doc (except
> > for that info box)
> >
> > On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <[hidden email]>
> > wrote:
> >
> >> +1, this is confusing (esp. for new users) and also creates more and
> more
> >> "annotation clutter" as new features are added.
> >>
> >> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]>
> >> wrote:
> >>
> >>> +1
> >>>
> >>> Yes, please!
> >>>
> >>> On 08.12.20 16:52, David Anderson wrote:
> >>>> I agree -- I think separating out the legacy planner info should make
> >>>> things clearer for everyone, and then some day we can simply drop it.
> >>> Plus,
> >>>> doing it now will make it easier to make improvements to the docs
> going
> >>>> forward.
> >>>>
> >>>> David
> >>>>
> >>>> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
> >> wrote:
> >>>>
> >>>>> Hi Seth,
> >>>>>
> >>>>> this is a very good idea. We might not be able to remove the legacy
> >>>>> planner immediately but at least we can make the docs easier for
> >> current
> >>>>> and future users of the Blink planner.
> >>>>>
> >>>>> Making the SQL docs Blink-only with a dedicated legacy planner page
> >>>>> sounds good to me.
> >>>>>
> >>>>> Regards,
> >>>>> Timo
> >>>>> On 08.12.20 16:36, Seth Wiesman wrote:
> >>>>>> Hi Everyone,
> >>>>>>
> >>>>>> I've been spending a lot of time recently working on the SQL
> >>>>> documentation
> >>>>>> and I'm finding it very difficult to explain semantics as the two
> >> table
> >>>>>> planners continue to diverge. As Blink has been the default planner
> >> for
> >>>>>> some time, and 1.12 now offers bounded data stream support, how does
> >>> the
> >>>>>> community feel about making the documentation "blink only"?
> >>>>>>
> >>>>>> We would update the documentation to assume users are always using
> >> the
> >>>>>> Blink planner. As the legacy planner still exists we would create a
> >>>>>> dedicated legacy planner page for users that have not migrated for
> >>>>> whatever
> >>>>>> reason - likely dataset interop. On this page, we would clearly list
> >>> the
> >>>>>> features that are not supported by the legacy planner and any
> >> semantics
> >>>>>> that differ from the Blink planner.
> >>>>>>
> >>>>>> Seth
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> > --
> >
> > Arvid Heise | Senior Java Developer
> >
> > <https://www.ververica.com/>
> >
> > Follow us @VervericaData
> >
> > --
> >
> > Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> > Conference
> >
> > Stream Processing | Event Driven | Real Time
> >
> > --
> >
> > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> >
> > --
> > Ververica GmbH
> > Registered at Amtsgericht Charlottenburg: HRB 158244 B
> > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> > (Toni) Cheng
>
>

[Jark Wu-2]

+1

This is a very good idea.

Best,
Jark

On Wed, 9 Dec 2020 at 10:43, Xingbo Huang <[hidden email]> wrote:

> +1
>
> This is a very good proposal.In release-1.12, many newly added features are
> only supported on the blink planner. For example, the newly added features
> of PyFlnk in FLIP-137[1] and FLIP-139[2] are only available on the blink
> planner.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-137%3A+Support+Pandas+UDAF+in+PyFlink
> [2]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-139%3A+General+Python+User-Defined+Aggregate+Function+Support+on+Table+API
>
> Best,
> Xingbo
>
> Leonard Xu <[hidden email]> 于2020年12月9日周三 上午9:46写道：
>
> > +1
> >
> > Very good proposal, blink planner has been the default planner since
> Flink
> > 1.11.0, and many new features are not supported in legacy planner.
> >
> > Best,
> > Leonard
> >
> > > 在 2020年12月9日，05:15，Arvid Heise <[hidden email]> 写道：
> > >
> > > +1, add a small info box about legacy planner and point to 1.11 doc
> > > (nothing should have changed)
> > >
> > > Ideally, "legacy" and "blink" does not appear anywhere in the doc
> (except
> > > for that info box)
> > >
> > > On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <[hidden email]
> >
> > > wrote:
> > >
> > >> +1, this is confusing (esp. for new users) and also creates more and
> > more
> > >> "annotation clutter" as new features are added.
> > >>
> > >> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]>
> > >> wrote:
> > >>
> > >>> +1
> > >>>
> > >>> Yes, please!
> > >>>
> > >>> On 08.12.20 16:52, David Anderson wrote:
> > >>>> I agree -- I think separating out the legacy planner info should
> make
> > >>>> things clearer for everyone, and then some day we can simply drop
> it.
> > >>> Plus,
> > >>>> doing it now will make it easier to make improvements to the docs
> > going
> > >>>> forward.
> > >>>>
> > >>>> David
> > >>>>
> > >>>> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
> > >> wrote:
> > >>>>
> > >>>>> Hi Seth,
> > >>>>>
> > >>>>> this is a very good idea. We might not be able to remove the legacy
> > >>>>> planner immediately but at least we can make the docs easier for
> > >> current
> > >>>>> and future users of the Blink planner.
> > >>>>>
> > >>>>> Making the SQL docs Blink-only with a dedicated legacy planner page
> > >>>>> sounds good to me.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Timo
> > >>>>> On 08.12.20 16:36, Seth Wiesman wrote:
> > >>>>>> Hi Everyone,
> > >>>>>>
> > >>>>>> I've been spending a lot of time recently working on the SQL
> > >>>>> documentation
> > >>>>>> and I'm finding it very difficult to explain semantics as the two
> > >> table
> > >>>>>> planners continue to diverge. As Blink has been the default
> planner
> > >> for
> > >>>>>> some time, and 1.12 now offers bounded data stream support, how
> does
> > >>> the
> > >>>>>> community feel about making the documentation "blink only"?
> > >>>>>>
> > >>>>>> We would update the documentation to assume users are always using
> > >> the
> > >>>>>> Blink planner. As the legacy planner still exists we would create
> a
> > >>>>>> dedicated legacy planner page for users that have not migrated for
> > >>>>> whatever
> > >>>>>> reason - likely dataset interop. On this page, we would clearly
> list
> > >>> the
> > >>>>>> features that are not supported by the legacy planner and any
> > >> semantics
> > >>>>>> that differ from the Blink planner.
> > >>>>>>
> > >>>>>> Seth
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> > > --
> > >
> > > Arvid Heise | Senior Java Developer
> > >
> > > <https://www.ververica.com/>
> > >
> > > Follow us @VervericaData
> > >
> > > --
> > >
> > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> > > Conference
> > >
> > > Stream Processing | Event Driven | Real Time
> > >
> > > --
> > >
> > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> > >
> > > --
> > > Ververica GmbH
> > > Registered at Amtsgericht Charlottenburg: HRB 158244 B
> > > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> > > (Toni) Cheng
> >
> >
>

[Wei Zhong-2]

+1

This is a very good proposal. The Python Table API documentation also contains some code that still uses the old planner. We may need to do the same for the Python Table API documentation in the future.

Best,
Wei

> 在 2020年12月9日，12:35，Jark Wu <[hidden email]> 写道：
>
> +1
>
> This is a very good idea.
>
> Best,
> Jark
>
> On Wed, 9 Dec 2020 at 10:43, Xingbo Huang <[hidden email]> wrote:
>
>> +1
>>
>> This is a very good proposal.In release-1.12, many newly added features are
>> only supported on the blink planner. For example, the newly added features
>> of PyFlnk in FLIP-137[1] and FLIP-139[2] are only available on the blink
>> planner.
>>
>> [1]
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-137%3A+Support+Pandas+UDAF+in+PyFlink
>> [2]
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-139%3A+General+Python+User-Defined+Aggregate+Function+Support+on+Table+API
>>
>> Best,
>> Xingbo
>>
>> Leonard Xu <[hidden email]> 于2020年12月9日周三 上午9:46写道：
>>
>>> +1
>>>
>>> Very good proposal, blink planner has been the default planner since
>> Flink
>>> 1.11.0, and many new features are not supported in legacy planner.
>>>
>>> Best,
>>> Leonard
>>>
>>>> 在 2020年12月9日，05:15，Arvid Heise <[hidden email]> 写道：
>>>>
>>>> +1, add a small info box about legacy planner and point to 1.11 doc
>>>> (nothing should have changed)
>>>>
>>>> Ideally, "legacy" and "blink" does not appear anywhere in the doc
>> (except
>>>> for that info box)
>>>>
>>>> On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <[hidden email]
>>>
>>>> wrote:
>>>>
>>>>> +1, this is confusing (esp. for new users) and also creates more and
>>> more
>>>>> "annotation clutter" as new features are added.
>>>>>
>>>>> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Yes, please!
>>>>>>
>>>>>> On 08.12.20 16:52, David Anderson wrote:
>>>>>>> I agree -- I think separating out the legacy planner info should
>> make
>>>>>>> things clearer for everyone, and then some day we can simply drop
>> it.
>>>>>> Plus,
>>>>>>> doing it now will make it easier to make improvements to the docs
>>> going
>>>>>>> forward.
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Seth,
>>>>>>>>
>>>>>>>> this is a very good idea. We might not be able to remove the legacy
>>>>>>>> planner immediately but at least we can make the docs easier for
>>>>> current
>>>>>>>> and future users of the Blink planner.
>>>>>>>>
>>>>>>>> Making the SQL docs Blink-only with a dedicated legacy planner page
>>>>>>>> sounds good to me.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Timo
>>>>>>>> On 08.12.20 16:36, Seth Wiesman wrote:
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I've been spending a lot of time recently working on the SQL
>>>>>>>> documentation
>>>>>>>>> and I'm finding it very difficult to explain semantics as the two
>>>>> table
>>>>>>>>> planners continue to diverge. As Blink has been the default
>> planner
>>>>> for
>>>>>>>>> some time, and 1.12 now offers bounded data stream support, how
>> does
>>>>>> the
>>>>>>>>> community feel about making the documentation "blink only"?
>>>>>>>>>
>>>>>>>>> We would update the documentation to assume users are always using
>>>>> the
>>>>>>>>> Blink planner. As the legacy planner still exists we would create
>> a
>>>>>>>>> dedicated legacy planner page for users that have not migrated for
>>>>>>>> whatever
>>>>>>>>> reason - likely dataset interop. On this page, we would clearly
>> list
>>>>>> the
>>>>>>>>> features that are not supported by the legacy planner and any
>>>>> semantics
>>>>>>>>> that differ from the Blink planner.
>>>>>>>>>
>>>>>>>>> Seth
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Arvid Heise | Senior Java Developer
>>>>
>>>> <https://www.ververica.com/>
>>>>
>>>> Follow us @VervericaData
>>>>
>>>> --
>>>>
>>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
>>>> Conference
>>>>
>>>> Stream Processing | Event Driven | Real Time
>>>>
>>>> --
>>>>
>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>>>>
>>>> --
>>>> Ververica GmbH
>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B
>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
>>>> (Toni) Cheng
>>>
>>>
>>

[Seth Wiesman-4]

Great. Easiest discussion thread ever.

https://issues.apache.org/jira/browse/FLINK-20551

On Wed, Dec 9, 2020 at 6:55 AM Wei Zhong <[hidden email]> wrote:

> +1
>
> This is a very good proposal. The Python Table API documentation also
> contains some code that still uses the old planner. We may need to do the
> same for the Python Table API documentation in the future.
>
> Best,
> Wei
>
> > 在 2020年12月9日，12:35，Jark Wu <[hidden email]> 写道：
> >
> > +1
> >
> > This is a very good idea.
> >
> > Best,
> > Jark
> >
> > On Wed, 9 Dec 2020 at 10:43, Xingbo Huang <[hidden email]> wrote:
> >
> >> +1
> >>
> >> This is a very good proposal.In release-1.12, many newly added features
> are
> >> only supported on the blink planner. For example, the newly added
> features
> >> of PyFlnk in FLIP-137[1] and FLIP-139[2] are only available on the blink
> >> planner.
> >>
> >> [1]
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-137%3A+Support+Pandas+UDAF+in+PyFlink
> >> [2]
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-139%3A+General+Python+User-Defined+Aggregate+Function+Support+on+Table+API
> >>
> >> Best,
> >> Xingbo
> >>
> >> Leonard Xu <[hidden email]> 于2020年12月9日周三 上午9:46写道：
> >>
> >>> +1
> >>>
> >>> Very good proposal, blink planner has been the default planner since
> >> Flink
> >>> 1.11.0, and many new features are not supported in legacy planner.
> >>>
> >>> Best,
> >>> Leonard
> >>>
> >>>> 在 2020年12月9日，05:15，Arvid Heise <[hidden email]> 写道：
> >>>>
> >>>> +1, add a small info box about legacy planner and point to 1.11 doc
> >>>> (nothing should have changed)
> >>>>
> >>>> Ideally, "legacy" and "blink" does not appear anywhere in the doc
> >> (except
> >>>> for that info box)
> >>>>
> >>>> On Tue, Dec 8, 2020 at 6:35 PM Marta Paes Moreira <
> [hidden email]
> >>>
> >>>> wrote:
> >>>>
> >>>>> +1, this is confusing (esp. for new users) and also creates more and
> >>> more
> >>>>> "annotation clutter" as new features are added.
> >>>>>
> >>>>> On Tue, Dec 8, 2020 at 5:30 PM Aljoscha Krettek <[hidden email]
> >
> >>>>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> Yes, please!
> >>>>>>
> >>>>>> On 08.12.20 16:52, David Anderson wrote:
> >>>>>>> I agree -- I think separating out the legacy planner info should
> >> make
> >>>>>>> things clearer for everyone, and then some day we can simply drop
> >> it.
> >>>>>> Plus,
> >>>>>>> doing it now will make it easier to make improvements to the docs
> >>> going
> >>>>>>> forward.
> >>>>>>>
> >>>>>>> David
> >>>>>>>
> >>>>>>> On Tue, Dec 8, 2020 at 4:38 PM Timo Walther <[hidden email]>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Seth,
> >>>>>>>>
> >>>>>>>> this is a very good idea. We might not be able to remove the
> legacy
> >>>>>>>> planner immediately but at least we can make the docs easier for
> >>>>> current
> >>>>>>>> and future users of the Blink planner.
> >>>>>>>>
> >>>>>>>> Making the SQL docs Blink-only with a dedicated legacy planner
> page
> >>>>>>>> sounds good to me.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Timo
> >>>>>>>> On 08.12.20 16:36, Seth Wiesman wrote:
> >>>>>>>>> Hi Everyone,
> >>>>>>>>>
> >>>>>>>>> I've been spending a lot of time recently working on the SQL
> >>>>>>>> documentation
> >>>>>>>>> and I'm finding it very difficult to explain semantics as the two
> >>>>> table
> >>>>>>>>> planners continue to diverge. As Blink has been the default
> >> planner
> >>>>> for
> >>>>>>>>> some time, and 1.12 now offers bounded data stream support, how
> >> does
> >>>>>> the
> >>>>>>>>> community feel about making the documentation "blink only"?
> >>>>>>>>>
> >>>>>>>>> We would update the documentation to assume users are always
> using
> >>>>> the
> >>>>>>>>> Blink planner. As the legacy planner still exists we would create
> >> a
> >>>>>>>>> dedicated legacy planner page for users that have not migrated
> for
> >>>>>>>> whatever
> >>>>>>>>> reason - likely dataset interop. On this page, we would clearly
> >> list
> >>>>>> the
> >>>>>>>>> features that are not supported by the legacy planner and any
> >>>>> semantics
> >>>>>>>>> that differ from the Blink planner.
> >>>>>>>>>
> >>>>>>>>> Seth
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>> Arvid Heise | Senior Java Developer
> >>>>
> >>>> <https://www.ververica.com/>
> >>>>
> >>>> Follow us @VervericaData
> >>>>
> >>>> --
> >>>>
> >>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> >>>> Conference
> >>>>
> >>>> Stream Processing | Event Driven | Real Time
> >>>>
> >>>> --
> >>>>
> >>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> >>>>
> >>>> --
> >>>> Ververica GmbH
> >>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> >>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason,
> Ji
> >>>> (Toni) Cheng
> >>>
> >>>
> >>
>
>