[DISCUSS] FLIP-71 - E2E View support in Flink SQL

[Zhenghua Gao]

Hi folks,

In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW grammar in
our module flink-sql-parser. But we don't support view objects in neither
blink planner nor old planner.

I'd like to kick off a discussion on end to end view support in Flink SQL
in blink planner. It's helpful to improve the usability of the framework
for SQL users.

https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p

In short, it:
- support define views and store them in catalog
- support drop view definitions from catalog
- support query views
- support other view related DDLs

Any comments and feedbacks are welcome and appreciated. Thanks.

*Best Regards,*
*Zhenghua Gao*

[Zhenghua Gao]

Hi community,

It's a long time since we started the discussion of supporting view in
FLINK SQL.
Flink also continues to move forward since then.
FLINK-10232 introduces the grammar and FLINK-12905 supports CatalogView in
blink planner.
The missing link is validate the view definition and store the
original/expanded text in the catalog.
I have updated the design doc of FLIP-71,
https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL

Any comments and feedbacks are welcome and appreciated. Thanks.

*Best Regards,*
*Zhenghua Gao*


On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]> wrote:

> Hi folks,
>
> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW grammar
> in our module flink-sql-parser. But we don't support view objects in
> neither blink planner nor old planner.
>
> I'd like to kick off a discussion on end to end view support in Flink SQL
> in blink planner. It's helpful to improve the usability of the framework
> for SQL users.
>
>
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>
> In short, it:
> - support define views and store them in catalog
> - support drop view definitions from catalog
> - support query views
> - support other view related DDLs
>
> Any comments and feedbacks are welcome and appreciated. Thanks.
>
> *Best Regards,*
> *Zhenghua Gao*
>

[Zhenghua Gao]

forward the reply to ML too.


*Best Regards,*
*Zhenghua Gao*


---------- Forwarded message ---------
From: Zhenghua Gao <[hidden email]>
Date: Tue, Apr 7, 2020 at 8:40 PM
Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
To: godfrey he <[hidden email]>


>> regarding to "Interoperability between Flink and Hive is not
guaranteed", can you explain this more?
We have several limitations of interoperability between flink objects and
hive objects (tables, functions, etc).
So we don't promise the interoperability of views between flink and hive
since a view is defined base on these objects.

>> "CREATE VIEW [ IF NOT EXISTS ]"
This should be "CREATE VIEW [OR REPLACE]".

>> "DESC"
It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".

I have updated the design doc, thanks.

*Best Regards,*
*Zhenghua Gao*


On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]> wrote:

> Hi Zhenghua,
>
> Thanks for driving this. It's one step forward that TableEnvironment
> supports more complete SQLs.
> I have a few minor questions:
> 1. regarding to "Interoperability between Flink and Hive is not
> guaranteed", can you explain this more?
> 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW [ IF
> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS" and
> "DESCRIBE VIEW xx" are not supported now. Calcite does not support describe
> an object through "DESC" . I think It's better this flip can support "SHOW
> VIEWS" and "DESCRIBE VIEW xx".
>
> Best,
> Godfrey
>
> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
>
>> Hi community,
>>
>> It's a long time since we started the discussion of supporting view in
>> FLINK SQL.
>> Flink also continues to move forward since then.
>> FLINK-10232 introduces the grammar and FLINK-12905 supports CatalogView
>> in blink planner.
>> The missing link is validate the view definition and store the
>> original/expanded text in the catalog.
>> I have updated the design doc of FLIP-71,
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
>>
>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>
>> *Best Regards,*
>> *Zhenghua Gao*
>>
>>
>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]> wrote:
>>
>>> Hi folks,
>>>
>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW grammar
>>> in our module flink-sql-parser. But we don't support view objects in
>>> neither blink planner nor old planner.
>>>
>>> I'd like to kick off a discussion on end to end view support in Flink
>>> SQL in blink planner. It's helpful to improve the usability of the
>>> framework for SQL users.
>>>
>>>
>>> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>>>
>>> In short, it:
>>> - support define views and store them in catalog
>>> - support drop view definitions from catalog
>>> - support query views
>>> - support other view related DDLs
>>>
>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>
>>> *Best Regards,*
>>> *Zhenghua Gao*
>>>
>>

[Kurt Young]

This FLIP seems to be quite straightforward, +1 from my side.

Best,
Kurt


On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:

> forward the reply to ML too.
>
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> ---------- Forwarded message ---------
> From: Zhenghua Gao <[hidden email]>
> Date: Tue, Apr 7, 2020 at 8:40 PM
> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> To: godfrey he <[hidden email]>
>
>
> >> regarding to "Interoperability between Flink and Hive is not
> guaranteed", can you explain this more?
> We have several limitations of interoperability between flink objects and
> hive objects (tables, functions, etc).
> So we don't promise the interoperability of views between flink and hive
> since a view is defined base on these objects.
>
> >> "CREATE VIEW [ IF NOT EXISTS ]"
> This should be "CREATE VIEW [OR REPLACE]".
>
> >> "DESC"
> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
>
> I have updated the design doc, thanks.
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]> wrote:
>
> > Hi Zhenghua,
> >
> > Thanks for driving this. It's one step forward that TableEnvironment
> > supports more complete SQLs.
> > I have a few minor questions:
> > 1. regarding to "Interoperability between Flink and Hive is not
> > guaranteed", can you explain this more?
> > 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW [ IF
> > NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS" and
> > "DESCRIBE VIEW xx" are not supported now. Calcite does not support
> describe
> > an object through "DESC" . I think It's better this flip can support
> "SHOW
> > VIEWS" and "DESCRIBE VIEW xx".
> >
> > Best,
> > Godfrey
> >
> > Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> >
> >> Hi community,
> >>
> >> It's a long time since we started the discussion of supporting view in
> >> FLINK SQL.
> >> Flink also continues to move forward since then.
> >> FLINK-10232 introduces the grammar and FLINK-12905 supports CatalogView
> >> in blink planner.
> >> The missing link is validate the view definition and store the
> >> original/expanded text in the catalog.
> >> I have updated the design doc of FLIP-71,
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> >>
> >> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>
> >> *Best Regards,*
> >> *Zhenghua Gao*
> >>
> >>
> >> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]> wrote:
> >>
> >>> Hi folks,
> >>>
> >>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
> grammar
> >>> in our module flink-sql-parser. But we don't support view objects in
> >>> neither blink planner nor old planner.
> >>>
> >>> I'd like to kick off a discussion on end to end view support in Flink
> >>> SQL in blink planner. It's helpful to improve the usability of the
> >>> framework for SQL users.
> >>>
> >>>
> >>>
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> >>>
> >>> In short, it:
> >>> - support define views and store them in catalog
> >>> - support drop view definitions from catalog
> >>> - support query views
> >>> - support other view related DDLs
> >>>
> >>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>
> >>> *Best Regards,*
> >>> *Zhenghua Gao*
> >>>
> >>
>

[Danny Chan]

+1 for the proposal, a small concern for drop view statement:

dropViewStatement:
      DROP VIEW name [ IF EXISTS ]
I think the drop statement should be
DROP VIEW [ IF EXISTS ] name

Best,
Danny Chan
在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：

> This FLIP seems to be quite straightforward, +1 from my side.
>
> Best,
> Kurt
>
>
> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
>
> > forward the reply to ML too.
> >
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > ---------- Forwarded message ---------
> > From: Zhenghua Gao <[hidden email]>
> > Date: Tue, Apr 7, 2020 at 8:40 PM
> > Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> > To: godfrey he <[hidden email]>
> >
> >
> > > > regarding to "Interoperability between Flink and Hive is not
> > guaranteed", can you explain this more?
> > We have several limitations of interoperability between flink objects and
> > hive objects (tables, functions, etc).
> > So we don't promise the interoperability of views between flink and hive
> > since a view is defined base on these objects.
> >
> > > > "CREATE VIEW [ IF NOT EXISTS ]"
> > This should be "CREATE VIEW [OR REPLACE]".
> >
> > > > "DESC"
> > It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> > In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
> >
> > I have updated the design doc, thanks.
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]> wrote:
> >
> > > Hi Zhenghua,
> > >
> > > Thanks for driving this. It's one step forward that TableEnvironment
> > > supports more complete SQLs.
> > > I have a few minor questions:
> > > 1. regarding to "Interoperability between Flink and Hive is not
> > > guaranteed", can you explain this more?
> > > 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW [ IF
> > > NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS" and
> > > "DESCRIBE VIEW xx" are not supported now. Calcite does not support
> > describe
> > > an object through "DESC" . I think It's better this flip can support
> > "SHOW
> > > VIEWS" and "DESCRIBE VIEW xx".
> > >
> > > Best,
> > > Godfrey
> > >
> > > Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> > >
> > > > Hi community,
> > > >
> > > > It's a long time since we started the discussion of supporting view in
> > > > FLINK SQL.
> > > > Flink also continues to move forward since then.
> > > > FLINK-10232 introduces the grammar and FLINK-12905 supports CatalogView
> > > > in blink planner.
> > > > The missing link is validate the view definition and store the
> > > > original/expanded text in the catalog.
> > > > I have updated the design doc of FLIP-71,
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> > > >
> > > > Any comments and feedbacks are welcome and appreciated. Thanks.
> > > >
> > > > *Best Regards,*
> > > > *Zhenghua Gao*
> > > >
> > > >
> > > > On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]> wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
> > grammar
> > > > > in our module flink-sql-parser. But we don't support view objects in
> > > > > neither blink planner nor old planner.
> > > > >
> > > > > I'd like to kick off a discussion on end to end view support in Flink
> > > > > SQL in blink planner. It's helpful to improve the usability of the
> > > > > framework for SQL users.
> > > > >
> > > > >
> > > > >
> > https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> > > > >
> > > > > In short, it:
> > > > > - support define views and store them in catalog
> > > > > - support drop view definitions from catalog
> > > > > - support query views
> > > > > - support other view related DDLs
> > > > >
> > > > > Any comments and feedbacks are welcome and appreciated. Thanks.
> > > > >
> > > > > *Best Regards,*
> > > > > *Zhenghua Gao*
> > > > >
> > > >
> >

[Zhenghua Gao]

@Danny Chan <[hidden email]>  you‘re right. I have updated the doc.

*Best Regards,*
*Zhenghua Gao*


On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]> wrote:

> +1 for the proposal, a small concern for drop view statement:
>
> dropViewStatement:
>       DROP VIEW name [ IF EXISTS ]
> I think the drop statement should be
> DROP VIEW [ IF EXISTS ] name
>
> Best,
> Danny Chan
> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
> > This FLIP seems to be quite straightforward, +1 from my side.
> >
> > Best,
> > Kurt
> >
> >
> > On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
> >
> > > forward the reply to ML too.
> > >
> > >
> > > *Best Regards,*
> > > *Zhenghua Gao*
> > >
> > >
> > > ---------- Forwarded message ---------
> > > From: Zhenghua Gao <[hidden email]>
> > > Date: Tue, Apr 7, 2020 at 8:40 PM
> > > Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> > > To: godfrey he <[hidden email]>
> > >
> > >
> > > > > regarding to "Interoperability between Flink and Hive is not
> > > guaranteed", can you explain this more?
> > > We have several limitations of interoperability between flink objects
> and
> > > hive objects (tables, functions, etc).
> > > So we don't promise the interoperability of views between flink and
> hive
> > > since a view is defined base on these objects.
> > >
> > > > > "CREATE VIEW [ IF NOT EXISTS ]"
> > > This should be "CREATE VIEW [OR REPLACE]".
> > >
> > > > > "DESC"
> > > It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> > > In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
> > >
> > > I have updated the design doc, thanks.
> > >
> > > *Best Regards,*
> > > *Zhenghua Gao*
> > >
> > >
> > > On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]> wrote:
> > >
> > > > Hi Zhenghua,
> > > >
> > > > Thanks for driving this. It's one step forward that TableEnvironment
> > > > supports more complete SQLs.
> > > > I have a few minor questions:
> > > > 1. regarding to "Interoperability between Flink and Hive is not
> > > > guaranteed", can you explain this more?
> > > > 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW
> [ IF
> > > > NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS"
> and
> > > > "DESCRIBE VIEW xx" are not supported now. Calcite does not support
> > > describe
> > > > an object through "DESC" . I think It's better this flip can support
> > > "SHOW
> > > > VIEWS" and "DESCRIBE VIEW xx".
> > > >
> > > > Best,
> > > > Godfrey
> > > >
> > > > Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> > > >
> > > > > Hi community,
> > > > >
> > > > > It's a long time since we started the discussion of supporting
> view in
> > > > > FLINK SQL.
> > > > > Flink also continues to move forward since then.
> > > > > FLINK-10232 introduces the grammar and FLINK-12905 supports
> CatalogView
> > > > > in blink planner.
> > > > > The missing link is validate the view definition and store the
> > > > > original/expanded text in the catalog.
> > > > > I have updated the design doc of FLIP-71,
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> > > > >
> > > > > Any comments and feedbacks are welcome and appreciated. Thanks.
> > > > >
> > > > > *Best Regards,*
> > > > > *Zhenghua Gao*
> > > > >
> > > > >
> > > > > On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
> wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
> > > grammar
> > > > > > in our module flink-sql-parser. But we don't support view
> objects in
> > > > > > neither blink planner nor old planner.
> > > > > >
> > > > > > I'd like to kick off a discussion on end to end view support in
> Flink
> > > > > > SQL in blink planner. It's helpful to improve the usability of
> the
> > > > > > framework for SQL users.
> > > > > >
> > > > > >
> > > > > >
> > >
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> > > > > >
> > > > > > In short, it:
> > > > > > - support define views and store them in catalog
> > > > > > - support drop view definitions from catalog
> > > > > > - support query views
> > > > > > - support other view related DDLs
> > > > > >
> > > > > > Any comments and feedbacks are welcome and appreciated. Thanks.
> > > > > >
> > > > > > *Best Regards,*
> > > > > > *Zhenghua Gao*
> > > > > >
> > > > >
> > >
>

[Timo Walther-2]

Hi Zhenghua,

VIEWS should also support the TEMPORARY keyword according to FLIP-64.

Otherwise the FLIP looks good to me.

Regards,
Timo


On 08.04.20 07:31, Zhenghua Gao wrote:

> @Danny Chan <[hidden email]>  you‘re right. I have updated the doc.
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]> wrote:
>
>> +1 for the proposal, a small concern for drop view statement:
>>
>> dropViewStatement:
>>        DROP VIEW name [ IF EXISTS ]
>> I think the drop statement should be
>> DROP VIEW [ IF EXISTS ] name
>>
>> Best,
>> Danny Chan
>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
>>> This FLIP seems to be quite straightforward, +1 from my side.
>>>
>>> Best,
>>> Kurt
>>>
>>>
>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
>>>
>>>> forward the reply to ML too.
>>>>
>>>>
>>>> *Best Regards,*
>>>> *Zhenghua Gao*
>>>>
>>>>
>>>> ---------- Forwarded message ---------
>>>> From: Zhenghua Gao <[hidden email]>
>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
>>>> To: godfrey he <[hidden email]>
>>>>
>>>>
>>>>>> regarding to "Interoperability between Flink and Hive is not
>>>> guaranteed", can you explain this more?
>>>> We have several limitations of interoperability between flink objects
>> and
>>>> hive objects (tables, functions, etc).
>>>> So we don't promise the interoperability of views between flink and
>> hive
>>>> since a view is defined base on these objects.
>>>>
>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
>>>> This should be "CREATE VIEW [OR REPLACE]".
>>>>
>>>>>> "DESC"
>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
>>>>
>>>> I have updated the design doc, thanks.
>>>>
>>>> *Best Regards,*
>>>> *Zhenghua Gao*
>>>>
>>>>
>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]> wrote:
>>>>
>>>>> Hi Zhenghua,
>>>>>
>>>>> Thanks for driving this. It's one step forward that TableEnvironment
>>>>> supports more complete SQLs.
>>>>> I have a few minor questions:
>>>>> 1. regarding to "Interoperability between Flink and Hive is not
>>>>> guaranteed", can you explain this more?
>>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW
>> [ IF
>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS"
>> and
>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
>>>> describe
>>>>> an object through "DESC" . I think It's better this flip can support
>>>> "SHOW
>>>>> VIEWS" and "DESCRIBE VIEW xx".
>>>>>
>>>>> Best,
>>>>> Godfrey
>>>>>
>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
>>>>>
>>>>>> Hi community,
>>>>>>
>>>>>> It's a long time since we started the discussion of supporting
>> view in
>>>>>> FLINK SQL.
>>>>>> Flink also continues to move forward since then.
>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
>> CatalogView
>>>>>> in blink planner.
>>>>>> The missing link is validate the view definition and store the
>>>>>> original/expanded text in the catalog.
>>>>>> I have updated the design doc of FLIP-71,
>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
>>>>>>
>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>
>>>>>> *Best Regards,*
>>>>>> *Zhenghua Gao*
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
>> wrote:
>>>>>>
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
>>>> grammar
>>>>>>> in our module flink-sql-parser. But we don't support view
>> objects in
>>>>>>> neither blink planner nor old planner.
>>>>>>>
>>>>>>> I'd like to kick off a discussion on end to end view support in
>> Flink
>>>>>>> SQL in blink planner. It's helpful to improve the usability of
>> the
>>>>>>> framework for SQL users.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>>>>>>>
>>>>>>> In short, it:
>>>>>>> - support define views and store them in catalog
>>>>>>> - support drop view definitions from catalog
>>>>>>> - support query views
>>>>>>> - support other view related DDLs
>>>>>>>
>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>>
>>>>>>> *Best Regards,*
>>>>>>> *Zhenghua Gao*
>>>>>>>
>>>>>>
>>>>
>>
>

[Zhenghua Gao]

Hi Timo,

Shall we postpone the support of 'TEMPORARY' keyword since it's not
mentioned in FLINK-10232?
<https://issues.apache.org/jira/browse/FLINK-10232>

*Best Regards,*
*Zhenghua Gao*


On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]> wrote:

> Hi Zhenghua,
>
> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
>
> Otherwise the FLIP looks good to me.
>
> Regards,
> Timo
>
>
> On 08.04.20 07:31, Zhenghua Gao wrote:
> > @Danny Chan <[hidden email]>  you‘re right. I have updated the
> doc.
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]> wrote:
> >
> >> +1 for the proposal, a small concern for drop view statement:
> >>
> >> dropViewStatement:
> >>        DROP VIEW name [ IF EXISTS ]
> >> I think the drop statement should be
> >> DROP VIEW [ IF EXISTS ] name
> >>
> >> Best,
> >> Danny Chan
> >> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
> >>> This FLIP seems to be quite straightforward, +1 from my side.
> >>>
> >>> Best,
> >>> Kurt
> >>>
> >>>
> >>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
> >>>
> >>>> forward the reply to ML too.
> >>>>
> >>>>
> >>>> *Best Regards,*
> >>>> *Zhenghua Gao*
> >>>>
> >>>>
> >>>> ---------- Forwarded message ---------
> >>>> From: Zhenghua Gao <[hidden email]>
> >>>> Date: Tue, Apr 7, 2020 at 8:40 PM
> >>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> >>>> To: godfrey he <[hidden email]>
> >>>>
> >>>>
> >>>>>> regarding to "Interoperability between Flink and Hive is not
> >>>> guaranteed", can you explain this more?
> >>>> We have several limitations of interoperability between flink objects
> >> and
> >>>> hive objects (tables, functions, etc).
> >>>> So we don't promise the interoperability of views between flink and
> >> hive
> >>>> since a view is defined base on these objects.
> >>>>
> >>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
> >>>> This should be "CREATE VIEW [OR REPLACE]".
> >>>>
> >>>>>> "DESC"
> >>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> >>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
> >>>>
> >>>> I have updated the design doc, thanks.
> >>>>
> >>>> *Best Regards,*
> >>>> *Zhenghua Gao*
> >>>>
> >>>>
> >>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
> wrote:
> >>>>
> >>>>> Hi Zhenghua,
> >>>>>
> >>>>> Thanks for driving this. It's one step forward that TableEnvironment
> >>>>> supports more complete SQLs.
> >>>>> I have a few minor questions:
> >>>>> 1. regarding to "Interoperability between Flink and Hive is not
> >>>>> guaranteed", can you explain this more?
> >>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW
> >> [ IF
> >>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS"
> >> and
> >>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
> >>>> describe
> >>>>> an object through "DESC" . I think It's better this flip can support
> >>>> "SHOW
> >>>>> VIEWS" and "DESCRIBE VIEW xx".
> >>>>>
> >>>>> Best,
> >>>>> Godfrey
> >>>>>
> >>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> >>>>>
> >>>>>> Hi community,
> >>>>>>
> >>>>>> It's a long time since we started the discussion of supporting
> >> view in
> >>>>>> FLINK SQL.
> >>>>>> Flink also continues to move forward since then.
> >>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
> >> CatalogView
> >>>>>> in blink planner.
> >>>>>> The missing link is validate the view definition and store the
> >>>>>> original/expanded text in the catalog.
> >>>>>> I have updated the design doc of FLIP-71,
> >>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> >>>>>>
> >>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>>>>
> >>>>>> *Best Regards,*
> >>>>>> *Zhenghua Gao*
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
> >> wrote:
> >>>>>>
> >>>>>>> Hi folks,
> >>>>>>>
> >>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
> >>>> grammar
> >>>>>>> in our module flink-sql-parser. But we don't support view
> >> objects in
> >>>>>>> neither blink planner nor old planner.
> >>>>>>>
> >>>>>>> I'd like to kick off a discussion on end to end view support in
> >> Flink
> >>>>>>> SQL in blink planner. It's helpful to improve the usability of
> >> the
> >>>>>>> framework for SQL users.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>
> >>
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> >>>>>>>
> >>>>>>> In short, it:
> >>>>>>> - support define views and store them in catalog
> >>>>>>> - support drop view definitions from catalog
> >>>>>>> - support query views
> >>>>>>> - support other view related DDLs
> >>>>>>>
> >>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>>>>>
> >>>>>>> *Best Regards,*
> >>>>>>> *Zhenghua Gao*
> >>>>>>>
> >>>>>>
> >>>>
> >>
> >
>
>

[Zhenghua Gao]

Another concern about "CREATE DDL" is:

FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior when a
view or table with the same name already exists.
And "OR REPLACE" for type/library/function DDL.

@godfrey he <[hidden email]> I will keep the "IF NOT EXISTS" syntax
and postpone the "OR REPLACE" syntax until we need it.


*Best Regards,*
*Zhenghua Gao*


On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]> wrote:

> Hi Timo,
>
> Shall we postpone the support of 'TEMPORARY' keyword since it's not
> mentioned in FLINK-10232?
> <https://issues.apache.org/jira/browse/FLINK-10232>
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]> wrote:
>
>> Hi Zhenghua,
>>
>> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
>>
>> Otherwise the FLIP looks good to me.
>>
>> Regards,
>> Timo
>>
>>
>> On 08.04.20 07:31, Zhenghua Gao wrote:
>> > @Danny Chan <[hidden email]>  you‘re right. I have updated the
>> doc.
>> >
>> > *Best Regards,*
>> > *Zhenghua Gao*
>> >
>> >
>> > On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]> wrote:
>> >
>> >> +1 for the proposal, a small concern for drop view statement:
>> >>
>> >> dropViewStatement:
>> >>        DROP VIEW name [ IF EXISTS ]
>> >> I think the drop statement should be
>> >> DROP VIEW [ IF EXISTS ] name
>> >>
>> >> Best,
>> >> Danny Chan
>> >> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
>> >>> This FLIP seems to be quite straightforward, +1 from my side.
>> >>>
>> >>> Best,
>> >>> Kurt
>> >>>
>> >>>
>> >>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
>> >>>
>> >>>> forward the reply to ML too.
>> >>>>
>> >>>>
>> >>>> *Best Regards,*
>> >>>> *Zhenghua Gao*
>> >>>>
>> >>>>
>> >>>> ---------- Forwarded message ---------
>> >>>> From: Zhenghua Gao <[hidden email]>
>> >>>> Date: Tue, Apr 7, 2020 at 8:40 PM
>> >>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
>> >>>> To: godfrey he <[hidden email]>
>> >>>>
>> >>>>
>> >>>>>> regarding to "Interoperability between Flink and Hive is not
>> >>>> guaranteed", can you explain this more?
>> >>>> We have several limitations of interoperability between flink objects
>> >> and
>> >>>> hive objects (tables, functions, etc).
>> >>>> So we don't promise the interoperability of views between flink and
>> >> hive
>> >>>> since a view is defined base on these objects.
>> >>>>
>> >>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
>> >>>> This should be "CREATE VIEW [OR REPLACE]".
>> >>>>
>> >>>>>> "DESC"
>> >>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
>> >>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
>> >>>>
>> >>>> I have updated the design doc, thanks.
>> >>>>
>> >>>> *Best Regards,*
>> >>>> *Zhenghua Gao*
>> >>>>
>> >>>>
>> >>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
>> wrote:
>> >>>>
>> >>>>> Hi Zhenghua,
>> >>>>>
>> >>>>> Thanks for driving this. It's one step forward that TableEnvironment
>> >>>>> supports more complete SQLs.
>> >>>>> I have a few minor questions:
>> >>>>> 1. regarding to "Interoperability between Flink and Hive is not
>> >>>>> guaranteed", can you explain this more?
>> >>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW
>> >> [ IF
>> >>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS"
>> >> and
>> >>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
>> >>>> describe
>> >>>>> an object through "DESC" . I think It's better this flip can support
>> >>>> "SHOW
>> >>>>> VIEWS" and "DESCRIBE VIEW xx".
>> >>>>>
>> >>>>> Best,
>> >>>>> Godfrey
>> >>>>>
>> >>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
>> >>>>>
>> >>>>>> Hi community,
>> >>>>>>
>> >>>>>> It's a long time since we started the discussion of supporting
>> >> view in
>> >>>>>> FLINK SQL.
>> >>>>>> Flink also continues to move forward since then.
>> >>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
>> >> CatalogView
>> >>>>>> in blink planner.
>> >>>>>> The missing link is validate the view definition and store the
>> >>>>>> original/expanded text in the catalog.
>> >>>>>> I have updated the design doc of FLIP-71,
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
>> >>>>>>
>> >>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>> >>>>>>
>> >>>>>> *Best Regards,*
>> >>>>>> *Zhenghua Gao*
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
>> >> wrote:
>> >>>>>>
>> >>>>>>> Hi folks,
>> >>>>>>>
>> >>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
>> >>>> grammar
>> >>>>>>> in our module flink-sql-parser. But we don't support view
>> >> objects in
>> >>>>>>> neither blink planner nor old planner.
>> >>>>>>>
>> >>>>>>> I'd like to kick off a discussion on end to end view support in
>> >> Flink
>> >>>>>>> SQL in blink planner. It's helpful to improve the usability of
>> >> the
>> >>>>>>> framework for SQL users.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>
>> >>
>> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>> >>>>>>>
>> >>>>>>> In short, it:
>> >>>>>>> - support define views and store them in catalog
>> >>>>>>> - support drop view definitions from catalog
>> >>>>>>> - support query views
>> >>>>>>> - support other view related DDLs
>> >>>>>>>
>> >>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>> >>>>>>>
>> >>>>>>> *Best Regards,*
>> >>>>>>> *Zhenghua Gao*
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> >
>>
>>

[Timo Walther-2]

Hi Zhenghua,

FLINK-10232 is quite old and a lot of stuff was discussed and agreed on
since then. I don't like to postpone the 'TEMPORARY' keyword because it
is a important concept that is already part of the Table API (see
TableEnvironment.createTemporaryView) and in function DDL and table DDL.
It is not complicated to supported it in this FLIP and just a couple of
line of code more.

Regards,
Timo

On 08.04.20 11:27, Zhenghua Gao wrote:

> Another concern about "CREATE DDL" is:
>
> FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior when a
> view or table with the same name already exists.
> And "OR REPLACE" for type/library/function DDL.
>
> @godfrey he <[hidden email]> I will keep the "IF NOT EXISTS" syntax
> and postpone the "OR REPLACE" syntax until we need it.
>
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]> wrote:
>
>> Hi Timo,
>>
>> Shall we postpone the support of 'TEMPORARY' keyword since it's not
>> mentioned in FLINK-10232?
>> <https://issues.apache.org/jira/browse/FLINK-10232>
>>
>> *Best Regards,*
>> *Zhenghua Gao*
>>
>>
>> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]> wrote:
>>
>>> Hi Zhenghua,
>>>
>>> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
>>>
>>> Otherwise the FLIP looks good to me.
>>>
>>> Regards,
>>> Timo
>>>
>>>
>>> On 08.04.20 07:31, Zhenghua Gao wrote:
>>>> @Danny Chan <[hidden email]>  you‘re right. I have updated the
>>> doc.
>>>>
>>>> *Best Regards,*
>>>> *Zhenghua Gao*
>>>>
>>>>
>>>> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]> wrote:
>>>>
>>>>> +1 for the proposal, a small concern for drop view statement:
>>>>>
>>>>> dropViewStatement:
>>>>>         DROP VIEW name [ IF EXISTS ]
>>>>> I think the drop statement should be
>>>>> DROP VIEW [ IF EXISTS ] name
>>>>>
>>>>> Best,
>>>>> Danny Chan
>>>>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
>>>>>> This FLIP seems to be quite straightforward, +1 from my side.
>>>>>>
>>>>>> Best,
>>>>>> Kurt
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]> wrote:
>>>>>>
>>>>>>> forward the reply to ML too.
>>>>>>>
>>>>>>>
>>>>>>> *Best Regards,*
>>>>>>> *Zhenghua Gao*
>>>>>>>
>>>>>>>
>>>>>>> ---------- Forwarded message ---------
>>>>>>> From: Zhenghua Gao <[hidden email]>
>>>>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
>>>>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
>>>>>>> To: godfrey he <[hidden email]>
>>>>>>>
>>>>>>>
>>>>>>>>> regarding to "Interoperability between Flink and Hive is not
>>>>>>> guaranteed", can you explain this more?
>>>>>>> We have several limitations of interoperability between flink objects
>>>>> and
>>>>>>> hive objects (tables, functions, etc).
>>>>>>> So we don't promise the interoperability of views between flink and
>>>>> hive
>>>>>>> since a view is defined base on these objects.
>>>>>>>
>>>>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
>>>>>>> This should be "CREATE VIEW [OR REPLACE]".
>>>>>>>
>>>>>>>>> "DESC"
>>>>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
>>>>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW xxx".
>>>>>>>
>>>>>>> I have updated the design doc, thanks.
>>>>>>>
>>>>>>> *Best Regards,*
>>>>>>> *Zhenghua Gao*
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Zhenghua,
>>>>>>>>
>>>>>>>> Thanks for driving this. It's one step forward that TableEnvironment
>>>>>>>> supports more complete SQLs.
>>>>>>>> I have a few minor questions:
>>>>>>>> 1. regarding to "Interoperability between Flink and Hive is not
>>>>>>>> guaranteed", can you explain this more?
>>>>>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE VIEW
>>>>> [ IF
>>>>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW VIEWS"
>>>>> and
>>>>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
>>>>>>> describe
>>>>>>>> an object through "DESC" . I think It's better this flip can support
>>>>>>> "SHOW
>>>>>>>> VIEWS" and "DESCRIBE VIEW xx".
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Godfrey
>>>>>>>>
>>>>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
>>>>>>>>
>>>>>>>>> Hi community,
>>>>>>>>>
>>>>>>>>> It's a long time since we started the discussion of supporting
>>>>> view in
>>>>>>>>> FLINK SQL.
>>>>>>>>> Flink also continues to move forward since then.
>>>>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
>>>>> CatalogView
>>>>>>>>> in blink planner.
>>>>>>>>> The missing link is validate the view definition and store the
>>>>>>>>> original/expanded text in the catalog.
>>>>>>>>> I have updated the design doc of FLIP-71,
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
>>>>>>>>>
>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>>>>
>>>>>>>>> *Best Regards,*
>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi folks,
>>>>>>>>>>
>>>>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
>>>>>>> grammar
>>>>>>>>>> in our module flink-sql-parser. But we don't support view
>>>>> objects in
>>>>>>>>>> neither blink planner nor old planner.
>>>>>>>>>>
>>>>>>>>>> I'd like to kick off a discussion on end to end view support in
>>>>> Flink
>>>>>>>>>> SQL in blink planner. It's helpful to improve the usability of
>>>>> the
>>>>>>>>>> framework for SQL users.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>
>>> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>>>>>>>>>>
>>>>>>>>>> In short, it:
>>>>>>>>>> - support define views and store them in catalog
>>>>>>>>>> - support drop view definitions from catalog
>>>>>>>>>> - support query views
>>>>>>>>>> - support other view related DDLs
>>>>>>>>>>
>>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>>>>>
>>>>>>>>>> *Best Regards,*
>>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
>

[Zhenghua Gao]

Hi Timo,

Actually "TEMPORARY" is not supported in table DDL now.
But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP.
And may be we should open a separate JIRA ticket to track supporting it in
table DDL？

*Best Regards,*
*Zhenghua Gao*


On Wed, Apr 8, 2020 at 7:48 PM Timo Walther <[hidden email]> wrote:

> Hi Zhenghua,
>
> FLINK-10232 is quite old and a lot of stuff was discussed and agreed on
> since then. I don't like to postpone the 'TEMPORARY' keyword because it
> is a important concept that is already part of the Table API (see
> TableEnvironment.createTemporaryView) and in function DDL and table DDL.
> It is not complicated to supported it in this FLIP and just a couple of
> line of code more.
>
> Regards,
> Timo
>
> On 08.04.20 11:27, Zhenghua Gao wrote:
> > Another concern about "CREATE DDL" is:
> >
> > FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior when a
> > view or table with the same name already exists.
> > And "OR REPLACE" for type/library/function DDL.
> >
> > @godfrey he <[hidden email]> I will keep the "IF NOT EXISTS" syntax
> > and postpone the "OR REPLACE" syntax until we need it.
> >
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]> wrote:
> >
> >> Hi Timo,
> >>
> >> Shall we postpone the support of 'TEMPORARY' keyword since it's not
> >> mentioned in FLINK-10232?
> >> <https://issues.apache.org/jira/browse/FLINK-10232>
> >>
> >> *Best Regards,*
> >> *Zhenghua Gao*
> >>
> >>
> >> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]> wrote:
> >>
> >>> Hi Zhenghua,
> >>>
> >>> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
> >>>
> >>> Otherwise the FLIP looks good to me.
> >>>
> >>> Regards,
> >>> Timo
> >>>
> >>>
> >>> On 08.04.20 07:31, Zhenghua Gao wrote:
> >>>> @Danny Chan <[hidden email]>  you‘re right. I have updated the
> >>> doc.
> >>>>
> >>>> *Best Regards,*
> >>>> *Zhenghua Gao*
> >>>>
> >>>>
> >>>> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]>
> wrote:
> >>>>
> >>>>> +1 for the proposal, a small concern for drop view statement:
> >>>>>
> >>>>> dropViewStatement:
> >>>>>         DROP VIEW name [ IF EXISTS ]
> >>>>> I think the drop statement should be
> >>>>> DROP VIEW [ IF EXISTS ] name
> >>>>>
> >>>>> Best,
> >>>>> Danny Chan
> >>>>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
> >>>>>> This FLIP seems to be quite straightforward, +1 from my side.
> >>>>>>
> >>>>>> Best,
> >>>>>> Kurt
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]>
> wrote:
> >>>>>>
> >>>>>>> forward the reply to ML too.
> >>>>>>>
> >>>>>>>
> >>>>>>> *Best Regards,*
> >>>>>>> *Zhenghua Gao*
> >>>>>>>
> >>>>>>>
> >>>>>>> ---------- Forwarded message ---------
> >>>>>>> From: Zhenghua Gao <[hidden email]>
> >>>>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
> >>>>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> >>>>>>> To: godfrey he <[hidden email]>
> >>>>>>>
> >>>>>>>
> >>>>>>>>> regarding to "Interoperability between Flink and Hive is not
> >>>>>>> guaranteed", can you explain this more?
> >>>>>>> We have several limitations of interoperability between flink
> objects
> >>>>> and
> >>>>>>> hive objects (tables, functions, etc).
> >>>>>>> So we don't promise the interoperability of views between flink and
> >>>>> hive
> >>>>>>> since a view is defined base on these objects.
> >>>>>>>
> >>>>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
> >>>>>>> This should be "CREATE VIEW [OR REPLACE]".
> >>>>>>>
> >>>>>>>>> "DESC"
> >>>>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> >>>>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW
> xxx".
> >>>>>>>
> >>>>>>> I have updated the design doc, thanks.
> >>>>>>>
> >>>>>>> *Best Regards,*
> >>>>>>> *Zhenghua Gao*
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Hi Zhenghua,
> >>>>>>>>
> >>>>>>>> Thanks for driving this. It's one step forward that
> TableEnvironment
> >>>>>>>> supports more complete SQLs.
> >>>>>>>> I have a few minor questions:
> >>>>>>>> 1. regarding to "Interoperability between Flink and Hive is not
> >>>>>>>> guaranteed", can you explain this more?
> >>>>>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE
> VIEW
> >>>>> [ IF
> >>>>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW
> VIEWS"
> >>>>> and
> >>>>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
> >>>>>>> describe
> >>>>>>>> an object through "DESC" . I think It's better this flip can
> support
> >>>>>>> "SHOW
> >>>>>>>> VIEWS" and "DESCRIBE VIEW xx".
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Godfrey
> >>>>>>>>
> >>>>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> >>>>>>>>
> >>>>>>>>> Hi community,
> >>>>>>>>>
> >>>>>>>>> It's a long time since we started the discussion of supporting
> >>>>> view in
> >>>>>>>>> FLINK SQL.
> >>>>>>>>> Flink also continues to move forward since then.
> >>>>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
> >>>>> CatalogView
> >>>>>>>>> in blink planner.
> >>>>>>>>> The missing link is validate the view definition and store the
> >>>>>>>>> original/expanded text in the catalog.
> >>>>>>>>> I have updated the design doc of FLIP-71,
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> >>>>>>>>>
> >>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>>>>>>>
> >>>>>>>>> *Best Regards,*
> >>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi folks,
> >>>>>>>>>>
> >>>>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
> >>>>>>> grammar
> >>>>>>>>>> in our module flink-sql-parser. But we don't support view
> >>>>> objects in
> >>>>>>>>>> neither blink planner nor old planner.
> >>>>>>>>>>
> >>>>>>>>>> I'd like to kick off a discussion on end to end view support in
> >>>>> Flink
> >>>>>>>>>> SQL in blink planner. It's helpful to improve the usability of
> >>>>> the
> >>>>>>>>>> framework for SQL users.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> >>>>>>>>>>
> >>>>>>>>>> In short, it:
> >>>>>>>>>> - support define views and store them in catalog
> >>>>>>>>>> - support drop view definitions from catalog
> >>>>>>>>>> - support query views
> >>>>>>>>>> - support other view related DDLs
> >>>>>>>>>>
> >>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>>>>>>>>
> >>>>>>>>>> *Best Regards,*
> >>>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >
>
>

[Timo Walther-2]

I didn't know that. We should definitely implement this asap. Please
open a JIRA issue.

Thanks,
Timo


On 08.04.20 14:29, Zhenghua Gao wrote:

> Hi Timo,
>
> Actually "TEMPORARY" is not supported in table DDL now.
> But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP.
> And may be we should open a separate JIRA ticket to track supporting it in
> table DDL？
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Wed, Apr 8, 2020 at 7:48 PM Timo Walther <[hidden email]> wrote:
>
>> Hi Zhenghua,
>>
>> FLINK-10232 is quite old and a lot of stuff was discussed and agreed on
>> since then. I don't like to postpone the 'TEMPORARY' keyword because it
>> is a important concept that is already part of the Table API (see
>> TableEnvironment.createTemporaryView) and in function DDL and table DDL.
>> It is not complicated to supported it in this FLIP and just a couple of
>> line of code more.
>>
>> Regards,
>> Timo
>>
>> On 08.04.20 11:27, Zhenghua Gao wrote:
>>> Another concern about "CREATE DDL" is:
>>>
>>> FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior when a
>>> view or table with the same name already exists.
>>> And "OR REPLACE" for type/library/function DDL.
>>>
>>> @godfrey he <[hidden email]> I will keep the "IF NOT EXISTS" syntax
>>> and postpone the "OR REPLACE" syntax until we need it.
>>>
>>>
>>> *Best Regards,*
>>> *Zhenghua Gao*
>>>
>>>
>>> On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]> wrote:
>>>
>>>> Hi Timo,
>>>>
>>>> Shall we postpone the support of 'TEMPORARY' keyword since it's not
>>>> mentioned in FLINK-10232?
>>>> <https://issues.apache.org/jira/browse/FLINK-10232>
>>>>
>>>> *Best Regards,*
>>>> *Zhenghua Gao*
>>>>
>>>>
>>>> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]> wrote:
>>>>
>>>>> Hi Zhenghua,
>>>>>
>>>>> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
>>>>>
>>>>> Otherwise the FLIP looks good to me.
>>>>>
>>>>> Regards,
>>>>> Timo
>>>>>
>>>>>
>>>>> On 08.04.20 07:31, Zhenghua Gao wrote:
>>>>>> @Danny Chan <[hidden email]>  you‘re right. I have updated the
>>>>> doc.
>>>>>>
>>>>>> *Best Regards,*
>>>>>> *Zhenghua Gao*
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]>
>> wrote:
>>>>>>
>>>>>>> +1 for the proposal, a small concern for drop view statement:
>>>>>>>
>>>>>>> dropViewStatement:
>>>>>>>          DROP VIEW name [ IF EXISTS ]
>>>>>>> I think the drop statement should be
>>>>>>> DROP VIEW [ IF EXISTS ] name
>>>>>>>
>>>>>>> Best,
>>>>>>> Danny Chan
>>>>>>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
>>>>>>>> This FLIP seems to be quite straightforward, +1 from my side.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Kurt
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]>
>> wrote:
>>>>>>>>
>>>>>>>>> forward the reply to ML too.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Best Regards,*
>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>> From: Zhenghua Gao <[hidden email]>
>>>>>>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
>>>>>>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
>>>>>>>>> To: godfrey he <[hidden email]>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> regarding to "Interoperability between Flink and Hive is not
>>>>>>>>> guaranteed", can you explain this more?
>>>>>>>>> We have several limitations of interoperability between flink
>> objects
>>>>>>> and
>>>>>>>>> hive objects (tables, functions, etc).
>>>>>>>>> So we don't promise the interoperability of views between flink and
>>>>>>> hive
>>>>>>>>> since a view is defined base on these objects.
>>>>>>>>>
>>>>>>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
>>>>>>>>> This should be "CREATE VIEW [OR REPLACE]".
>>>>>>>>>
>>>>>>>>>>> "DESC"
>>>>>>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
>>>>>>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW
>> xxx".
>>>>>>>>>
>>>>>>>>> I have updated the design doc, thanks.
>>>>>>>>>
>>>>>>>>> *Best Regards,*
>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Zhenghua,
>>>>>>>>>>
>>>>>>>>>> Thanks for driving this. It's one step forward that
>> TableEnvironment
>>>>>>>>>> supports more complete SQLs.
>>>>>>>>>> I have a few minor questions:
>>>>>>>>>> 1. regarding to "Interoperability between Flink and Hive is not
>>>>>>>>>> guaranteed", can you explain this more?
>>>>>>>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE
>> VIEW
>>>>>>> [ IF
>>>>>>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW
>> VIEWS"
>>>>>>> and
>>>>>>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not support
>>>>>>>>> describe
>>>>>>>>>> an object through "DESC" . I think It's better this flip can
>> support
>>>>>>>>> "SHOW
>>>>>>>>>> VIEWS" and "DESCRIBE VIEW xx".
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Godfrey
>>>>>>>>>>
>>>>>>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
>>>>>>>>>>
>>>>>>>>>>> Hi community,
>>>>>>>>>>>
>>>>>>>>>>> It's a long time since we started the discussion of supporting
>>>>>>> view in
>>>>>>>>>>> FLINK SQL.
>>>>>>>>>>> Flink also continues to move forward since then.
>>>>>>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
>>>>>>> CatalogView
>>>>>>>>>>> in blink planner.
>>>>>>>>>>> The missing link is validate the view definition and store the
>>>>>>>>>>> original/expanded text in the catalog.
>>>>>>>>>>> I have updated the design doc of FLIP-71,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
>>>>>>>>>>>
>>>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>>>>>>
>>>>>>>>>>> *Best Regards,*
>>>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <[hidden email]>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi folks,
>>>>>>>>>>>>
>>>>>>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP VIEW
>>>>>>>>> grammar
>>>>>>>>>>>> in our module flink-sql-parser. But we don't support view
>>>>>>> objects in
>>>>>>>>>>>> neither blink planner nor old planner.
>>>>>>>>>>>>
>>>>>>>>>>>> I'd like to kick off a discussion on end to end view support in
>>>>>>> Flink
>>>>>>>>>>>> SQL in blink planner. It's helpful to improve the usability of
>>>>>>> the
>>>>>>>>>>>> framework for SQL users.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
>>>>>>>>>>>>
>>>>>>>>>>>> In short, it:
>>>>>>>>>>>> - support define views and store them in catalog
>>>>>>>>>>>> - support drop view definitions from catalog
>>>>>>>>>>>> - support query views
>>>>>>>>>>>> - support other view related DDLs
>>>>>>>>>>>>
>>>>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>> *Best Regards,*
>>>>>>>>>>>> *Zhenghua Gao*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>>
>

[Jark Wu-2]

Sorry for the late reply,

I have some concern around "Supporting SHOW VIEWS|DESCRIBE VIEW name".
Currently, in SQL CLI, the "SHOW TABLES" will also list views and "DESCRIBE
name" can also describe a view.
Shall we remove the view support in those commands if we want to support a
dedicate "SHOW VIEWS|DESCRIBE VIEW name"?

Brest,
Jark

On Wed, 8 Apr 2020 at 23:49, Timo Walther <[hidden email]> wrote:

> I didn't know that. We should definitely implement this asap. Please
> open a JIRA issue.
>
> Thanks,
> Timo
>
>
> On 08.04.20 14:29, Zhenghua Gao wrote:
> > Hi Timo,
> >
> > Actually "TEMPORARY" is not supported in table DDL now.
> > But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP.
> > And may be we should open a separate JIRA ticket to track supporting it
> in
> > table DDL？
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > On Wed, Apr 8, 2020 at 7:48 PM Timo Walther <[hidden email]> wrote:
> >
> >> Hi Zhenghua,
> >>
> >> FLINK-10232 is quite old and a lot of stuff was discussed and agreed on
> >> since then. I don't like to postpone the 'TEMPORARY' keyword because it
> >> is a important concept that is already part of the Table API (see
> >> TableEnvironment.createTemporaryView) and in function DDL and table DDL.
> >> It is not complicated to supported it in this FLIP and just a couple of
> >> line of code more.
> >>
> >> Regards,
> >> Timo
> >>
> >> On 08.04.20 11:27, Zhenghua Gao wrote:
> >>> Another concern about "CREATE DDL" is:
> >>>
> >>> FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior
> when a
> >>> view or table with the same name already exists.
> >>> And "OR REPLACE" for type/library/function DDL.
> >>>
> >>> @godfrey he <[hidden email]> I will keep the "IF NOT EXISTS"
> syntax
> >>> and postpone the "OR REPLACE" syntax until we need it.
> >>>
> >>>
> >>> *Best Regards,*
> >>> *Zhenghua Gao*
> >>>
> >>>
> >>> On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]> wrote:
> >>>
> >>>> Hi Timo,
> >>>>
> >>>> Shall we postpone the support of 'TEMPORARY' keyword since it's not
> >>>> mentioned in FLINK-10232?
> >>>> <https://issues.apache.org/jira/browse/FLINK-10232>
> >>>>
> >>>> *Best Regards,*
> >>>> *Zhenghua Gao*
> >>>>
> >>>>
> >>>> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]>
> wrote:
> >>>>
> >>>>> Hi Zhenghua,
> >>>>>
> >>>>> VIEWS should also support the TEMPORARY keyword according to FLIP-64.
> >>>>>
> >>>>> Otherwise the FLIP looks good to me.
> >>>>>
> >>>>> Regards,
> >>>>> Timo
> >>>>>
> >>>>>
> >>>>> On 08.04.20 07:31, Zhenghua Gao wrote:
> >>>>>> @Danny Chan <[hidden email]>  you‘re right. I have updated
> the
> >>>>> doc.
> >>>>>>
> >>>>>> *Best Regards,*
> >>>>>> *Zhenghua Gao*
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]>
> >> wrote:
> >>>>>>
> >>>>>>> +1 for the proposal, a small concern for drop view statement:
> >>>>>>>
> >>>>>>> dropViewStatement:
> >>>>>>>          DROP VIEW name [ IF EXISTS ]
> >>>>>>> I think the drop statement should be
> >>>>>>> DROP VIEW [ IF EXISTS ] name
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Danny Chan
> >>>>>>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
> >>>>>>>> This FLIP seems to be quite straightforward, +1 from my side.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Kurt
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]>
> >> wrote:
> >>>>>>>>
> >>>>>>>>> forward the reply to ML too.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> *Best Regards,*
> >>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ---------- Forwarded message ---------
> >>>>>>>>> From: Zhenghua Gao <[hidden email]>
> >>>>>>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
> >>>>>>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> >>>>>>>>> To: godfrey he <[hidden email]>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>> regarding to "Interoperability between Flink and Hive is not
> >>>>>>>>> guaranteed", can you explain this more?
> >>>>>>>>> We have several limitations of interoperability between flink
> >> objects
> >>>>>>> and
> >>>>>>>>> hive objects (tables, functions, etc).
> >>>>>>>>> So we don't promise the interoperability of views between flink
> and
> >>>>>>> hive
> >>>>>>>>> since a view is defined base on these objects.
> >>>>>>>>>
> >>>>>>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
> >>>>>>>>> This should be "CREATE VIEW [OR REPLACE]".
> >>>>>>>>>
> >>>>>>>>>>> "DESC"
> >>>>>>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table xxx).
> >>>>>>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW
> >> xxx".
> >>>>>>>>>
> >>>>>>>>> I have updated the design doc, thanks.
> >>>>>>>>>
> >>>>>>>>> *Best Regards,*
> >>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Zhenghua,
> >>>>>>>>>>
> >>>>>>>>>> Thanks for driving this. It's one step forward that
> >> TableEnvironment
> >>>>>>>>>> supports more complete SQLs.
> >>>>>>>>>> I have a few minor questions:
> >>>>>>>>>> 1. regarding to "Interoperability between Flink and Hive is not
> >>>>>>>>>> guaranteed", can you explain this more?
> >>>>>>>>>> 2. regarding to "The Grammar", Calcite does not support "CREATE
> >> VIEW
> >>>>>>> [ IF
> >>>>>>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW
> >> VIEWS"
> >>>>>>> and
> >>>>>>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not
> support
> >>>>>>>>> describe
> >>>>>>>>>> an object through "DESC" . I think It's better this flip can
> >> support
> >>>>>>>>> "SHOW
> >>>>>>>>>> VIEWS" and "DESCRIBE VIEW xx".
> >>>>>>>>>>
> >>>>>>>>>> Best,
> >>>>>>>>>> Godfrey
> >>>>>>>>>>
> >>>>>>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> >>>>>>>>>>
> >>>>>>>>>>> Hi community,
> >>>>>>>>>>>
> >>>>>>>>>>> It's a long time since we started the discussion of supporting
> >>>>>>> view in
> >>>>>>>>>>> FLINK SQL.
> >>>>>>>>>>> Flink also continues to move forward since then.
> >>>>>>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
> >>>>>>> CatalogView
> >>>>>>>>>>> in blink planner.
> >>>>>>>>>>> The missing link is validate the view definition and store the
> >>>>>>>>>>> original/expanded text in the catalog.
> >>>>>>>>>>> I have updated the design doc of FLIP-71,
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> >>>>>>>>>>>
> >>>>>>>>>>> Any comments and feedbacks are welcome and appreciated. Thanks.
> >>>>>>>>>>>
> >>>>>>>>>>> *Best Regards,*
> >>>>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <
> [hidden email]>
> >>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi folks,
> >>>>>>>>>>>>
> >>>>>>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP
> VIEW
> >>>>>>>>> grammar
> >>>>>>>>>>>> in our module flink-sql-parser. But we don't support view
> >>>>>>> objects in
> >>>>>>>>>>>> neither blink planner nor old planner.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'd like to kick off a discussion on end to end view support
> in
> >>>>>>> Flink
> >>>>>>>>>>>> SQL in blink planner. It's helpful to improve the usability of
> >>>>>>> the
> >>>>>>>>>>>> framework for SQL users.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> >>>>>>>>>>>>
> >>>>>>>>>>>> In short, it:
> >>>>>>>>>>>> - support define views and store them in catalog
> >>>>>>>>>>>> - support drop view definitions from catalog
> >>>>>>>>>>>> - support query views
> >>>>>>>>>>>> - support other view related DDLs
> >>>>>>>>>>>>
> >>>>>>>>>>>> Any comments and feedbacks are welcome and appreciated.
> Thanks.
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Best Regards,*
> >>>>>>>>>>>> *Zhenghua Gao*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>
> >>
> >
>
>

[Zhenghua Gao]

Hi Jark,

>> Shall we remove the view support in those commands if we want to support
a
>> dedicate "SHOW VIEWS|DESCRIBE VIEW name"?
Yes, we should correct those commands in SQL client. Will open tickets
after the vote.


*Best Regards,*
*Zhenghua Gao*


On Sat, Apr 11, 2020 at 11:25 AM Jark Wu <[hidden email]> wrote:

> Sorry for the late reply,
>
> I have some concern around "Supporting SHOW VIEWS|DESCRIBE VIEW name".
> Currently, in SQL CLI, the "SHOW TABLES" will also list views and "DESCRIBE
> name" can also describe a view.
> Shall we remove the view support in those commands if we want to support a
> dedicate "SHOW VIEWS|DESCRIBE VIEW name"?
>
> Brest,
> Jark
>
> On Wed, 8 Apr 2020 at 23:49, Timo Walther <[hidden email]> wrote:
>
> > I didn't know that. We should definitely implement this asap. Please
> > open a JIRA issue.
> >
> > Thanks,
> > Timo
> >
> >
> > On 08.04.20 14:29, Zhenghua Gao wrote:
> > > Hi Timo,
> > >
> > > Actually "TEMPORARY" is not supported in table DDL now.
> > > But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP.
> > > And may be we should open a separate JIRA ticket to track supporting it
> > in
> > > table DDL？
> > >
> > > *Best Regards,*
> > > *Zhenghua Gao*
> > >
> > >
> > > On Wed, Apr 8, 2020 at 7:48 PM Timo Walther <[hidden email]>
> wrote:
> > >
> > >> Hi Zhenghua,
> > >>
> > >> FLINK-10232 is quite old and a lot of stuff was discussed and agreed
> on
> > >> since then. I don't like to postpone the 'TEMPORARY' keyword because
> it
> > >> is a important concept that is already part of the Table API (see
> > >> TableEnvironment.createTemporaryView) and in function DDL and table
> DDL.
> > >> It is not complicated to supported it in this FLIP and just a couple
> of
> > >> line of code more.
> > >>
> > >> Regards,
> > >> Timo
> > >>
> > >> On 08.04.20 11:27, Zhenghua Gao wrote:
> > >>> Another concern about "CREATE DDL" is:
> > >>>
> > >>> FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior
> > when a
> > >>> view or table with the same name already exists.
> > >>> And "OR REPLACE" for type/library/function DDL.
> > >>>
> > >>> @godfrey he <[hidden email]> I will keep the "IF NOT EXISTS"
> > syntax
> > >>> and postpone the "OR REPLACE" syntax until we need it.
> > >>>
> > >>>
> > >>> *Best Regards,*
> > >>> *Zhenghua Gao*
> > >>>
> > >>>
> > >>> On Wed, Apr 8, 2020 at 4:46 PM Zhenghua Gao <[hidden email]>
> wrote:
> > >>>
> > >>>> Hi Timo,
> > >>>>
> > >>>> Shall we postpone the support of 'TEMPORARY' keyword since it's not
> > >>>> mentioned in FLINK-10232?
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10232>
> > >>>>
> > >>>> *Best Regards,*
> > >>>> *Zhenghua Gao*
> > >>>>
> > >>>>
> > >>>> On Wed, Apr 8, 2020 at 3:30 PM Timo Walther <[hidden email]>
> > wrote:
> > >>>>
> > >>>>> Hi Zhenghua,
> > >>>>>
> > >>>>> VIEWS should also support the TEMPORARY keyword according to
> FLIP-64.
> > >>>>>
> > >>>>> Otherwise the FLIP looks good to me.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Timo
> > >>>>>
> > >>>>>
> > >>>>> On 08.04.20 07:31, Zhenghua Gao wrote:
> > >>>>>> @Danny Chan <[hidden email]>  you‘re right. I have updated
> > the
> > >>>>> doc.
> > >>>>>>
> > >>>>>> *Best Regards,*
> > >>>>>> *Zhenghua Gao*
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, Apr 8, 2020 at 1:20 PM Danny Chan <[hidden email]>
> > >> wrote:
> > >>>>>>
> > >>>>>>> +1 for the proposal, a small concern for drop view statement:
> > >>>>>>>
> > >>>>>>> dropViewStatement:
> > >>>>>>>          DROP VIEW name [ IF EXISTS ]
> > >>>>>>> I think the drop statement should be
> > >>>>>>> DROP VIEW [ IF EXISTS ] name
> > >>>>>>>
> > >>>>>>> Best,
> > >>>>>>> Danny Chan
> > >>>>>>> 在 2020年4月8日 +0800 AM11:54，Kurt Young <[hidden email]>，写道：
> > >>>>>>>> This FLIP seems to be quite straightforward, +1 from my side.
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>> Kurt
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 7, 2020 at 8:42 PM Zhenghua Gao <[hidden email]>
> > >> wrote:
> > >>>>>>>>
> > >>>>>>>>> forward the reply to ML too.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> *Best Regards,*
> > >>>>>>>>> *Zhenghua Gao*
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> ---------- Forwarded message ---------
> > >>>>>>>>> From: Zhenghua Gao <[hidden email]>
> > >>>>>>>>> Date: Tue, Apr 7, 2020 at 8:40 PM
> > >>>>>>>>> Subject: Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL
> > >>>>>>>>> To: godfrey he <[hidden email]>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>> regarding to "Interoperability between Flink and Hive is not
> > >>>>>>>>> guaranteed", can you explain this more?
> > >>>>>>>>> We have several limitations of interoperability between flink
> > >> objects
> > >>>>>>> and
> > >>>>>>>>> hive objects (tables, functions, etc).
> > >>>>>>>>> So we don't promise the interoperability of views between flink
> > and
> > >>>>>>> hive
> > >>>>>>>>> since a view is defined base on these objects.
> > >>>>>>>>>
> > >>>>>>>>>>> "CREATE VIEW [ IF NOT EXISTS ]"
> > >>>>>>>>> This should be "CREATE VIEW [OR REPLACE]".
> > >>>>>>>>>
> > >>>>>>>>>>> "DESC"
> > >>>>>>>>> It's a shortcut of "DESCRIBE" in SQL Client (See desc table
> xxx).
> > >>>>>>>>> In DDL, we should only support "SHOW VIEWS" and "DESCRIBE VIEW
> > >> xxx".
> > >>>>>>>>>
> > >>>>>>>>> I have updated the design doc, thanks.
> > >>>>>>>>>
> > >>>>>>>>> *Best Regards,*
> > >>>>>>>>> *Zhenghua Gao*
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Apr 7, 2020 at 8:09 PM godfrey he <[hidden email]
> >
> > >>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi Zhenghua,
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks for driving this. It's one step forward that
> > >> TableEnvironment
> > >>>>>>>>>> supports more complete SQLs.
> > >>>>>>>>>> I have a few minor questions:
> > >>>>>>>>>> 1. regarding to "Interoperability between Flink and Hive is
> not
> > >>>>>>>>>> guaranteed", can you explain this more?
> > >>>>>>>>>> 2. regarding to "The Grammar", Calcite does not support
> "CREATE
> > >> VIEW
> > >>>>>>> [ IF
> > >>>>>>>>>> NOT EXISTS ]", instead supports "CREATE [OR REPLACE]". "SHOW
> > >> VIEWS"
> > >>>>>>> and
> > >>>>>>>>>> "DESCRIBE VIEW xx" are not supported now. Calcite does not
> > support
> > >>>>>>>>> describe
> > >>>>>>>>>> an object through "DESC" . I think It's better this flip can
> > >> support
> > >>>>>>>>> "SHOW
> > >>>>>>>>>> VIEWS" and "DESCRIBE VIEW xx".
> > >>>>>>>>>>
> > >>>>>>>>>> Best,
> > >>>>>>>>>> Godfrey
> > >>>>>>>>>>
> > >>>>>>>>>> Zhenghua Gao <[hidden email]> 于2020年4月3日周五 下午3:04写道：
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi community,
> > >>>>>>>>>>>
> > >>>>>>>>>>> It's a long time since we started the discussion of
> supporting
> > >>>>>>> view in
> > >>>>>>>>>>> FLINK SQL.
> > >>>>>>>>>>> Flink also continues to move forward since then.
> > >>>>>>>>>>> FLINK-10232 introduces the grammar and FLINK-12905 supports
> > >>>>>>> CatalogView
> > >>>>>>>>>>> in blink planner.
> > >>>>>>>>>>> The missing link is validate the view definition and store
> the
> > >>>>>>>>>>> original/expanded text in the catalog.
> > >>>>>>>>>>> I have updated the design doc of FLIP-71,
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71%3A+E2E+View+support+in+FLINK+SQL
> > >>>>>>>>>>>
> > >>>>>>>>>>> Any comments and feedbacks are welcome and appreciated.
> Thanks.
> > >>>>>>>>>>>
> > >>>>>>>>>>> *Best Regards,*
> > >>>>>>>>>>> *Zhenghua Gao*
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Sep 17, 2019 at 11:51 AM Zhenghua Gao <
> > [hidden email]>
> > >>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi folks,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> In umbrella task FLINK-10232 we have introduced CREATE/DROP
> > VIEW
> > >>>>>>>>> grammar
> > >>>>>>>>>>>> in our module flink-sql-parser. But we don't support view
> > >>>>>>> objects in
> > >>>>>>>>>>>> neither blink planner nor old planner.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'd like to kick off a discussion on end to end view support
> > in
> > >>>>>>> Flink
> > >>>>>>>>>>>> SQL in blink planner. It's helpful to improve the usability
> of
> > >>>>>>> the
> > >>>>>>>>>>>> framework for SQL users.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>
> >
> https://docs.google.com/document/d/14bx0t8wYH7_o4ChNkDoBFGn-i0T-Q7kUiOFvDd13_Fk/edit#heading=h.m031smarjj9p
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> In short, it:
> > >>>>>>>>>>>> - support define views and store them in catalog
> > >>>>>>>>>>>> - support drop view definitions from catalog
> > >>>>>>>>>>>> - support query views
> > >>>>>>>>>>>> - support other view related DDLs
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Any comments and feedbacks are welcome and appreciated.
> > Thanks.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> *Best Regards,*
> > >>>>>>>>>>>> *Zhenghua Gao*
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>
> > >>
> > >
> >
> >
>