[DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
Locked
[DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
 
|
Hi everyone,
In flink 1.9, we have introduced some awesome features such as complete catalog support[1] and sql ddl support[2]. These features have been a critical integration for Flink to be able to manage data and metadata like a classic RDBMS and make developers more easy to construct their real-time/off-line warehouse or sth similar base on flink. But there is still a lack of support on how Flink SQL DDL to manage metadata and data like classic RDBMS such as `alter table rename` and so on. So I’d like to kick off a discussion on enhancing Flink Sql Ddls: https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> In short, it: - Add Catalog DDL enhancement support: show catalogs / describe catalog / use catalog - Add Database DDL enhancement support: show databses / create database / drop database/ alter database - Add Table DDL enhancement support: show tables/ describe table / alter table - Add Function DDL enhancement support: show functions/ create function /drop function Looking forward to your opinions. Best, Terry Wang [1]:https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> [2]:https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-11275>0232 <https://issues.apache.org/jira/browse/FLINK-11275> |
|
|
Hi Terry,
Thanks for driving the effort! I left some comments in the doc. AFAIU, the biggest motivation is to support DDLs in sql parser so that both Table API and SQL CLI can share the stack, despite that SQL CLI has already supported some commands itself. However, I don't see details on how SQL CLI would migrate and depend on sql parser, and how Table API and SQL CLI would actually share SQL parser. I'm not sure yet how much work that will take, just want to double check that you didn't include them because they are very trivial according to your estimate? On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email]> wrote: > Hi everyone, > > In flink 1.9, we have introduced some awesome features such as complete > catalog support[1] and sql ddl support[2]. These features have been a > critical integration for Flink to be able to manage data and metadata like > a classic RDBMS and make developers more easy to construct their > real-time/off-line warehouse or sth similar base on flink. > > But there is still a lack of support on how Flink SQL DDL to manage > metadata and data like classic RDBMS such as `alter table rename` and so on. > > So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > In short, it: > - Add Catalog DDL enhancement support: show catalogs / describe > catalog / use catalog > - Add Database DDL enhancement support: show databses / create > database / drop database/ alter database > - Add Table DDL enhancement support: show tables/ describe > table / alter table > - Add Function DDL enhancement support: show functions/ create > function /drop function > > Looking forward to your opinions. > > Best, > Terry Wang > > > > [1]:https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> > [2]:https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-11275>0232 > <https://issues.apache.org/jira/browse/FLINK-11275> |
|
|
BTW, will there be a "CREATE/DROP CATALOG" DDL?
Though it's not SQL standard, I can see it'll be useful and handy for our end users in many cases. On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email]> wrote: > Hi Terry, > > Thanks for driving the effort! I left some comments in the doc. > > AFAIU, the biggest motivation is to support DDLs in sql parser so that > both Table API and SQL CLI can share the stack, despite that SQL CLI has > already supported some commands itself. However, I don't see details on how > SQL CLI would migrate and depend on sql parser, and how Table API and SQL > CLI would actually share SQL parser. I'm not sure yet how much work that > will take, just want to double check that you didn't include them because > they are very trivial according to your estimate? > > > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email]> wrote: > >> Hi everyone, >> >> In flink 1.9, we have introduced some awesome features such as complete >> catalog support[1] and sql ddl support[2]. These features have been a >> critical integration for Flink to be able to manage data and metadata like >> a classic RDBMS and make developers more easy to construct their >> real-time/off-line warehouse or sth similar base on flink. >> >> But there is still a lack of support on how Flink SQL DDL to manage >> metadata and data like classic RDBMS such as `alter table rename` and so on. >> >> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: >> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing >> > >> >> In short, it: >> - Add Catalog DDL enhancement support: show catalogs / describe >> catalog / use catalog >> - Add Database DDL enhancement support: show databses / create >> database / drop database/ alter database >> - Add Table DDL enhancement support: show tables/ describe >> table / alter table >> - Add Function DDL enhancement support: show functions/ create >> function /drop function >> >> Looking forward to your opinions. >> >> Best, >> Terry Wang >> >> >> >> [1]:https://issues.apache.org/jira/browse/FLINK-11275 < >> https://issues.apache.org/jira/browse/FLINK-11275> >> [2]:https://issues.apache.org/jira/browse/FLINK-1 < >> https://issues.apache.org/jira/browse/FLINK-11275>0232 >> <https://issues.apache.org/jira/browse/FLINK-11275> > > |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Thanks Bowen for your insightful comments, I’ll think twice and do corresponding improvement.
After finished, I’ll update in this mailing thread again. Best, Terry Wang > 在 2019年9月25日，上午8:28，Bowen Li <[hidden email]> 写道： > > BTW, will there be a "CREATE/DROP CATALOG" DDL? > > Though it's not SQL standard, I can see it'll be useful and handy for our end users in many cases. > > On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] <mailto:[hidden email]>> wrote: > Hi Terry, > > Thanks for driving the effort! I left some comments in the doc. > > AFAIU, the biggest motivation is to support DDLs in sql parser so that both Table API and SQL CLI can share the stack, despite that SQL CLI has already supported some commands itself. However, I don't see details on how SQL CLI would migrate and depend on sql parser, and how Table API and SQL CLI would actually share SQL parser. I'm not sure yet how much work that will take, just want to double check that you didn't include them because they are very trivial according to your estimate? > > > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] <mailto:[hidden email]>> wrote: > Hi everyone, > > In flink 1.9, we have introduced some awesome features such as complete catalog support[1] and sql ddl support[2]. These features have been a critical integration for Flink to be able to manage data and metadata like a classic RDBMS and make developers more easy to construct their real-time/off-line warehouse or sth similar base on flink. > > But there is still a lack of support on how Flink SQL DDL to manage metadata and data like classic RDBMS such as `alter table rename` and so on. > > So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing>> > > In short, it: > - Add Catalog DDL enhancement support: show catalogs / describe catalog / use catalog > - Add Database DDL enhancement support: show databses / create database / drop database/ alter database > - Add Table DDL enhancement support: show tables/ describe table / alter table > - Add Function DDL enhancement support: show functions/ create function /drop function > > Looking forward to your opinions. > > Best, > Terry Wang > > > > [1]:https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> > [2]:https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1> <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>0232 > <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> |
|
|
Actually catalogs are more of system settings than of user objects that a
user might create or drop constantly. Thus, it's probably sufficient to set up catalog information in the config file, at least for now. Thanks, Xuefu On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email]> wrote: > Thanks Bowen for your insightful comments, I’ll think twice and do > corresponding improvement. > After finished, I’ll update in this mailing thread again. > Best, > Terry Wang > > > > > 在 2019年9月25日，上午8:28，Bowen Li <[hidden email]> 写道： > > > > BTW, will there be a "CREATE/DROP CATALOG" DDL? > > > > Though it's not SQL standard, I can see it'll be useful and handy for > our end users in many cases. > > > > On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] <mailto: > [hidden email]>> wrote: > > Hi Terry, > > > > Thanks for driving the effort! I left some comments in the doc. > > > > AFAIU, the biggest motivation is to support DDLs in sql parser so that > both Table API and SQL CLI can share the stack, despite that SQL CLI has > already supported some commands itself. However, I don't see details on how > SQL CLI would migrate and depend on sql parser, and how Table API and SQL > CLI would actually share SQL parser. I'm not sure yet how much work that > will take, just want to double check that you didn't include them because > they are very trivial according to your estimate? > > > > > > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] <mailto: > [hidden email]>> wrote: > > Hi everyone, > > > > In flink 1.9, we have introduced some awesome features such as complete > catalog support[1] and sql ddl support[2]. These features have been a > critical integration for Flink to be able to manage data and metadata like > a classic RDBMS and make developers more easy to construct their > real-time/off-line warehouse or sth similar base on flink. > > > > But there is still a lack of support on how Flink SQL DDL to manage > metadata and data like classic RDBMS such as `alter table rename` and so on. > > > > So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > >> > > > > In short, it: > > - Add Catalog DDL enhancement support: show catalogs / describe > catalog / use catalog > > - Add Database DDL enhancement support: show databses / create > database / drop database/ alter database > > - Add Table DDL enhancement support: show tables/ describe > table / alter table > > - Add Function DDL enhancement support: show functions/ create > function /drop function > > > > Looking forward to your opinions. > > > > Best, > > Terry Wang > > > > > > > > [1]:https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> > > [2]:https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1> < > https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>0232 > > <https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> > > -- Xuefu Zhang "In Honey We Trust!" |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Hi, everyone~
Sorry so late to reply this this thread again. I am working on FLIP-84 recently to make sql api support return value which this flip depends on. I think it’s time to pick up this discussion again and there are some new updates in this flip design: https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> 1. Remove function ddl section. For Peter Huang are working on FLIP-79 to convege function ddl support in a more overall way 2. Remove the proposed TableEnvironment SQL API Changes which is covered in FLIP-84 3. Update the design doc accoding to review comments. Looking forward to receiving more comments ~ Best, Terry Wang > 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： > > Actually catalogs are more of system settings than of user objects that a > user might create or drop constantly. Thus, it's probably sufficient to set > up catalog information in the config file, at least for now. > > Thanks, > Xuefu > > On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] <mailto:[hidden email]>> wrote: > >> Thanks Bowen for your insightful comments, I’ll think twice and do >> corresponding improvement. >> After finished, I’ll update in this mailing thread again. >> Best, >> Terry Wang >> >> >> >>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email]> 写道： >>> >>> BTW, will there be a "CREATE/DROP CATALOG" DDL? >>> >>> Though it's not SQL standard, I can see it'll be useful and handy for >> our end users in many cases. >>> >>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] <mailto:[hidden email]> <mailto: >> [hidden email] <mailto:[hidden email]>>> wrote: >>> Hi Terry, >>> >>> Thanks for driving the effort! I left some comments in the doc. >>> >>> AFAIU, the biggest motivation is to support DDLs in sql parser so that >> both Table API and SQL CLI can share the stack, despite that SQL CLI has >> already supported some commands itself. However, I don't see details on how >> SQL CLI would migrate and depend on sql parser, and how Table API and SQL >> CLI would actually share SQL parser. I'm not sure yet how much work that >> will take, just want to double check that you didn't include them because >> they are very trivial according to your estimate? >>> >>> >>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] <mailto:[hidden email]> <mailto: >> [hidden email] <mailto:[hidden email]>>> wrote: >>> Hi everyone, >>> >>> In flink 1.9, we have introduced some awesome features such as complete >> catalog support[1] and sql ddl support[2]. These features have been a >> critical integration for Flink to be able to manage data and metadata like >> a classic RDBMS and make developers more easy to construct their >> real-time/off-line warehouse or sth similar base on flink. >>> >>> But there is still a lack of support on how Flink SQL DDL to manage >> metadata and data like classic RDBMS such as `alter table rename` and so on. >>> >>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: >>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing>> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>>> >>> >>> In short, it: >>> - Add Catalog DDL enhancement support: show catalogs / describe >> catalog / use catalog >>> - Add Database DDL enhancement support: show databses / create >> database / drop database/ alter database >>> - Add Table DDL enhancement support: show tables/ describe >> table / alter table >>> - Add Function DDL enhancement support: show functions/ create >> function /drop function >>> >>> Looking forward to your opinions. >>> >>> Best, >>> Terry Wang >>> >>> >>> >>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>> >>> [2]:https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1> < >> https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1>> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>>0232 >>> <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>> >> >> > > -- > Xuefu Zhang > > "In Honey We Trust!" |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
In reply to this post by Xuefu Z
Hi Bowen~
We don’t intend to support create/drop catalog syntax in this flip, we may support it if there indeed has a strong desire. And I’m going to kick off a vote for this flip, feel free to review again. Best, Terry Wang > 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： > > Actually catalogs are more of system settings than of user objects that a > user might create or drop constantly. Thus, it's probably sufficient to set > up catalog information in the config file, at least for now. > > Thanks, > Xuefu > > On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] <mailto:[hidden email]>> wrote: > >> Thanks Bowen for your insightful comments, I’ll think twice and do >> corresponding improvement. >> After finished, I’ll update in this mailing thread again. >> Best, >> Terry Wang >> >> >> >>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email]> 写道： >>> >>> BTW, will there be a "CREATE/DROP CATALOG" DDL? >>> >>> Though it's not SQL standard, I can see it'll be useful and handy for >> our end users in many cases. >>> >>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] <mailto:[hidden email]> <mailto: >> [hidden email] <mailto:[hidden email]>>> wrote: >>> Hi Terry, >>> >>> Thanks for driving the effort! I left some comments in the doc. >>> >>> AFAIU, the biggest motivation is to support DDLs in sql parser so that >> both Table API and SQL CLI can share the stack, despite that SQL CLI has >> already supported some commands itself. However, I don't see details on how >> SQL CLI would migrate and depend on sql parser, and how Table API and SQL >> CLI would actually share SQL parser. I'm not sure yet how much work that >> will take, just want to double check that you didn't include them because >> they are very trivial according to your estimate? >>> >>> >>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] <mailto:[hidden email]> <mailto: >> [hidden email] <mailto:[hidden email]>>> wrote: >>> Hi everyone, >>> >>> In flink 1.9, we have introduced some awesome features such as complete >> catalog support[1] and sql ddl support[2]. These features have been a >> critical integration for Flink to be able to manage data and metadata like >> a classic RDBMS and make developers more easy to construct their >> real-time/off-line warehouse or sth similar base on flink. >>> >>> But there is still a lack of support on how Flink SQL DDL to manage >> metadata and data like classic RDBMS such as `alter table rename` and so on. >>> >>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: >>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing>> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>>> >>> >>> In short, it: >>> - Add Catalog DDL enhancement support: show catalogs / describe >> catalog / use catalog >>> - Add Database DDL enhancement support: show databses / create >> database / drop database/ alter database >>> - Add Table DDL enhancement support: show tables/ describe >> table / alter table >>> - Add Function DDL enhancement support: show functions/ create >> function /drop function >>> >>> Looking forward to your opinions. >>> >>> Best, >>> Terry Wang >>> >>> >>> >>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>> >>> [2]:https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1> < >> https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1>> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>>0232 >>> <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>> >> >> > > -- > Xuefu Zhang > > "In Honey We Trust!" |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Hi,
Sorry to join this so late and thanks for proposing this FLIP. After going through the proposal details, I would +1 for the changes. However, the FLIP name is kind of confusing me. It says will do DDL enhancement, and picked up a few new features to do. It looks to me the goal and content of this FLIP is kind of random. Each topic of this FLIP touched is super big, e.g. to enhance alter table command. According to SQL 2011 standard, it would contains at least so many features like: <alter table statement> ::= ALTER TABLE <table name> <alter table action> <alter table action> ::= <add column definition> | <alter column definition> | <drop column definition> | <add table constraint definition> | <alter table constraint definition> | <drop table constraint definition> | <add table period definition> | <drop table period definition> | <add system versioning clause> | <drop system versioning clause> I'm not suggesting to do all these at once, but I also didn't see any future plan or goals in the FLIP to describe the full picture here. We just picked up some random chosen features to start. But still I'm +1 to this FLIP since they are all good enhancements. Best, Kurt On Tue, Nov 5, 2019 at 10:32 PM Terry Wang <[hidden email]> wrote: > Hi Bowen~ > > We don’t intend to support create/drop catalog syntax in this flip, we > may support it if there indeed has a strong desire. > And I’m going to kick off a vote for this flip, feel free to review again. > > Best, > Terry Wang > > > > > 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： > > > > Actually catalogs are more of system settings than of user objects that a > > user might create or drop constantly. Thus, it's probably sufficient to > set > > up catalog information in the config file, at least for now. > > > > Thanks, > > Xuefu > > > > On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] <mailto: > [hidden email]>> wrote: > > > >> Thanks Bowen for your insightful comments, I’ll think twice and do > >> corresponding improvement. > >> After finished, I’ll update in this mailing thread again. > >> Best, > >> Terry Wang > >> > >> > >> > >>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email]> 写道： > >>> > >>> BTW, will there be a "CREATE/DROP CATALOG" DDL? > >>> > >>> Though it's not SQL standard, I can see it'll be useful and handy for > >> our end users in many cases. > >>> > >>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] > <mailto:[hidden email]> <mailto: > >> [hidden email] <mailto:[hidden email]>>> wrote: > >>> Hi Terry, > >>> > >>> Thanks for driving the effort! I left some comments in the doc. > >>> > >>> AFAIU, the biggest motivation is to support DDLs in sql parser so that > >> both Table API and SQL CLI can share the stack, despite that SQL CLI has > >> already supported some commands itself. However, I don't see details on > how > >> SQL CLI would migrate and depend on sql parser, and how Table API and > SQL > >> CLI would actually share SQL parser. I'm not sure yet how much work that > >> will take, just want to double check that you didn't include them > because > >> they are very trivial according to your estimate? > >>> > >>> > >>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] > <mailto:[hidden email]> <mailto: > >> [hidden email] <mailto:[hidden email]>>> wrote: > >>> Hi everyone, > >>> > >>> In flink 1.9, we have introduced some awesome features such as complete > >> catalog support[1] and sql ddl support[2]. These features have been a > >> critical integration for Flink to be able to manage data and metadata > like > >> a classic RDBMS and make developers more easy to construct their > >> real-time/off-line warehouse or sth similar base on flink. > >>> > >>> But there is still a lack of support on how Flink SQL DDL to manage > >> metadata and data like classic RDBMS such as `alter table rename` and > so on. > >>> > >>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > >>> > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > >> > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >>>> > >>> > >>> In short, it: > >>> - Add Catalog DDL enhancement support: show catalogs / describe > >> catalog / use catalog > >>> - Add Database DDL enhancement support: show databses / create > >> database / drop database/ alter database > >>> - Add Table DDL enhancement support: show tables/ describe > >> table / alter table > >>> - Add Function DDL enhancement support: show functions/ create > >> function /drop function > >>> > >>> Looking forward to your opinions. > >>> > >>> Best, > >>> Terry Wang > >>> > >>> > >>> > >>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>> > >>> [2]:https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1> < > >> https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1>> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>>0232 > >>> <https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>> > >> > >> > > > > -- > > Xuefu Zhang > > > > "In Honey We Trust!" > > |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Hi, Kurt~
Thanks for your vote and pointing out some deficiency of this flip. I’ll try to avoid making similar mistakes. Best, Terry Wang > 2019年11月8日 11:28，Kurt Young <[hidden email]> 写道： > > Hi, > > Sorry to join this so late and thanks for proposing this FLIP. After > going through the proposal details, I would +1 for the changes. > > However, the FLIP name is kind of confusing me. It says will do > DDL enhancement, and picked up a few new features to do. It looks > to me the goal and content of this FLIP is kind of random. > > Each topic of this FLIP touched is super big, e.g. to enhance > alter table command. According to SQL 2011 standard, it would contains > at least so many features like: > > <alter table statement> ::= > ALTER TABLE <table name> <alter table action> > <alter table action> ::= > <add column definition> > | <alter column definition> > | <drop column definition> > | <add table constraint definition> > | <alter table constraint definition> > | <drop table constraint definition> > | <add table period definition> > | <drop table period definition> > | <add system versioning clause> > | <drop system versioning clause> > > I'm not suggesting to do all these at once, but I also didn't see any > future plan or goals in the FLIP to describe the full picture here. We just > picked up some random chosen features to start. > > But still I'm +1 to this FLIP since they are all good enhancements. > > Best, > Kurt > > > On Tue, Nov 5, 2019 at 10:32 PM Terry Wang <[hidden email] <mailto:[hidden email]>> wrote: > >> Hi Bowen~ >> >> We don’t intend to support create/drop catalog syntax in this flip, we >> may support it if there indeed has a strong desire. >> And I’m going to kick off a vote for this flip, feel free to review again. >> >> Best, >> Terry Wang >> >> >> >>> 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： >>> >>> Actually catalogs are more of system settings than of user objects that a >>> user might create or drop constantly. Thus, it's probably sufficient to >> set >>> up catalog information in the config file, at least for now. >>> >>> Thanks, >>> Xuefu >>> >>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] <mailto:[hidden email]> <mailto: >> [hidden email] <mailto:[hidden email]>>> wrote: >>> >>>> Thanks Bowen for your insightful comments, I’ll think twice and do >>>> corresponding improvement. >>>> After finished, I’ll update in this mailing thread again. >>>> Best, >>>> Terry Wang >>>> >>>> >>>> >>>>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email] <mailto:[hidden email]>> 写道： >>>>> >>>>> BTW, will there be a "CREATE/DROP CATALOG" DDL? >>>>> >>>>> Though it's not SQL standard, I can see it'll be useful and handy for >>>> our end users in many cases. >>>>> >>>>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] <mailto:[hidden email]> >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: >>>> [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>> wrote: >>>>> Hi Terry, >>>>> >>>>> Thanks for driving the effort! I left some comments in the doc. >>>>> >>>>> AFAIU, the biggest motivation is to support DDLs in sql parser so that >>>> both Table API and SQL CLI can share the stack, despite that SQL CLI has >>>> already supported some commands itself. However, I don't see details on >> how >>>> SQL CLI would migrate and depend on sql parser, and how Table API and >> SQL >>>> CLI would actually share SQL parser. I'm not sure yet how much work that >>>> will take, just want to double check that you didn't include them >> because >>>> they are very trivial according to your estimate? >>>>> >>>>> >>>>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] <mailto:[hidden email]> >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: >>>> [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>> wrote: >>>>> Hi everyone, >>>>> >>>>> In flink 1.9, we have introduced some awesome features such as complete >>>> catalog support[1] and sql ddl support[2]. These features have been a >>>> critical integration for Flink to be able to manage data and metadata >> like >>>> a classic RDBMS and make developers more easy to construct their >>>> real-time/off-line warehouse or sth similar base on flink. >>>>> >>>>> But there is still a lack of support on how Flink SQL DDL to manage >>>> metadata and data like classic RDBMS such as `alter table rename` and >> so on. >>>>> >>>>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: >>>>> >>>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>> >>>> < >>>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>>> >>>> < >>>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>> >>>> < >>>> >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >> < >> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing <https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing> >>> >>>>>> >>>>> >>>>> In short, it: >>>>> - Add Catalog DDL enhancement support: show catalogs / describe >>>> catalog / use catalog >>>>> - Add Database DDL enhancement support: show databses / create >>>> database / drop database/ alter database >>>>> - Add Table DDL enhancement support: show tables/ describe >>>> table / alter table >>>>> - Add Function DDL enhancement support: show functions/ create >>>> function /drop function >>>>> >>>>> Looking forward to your opinions. >>>>> >>>>> Best, >>>>> Terry Wang >>>>> >>>>> >>>>> >>>>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>>> >>>>> [2]:https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1> < >> https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1>> < >>>> https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1> < >> https://issues.apache.org/jira/browse/FLINK-1 <https://issues.apache.org/jira/browse/FLINK-1>>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>>>0232 >>>>> <https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>> < >>>> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275> < >> https://issues.apache.org/jira/browse/FLINK-11275 <https://issues.apache.org/jira/browse/FLINK-11275>>>> >>>> >>>> >>> >>> -- >>> Xuefu Zhang >>> >>> "In Honey We Trust!" |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Hi Terry,
I wouldn't say it's a mistake, I also don't have any suggestions about the issue. I just saw this and want to point it out to bring more attention. Maybe someone has some good opinion on that part and we can discuss around it and have some good advises for the community in the future. Best, Kurt On Fri, Nov 8, 2019 at 2:16 PM Terry Wang <[hidden email]> wrote: > Hi, Kurt~ > > Thanks for your vote and pointing out some deficiency of this flip. I’ll > try to avoid making similar mistakes. > > Best, > Terry Wang > > > > > 2019年11月8日 11:28，Kurt Young <[hidden email]> 写道： > > > > Hi, > > > > Sorry to join this so late and thanks for proposing this FLIP. After > > going through the proposal details, I would +1 for the changes. > > > > However, the FLIP name is kind of confusing me. It says will do > > DDL enhancement, and picked up a few new features to do. It looks > > to me the goal and content of this FLIP is kind of random. > > > > Each topic of this FLIP touched is super big, e.g. to enhance > > alter table command. According to SQL 2011 standard, it would contains > > at least so many features like: > > > > <alter table statement> ::= > > ALTER TABLE <table name> <alter table action> > > <alter table action> ::= > > <add column definition> > > | <alter column definition> > > | <drop column definition> > > | <add table constraint definition> > > | <alter table constraint definition> > > | <drop table constraint definition> > > | <add table period definition> > > | <drop table period definition> > > | <add system versioning clause> > > | <drop system versioning clause> > > > > I'm not suggesting to do all these at once, but I also didn't see any > > future plan or goals in the FLIP to describe the full picture here. We > just > > picked up some random chosen features to start. > > > > But still I'm +1 to this FLIP since they are all good enhancements. > > > > Best, > > Kurt > > > > > > On Tue, Nov 5, 2019 at 10:32 PM Terry Wang <[hidden email] <mailto: > [hidden email]>> wrote: > > > >> Hi Bowen~ > >> > >> We don’t intend to support create/drop catalog syntax in this flip, we > >> may support it if there indeed has a strong desire. > >> And I’m going to kick off a vote for this flip, feel free to review > again. > >> > >> Best, > >> Terry Wang > >> > >> > >> > >>> 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： > >>> > >>> Actually catalogs are more of system settings than of user objects > that a > >>> user might create or drop constantly. Thus, it's probably sufficient to > >> set > >>> up catalog information in the config file, at least for now. > >>> > >>> Thanks, > >>> Xuefu > >>> > >>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] > <mailto:[hidden email]> <mailto: > >> [hidden email] <mailto:[hidden email]>>> wrote: > >>> > >>>> Thanks Bowen for your insightful comments, I’ll think twice and do > >>>> corresponding improvement. > >>>> After finished, I’ll update in this mailing thread again. > >>>> Best, > >>>> Terry Wang > >>>> > >>>> > >>>> > >>>>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email] <mailto: > [hidden email]>> 写道： > >>>>> > >>>>> BTW, will there be a "CREATE/DROP CATALOG" DDL? > >>>>> > >>>>> Though it's not SQL standard, I can see it'll be useful and handy for > >>>> our end users in many cases. > >>>>> > >>>>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] > <mailto:[hidden email]> > >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: > >>>> [hidden email] <mailto:[hidden email]> <mailto: > [hidden email] <mailto:[hidden email]>>>> wrote: > >>>>> Hi Terry, > >>>>> > >>>>> Thanks for driving the effort! I left some comments in the doc. > >>>>> > >>>>> AFAIU, the biggest motivation is to support DDLs in sql parser so > that > >>>> both Table API and SQL CLI can share the stack, despite that SQL CLI > has > >>>> already supported some commands itself. However, I don't see details > on > >> how > >>>> SQL CLI would migrate and depend on sql parser, and how Table API and > >> SQL > >>>> CLI would actually share SQL parser. I'm not sure yet how much work > that > >>>> will take, just want to double check that you didn't include them > >> because > >>>> they are very trivial according to your estimate? > >>>>> > >>>>> > >>>>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] > <mailto:[hidden email]> > >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: > >>>> [hidden email] <mailto:[hidden email]> <mailto: > [hidden email] <mailto:[hidden email]>>>> wrote: > >>>>> Hi everyone, > >>>>> > >>>>> In flink 1.9, we have introduced some awesome features such as > complete > >>>> catalog support[1] and sql ddl support[2]. These features have been a > >>>> critical integration for Flink to be able to manage data and metadata > >> like > >>>> a classic RDBMS and make developers more easy to construct their > >>>> real-time/off-line warehouse or sth similar base on flink. > >>>>> > >>>>> But there is still a lack of support on how Flink SQL DDL to manage > >>>> metadata and data like classic RDBMS such as `alter table rename` and > >> so on. > >>>>> > >>>>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > >>>>> > >>>> > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >>> > >>>> < > >>>> > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >>>> > >>>> < > >>>> > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >>> > >>>> < > >>>> > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >> < > >> > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > < > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > >>> > >>>>>> > >>>>> > >>>>> In short, it: > >>>>> - Add Catalog DDL enhancement support: show catalogs / > describe > >>>> catalog / use catalog > >>>>> - Add Database DDL enhancement support: show databses / create > >>>> database / drop database/ alter database > >>>>> - Add Table DDL enhancement support: show tables/ describe > >>>> table / alter table > >>>>> - Add Function DDL enhancement support: show functions/ create > >>>> function /drop function > >>>>> > >>>>> Looking forward to your opinions. > >>>>> > >>>>> Best, > >>>>> Terry Wang > >>>>> > >>>>> > >>>>> > >>>>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>>> > >>>>> [2]:https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1> < > >> https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1>> < > >>>> https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1> < > >> https://issues.apache.org/jira/browse/FLINK-1 < > https://issues.apache.org/jira/browse/FLINK-1>>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>>>0232 > >>>>> <https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>> < > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275> < > >> https://issues.apache.org/jira/browse/FLINK-11275 < > https://issues.apache.org/jira/browse/FLINK-11275>>>> > >>>> > >>>> > >>> > >>> -- > >>> Xuefu Zhang > >>> > >>> "In Honey We Trust!" > > |
Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement
|
|
Thanks Kurt and Terry for bringing this to my attention.
Agree each topic of this FLIP touched is super big. But because it's too big, we can't cover everything in one FLIP. So I think there are two ways: 1. Decompose this big thing into an independent topic(FLIP), so that each FLIP can be explained very clearly. 2. Only some basic concepts are put forward in this FLIP. It should be noted that these basic things cannot conflict with the overall design (nor with SQL standards), nor can API change in the future development. Then we will improve each point step by step. I think our road is #2 at present. This path may be more necessary for users, because most users only need a basic framework for their common cases. Best, Jingsong Lee On Fri, Nov 8, 2019 at 2:20 PM Kurt Young <[hidden email]> wrote: > Hi Terry, > > I wouldn't say it's a mistake, I also don't have any suggestions about > the issue. I just saw this and want to point it out to bring more > attention. > Maybe someone has some good opinion on that part and we can > discuss around it and have some good advises for the community in > the future. > > Best, > Kurt > > > On Fri, Nov 8, 2019 at 2:16 PM Terry Wang <[hidden email]> wrote: > > > Hi, Kurt~ > > > > Thanks for your vote and pointing out some deficiency of this flip. I’ll > > try to avoid making similar mistakes. > > > > Best, > > Terry Wang > > > > > > > > > 2019年11月8日 11:28，Kurt Young <[hidden email]> 写道： > > > > > > Hi, > > > > > > Sorry to join this so late and thanks for proposing this FLIP. After > > > going through the proposal details, I would +1 for the changes. > > > > > > However, the FLIP name is kind of confusing me. It says will do > > > DDL enhancement, and picked up a few new features to do. It looks > > > to me the goal and content of this FLIP is kind of random. > > > > > > Each topic of this FLIP touched is super big, e.g. to enhance > > > alter table command. According to SQL 2011 standard, it would contains > > > at least so many features like: > > > > > > <alter table statement> ::= > > > ALTER TABLE <table name> <alter table action> > > > <alter table action> ::= > > > <add column definition> > > > | <alter column definition> > > > | <drop column definition> > > > | <add table constraint definition> > > > | <alter table constraint definition> > > > | <drop table constraint definition> > > > | <add table period definition> > > > | <drop table period definition> > > > | <add system versioning clause> > > > | <drop system versioning clause> > > > > > > I'm not suggesting to do all these at once, but I also didn't see any > > > future plan or goals in the FLIP to describe the full picture here. We > > just > > > picked up some random chosen features to start. > > > > > > But still I'm +1 to this FLIP since they are all good enhancements. > > > > > > Best, > > > Kurt > > > > > > > > > On Tue, Nov 5, 2019 at 10:32 PM Terry Wang <[hidden email] > <mailto: > > [hidden email]>> wrote: > > > > > >> Hi Bowen~ > > >> > > >> We don’t intend to support create/drop catalog syntax in this flip, > we > > >> may support it if there indeed has a strong desire. > > >> And I’m going to kick off a vote for this flip, feel free to review > > again. > > >> > > >> Best, > > >> Terry Wang > > >> > > >> > > >> > > >>> 2019年9月26日 00:44，Xuefu Z <[hidden email]> 写道： > > >>> > > >>> Actually catalogs are more of system settings than of user objects > > that a > > >>> user might create or drop constantly. Thus, it's probably sufficient > to > > >> set > > >>> up catalog information in the config file, at least for now. > > >>> > > >>> Thanks, > > >>> Xuefu > > >>> > > >>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang <[hidden email] > > <mailto:[hidden email]> <mailto: > > >> [hidden email] <mailto:[hidden email]>>> wrote: > > >>> > > >>>> Thanks Bowen for your insightful comments, I’ll think twice and do > > >>>> corresponding improvement. > > >>>> After finished, I’ll update in this mailing thread again. > > >>>> Best, > > >>>> Terry Wang > > >>>> > > >>>> > > >>>> > > >>>>> 在 2019年9月25日，上午8:28，Bowen Li <[hidden email] <mailto: > > [hidden email]>> 写道： > > >>>>> > > >>>>> BTW, will there be a "CREATE/DROP CATALOG" DDL? > > >>>>> > > >>>>> Though it's not SQL standard, I can see it'll be useful and handy > for > > >>>> our end users in many cases. > > >>>>> > > >>>>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li <[hidden email] > > <mailto:[hidden email]> > > >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: > > >>>> [hidden email] <mailto:[hidden email]> <mailto: > > [hidden email] <mailto:[hidden email]>>>> wrote: > > >>>>> Hi Terry, > > >>>>> > > >>>>> Thanks for driving the effort! I left some comments in the doc. > > >>>>> > > >>>>> AFAIU, the biggest motivation is to support DDLs in sql parser so > > that > > >>>> both Table API and SQL CLI can share the stack, despite that SQL CLI > > has > > >>>> already supported some commands itself. However, I don't see details > > on > > >> how > > >>>> SQL CLI would migrate and depend on sql parser, and how Table API > and > > >> SQL > > >>>> CLI would actually share SQL parser. I'm not sure yet how much work > > that > > >>>> will take, just want to double check that you didn't include them > > >> because > > >>>> they are very trivial according to your estimate? > > >>>>> > > >>>>> > > >>>>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang <[hidden email] > > <mailto:[hidden email]> > > >> <mailto:[hidden email] <mailto:[hidden email]>> <mailto: > > >>>> [hidden email] <mailto:[hidden email]> <mailto: > > [hidden email] <mailto:[hidden email]>>>> wrote: > > >>>>> Hi everyone, > > >>>>> > > >>>>> In flink 1.9, we have introduced some awesome features such as > > complete > > >>>> catalog support[1] and sql ddl support[2]. These features have been > a > > >>>> critical integration for Flink to be able to manage data and > metadata > > >> like > > >>>> a classic RDBMS and make developers more easy to construct their > > >>>> real-time/off-line warehouse or sth similar base on flink. > > >>>>> > > >>>>> But there is still a lack of support on how Flink SQL DDL to manage > > >>>> metadata and data like classic RDBMS such as `alter table rename` > and > > >> so on. > > >>>>> > > >>>>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls: > > >>>>> > > >>>> > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >> < > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >>> > > >>>> < > > >>>> > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >> < > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >>>> > > >>>> < > > >>>> > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >> < > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >>> > > >>>> < > > >>>> > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >> < > > >> > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > < > > > https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing > > > > > >>> > > >>>>>> > > >>>>> > > >>>>> In short, it: > > >>>>> - Add Catalog DDL enhancement support: show catalogs / > > describe > > >>>> catalog / use catalog > > >>>>> - Add Database DDL enhancement support: show databses / > create > > >>>> database / drop database/ alter database > > >>>>> - Add Table DDL enhancement support: show tables/ describe > > >>>> table / alter table > > >>>>> - Add Function DDL enhancement support: show functions/ > create > > >>>> function /drop function > > >>>>> > > >>>>> Looking forward to your opinions. > > >>>>> > > >>>>> Best, > > >>>>> Terry Wang > > >>>>> > > >>>>> > > >>>>> > > >>>>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>>>> > > >>>>> [2]:https://issues.apache.org/jira/browse/FLINK-1 < > > https://issues.apache.org/jira/browse/FLINK-1> < > > >> https://issues.apache.org/jira/browse/FLINK-1 < > > https://issues.apache.org/jira/browse/FLINK-1>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-1 < > > https://issues.apache.org/jira/browse/FLINK-1> < > > >> https://issues.apache.org/jira/browse/FLINK-1 < > > https://issues.apache.org/jira/browse/FLINK-1>>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>>>>0232 > > >>>>> <https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>> < > > >>>> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275> < > > >> https://issues.apache.org/jira/browse/FLINK-11275 < > > https://issues.apache.org/jira/browse/FLINK-11275>>>> > > >>>> > > >>>> > > >>> > > >>> -- > > >>> Xuefu Zhang > > >>> > > >>> "In Honey We Trust!" > > > > > -- Best, Jingsong Lee |
«
Return to (DEPRECATED) Apache Flink Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |