[DISCUSS] Support computed column for Flink SQL

[Danny Chan]

In umbrella task FLINK-10232[1] we have introduced CREATE TABLE grammar in our new module flink-sql-parser. And we proposed to use computed column to describe the time attribute of process time in the design doc FLINK SQL DDL[2], so user may create a table with process time attribute as following:

create table T1(
a int,
b bigint,
c varchar,
d as PROC_TIME,
) with (
k1 = v1,
k2 = v2
);

The column d would be a process time attribute for table T1. There are also many other use cases for computed columns[3].

It may not be a big change here, but may touch the TableSchema, which is a public API for user now, so i'm very appreciate for your suggestions(especially its relationship with the TableSchema).

I write a simple design doc here[3].

[1] https://issues.apache.org/jira/browse/FLINK-10232
[2] https://docs.google.com/document/d/1OmVyuPk9ibGUC-CnPHbXvCg_fdG1TeC3lXSnqcUEYmM
[3] https://docs.google.com/document/d/110TseRtTCphxETPY7uhiHpu-dph3NEesh3mYKtJ7QOY/edit?usp=sharing

Best,
Danny Chan

[Timo Walther-2]

Hi Danny,

thanks for working on this issue and writing down the concept
suggestion. We are currently still in the progress of finalizing the 1.9
release. Having proper streaming DDL support will definitely be part of
Flink 1.10. I will take a look at the whole DDL efforts very soon once
the 1.9 release is out.

Thanks,
Timo

Am 23.07.19 um 11:00 schrieb Danny Chan:

> In umbrella task FLINK-10232[1] we have introduced CREATE TABLE grammar in our new module flink-sql-parser. And we proposed to use computed column to describe the time attribute of process time in the design doc FLINK SQL DDL[2], so user may create a table with process time attribute as following:
>
> create table T1(
> a int,
> b bigint,
> c varchar,
> d as PROC_TIME,
> ) with (
> k1 = v1,
> k2 = v2
> );
>
> The column d would be a process time attribute for table T1. There are also many other use cases for computed columns[3].
>
> It may not be a big change here, but may touch the TableSchema, which is a public API for user now, so i'm very appreciate for your suggestions(especially its relationship with the TableSchema).
>
> I write a simple design doc here[3].
>
> [1] https://issues.apache.org/jira/browse/FLINK-10232
> [2] https://docs.google.com/document/d/1OmVyuPk9ibGUC-CnPHbXvCg_fdG1TeC3lXSnqcUEYmM
> [3] https://docs.google.com/document/d/110TseRtTCphxETPY7uhiHpu-dph3NEesh3mYKtJ7QOY/edit?usp=sharing
>
> Best,
> Danny Chan
>

[Jark Wu-2]

Hi Danny,

Thanks for bringing this. I agree with Timo. We can have a thorough
discussion once 1.9 release is published.
And the computed column support & implementation might be good to be
discussed with "table source and sink concept"[1].
Whatever, I left some my initial thoughts in the doc.

Best,
Jark

[1]:
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Ground-Source-and-Sink-Concepts-in-Flink-SQL-tt29126.html

On Mon, 29 Jul 2019 at 16:52, Timo Walther <[hidden email]> wrote:

> Hi Danny,
>
> thanks for working on this issue and writing down the concept
> suggestion. We are currently still in the progress of finalizing the 1.9
> release. Having proper streaming DDL support will definitely be part of
> Flink 1.10. I will take a look at the whole DDL efforts very soon once
> the 1.9 release is out.
>
> Thanks,
> Timo
>
> Am 23.07.19 um 11:00 schrieb Danny Chan:
> > In umbrella task FLINK-10232[1] we have introduced CREATE TABLE grammar
> in our new module flink-sql-parser. And we proposed to use computed column
> to describe the time attribute of process time in the design doc FLINK SQL
> DDL[2], so user may create a table with process time attribute as following:
> >
> > create table T1(
> > a int,
> > b bigint,
> > c varchar,
> > d as PROC_TIME,
> > ) with (
> > k1 = v1,
> > k2 = v2
> > );
> >
> > The column d would be a process time attribute for table T1. There are
> also many other use cases for computed columns[3].
> >
> > It may not be a big change here, but may touch the TableSchema, which is
> a public API for user now, so i'm very appreciate for your
> suggestions(especially its relationship with the TableSchema).
> >
> > I write a simple design doc here[3].
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-10232
> > [2]
> https://docs.google.com/document/d/1OmVyuPk9ibGUC-CnPHbXvCg_fdG1TeC3lXSnqcUEYmM
> > [3]
> https://docs.google.com/document/d/110TseRtTCphxETPY7uhiHpu-dph3NEesh3mYKtJ7QOY/edit?usp=sharing
> >
> > Best,
> > Danny Chan
> >
>
>