Timo Walther created FLINK-11184:
------------------------------------
Summary: Rework TableSource and TableSink interfaces
Key: FLINK-11184
URL:
https://issues.apache.org/jira/browse/FLINK-11184 Project: Flink
Issue Type: New Feature
Components: Table API & SQL
Reporter: Timo Walther
Assignee: Timo Walther
There are a couple of shortcomings with the current {{TableSource}} and {{TableSink}} interface design. Some of the issues are covered in a [basic design document|
https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit#heading=h.41fd6rs7b3cf] that was published a while ago.
The design document has not been updated for some time and partially overlaps with the [current SQL DDL discussion|
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-SQL-DDL-Design-td25006.html] for the {{CREATE TABLE}} statement on the ML.
What needs to be solved:
- How to unify sources and sinks in regards of schema and time attributes?
- How to define watermarks, timestamp extractors or timestamp ingestion?
- How to define primary keys and partitioning keys?
- How to differentiate between update modes for tables (i.e. how to read from a append, retraction, or update table)?
- How to express all of the above without pulling in to many dependencies on other Flink modules if source and sink interfaces are located in {{flink-table-spi}} package?
As of the current state of the discussion, it seems that we might extend {{TableSchema}} to allow for returning the information above and remove current interfaces such as {{DefinedRowtimeAttribute}} or {{DefinedFieldMapping}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
