Timo Walther created FLINK-17158:
------------------------------------
Summary: Watermark strategy property cannot be expressed in YAML
Key: FLINK-17158
URL:
https://issues.apache.org/jira/browse/FLINK-17158 Project: Flink
Issue Type: Bug
Components: Table SQL / API, Table SQL / Ecosystem
Reporter: Timo Walther
If the watermark strategy needs to be serialized into string-based properties for catalog storage, it is serialized into the following string-based properties:
{code}
DescriptorProperties properties = new DescriptorProperties();
properties.putTableSchema("schema", schema);
Map<String, String> actual = properties.asMap();
Map<String, String> expected = new HashMap<>();
expected.put("schema.0.name", "f0");
expected.put("schema.0.data-type", "BIGINT");
expected.put("schema.1.name", "f1");
expected.put("schema.1.data-type", "ROW<`q1` VARCHAR(2147483647), `q2` TIMESTAMP(9)>");
expected.put("schema.2.name", "f2");
expected.put("schema.2.data-type", "VARCHAR(2147483647)");
expected.put("schema.3.name", "f3");
expected.put("schema.3.data-type", "BIGINT");
expected.put("schema.3.expr", "f0 + 1");
expected.put("schema.4.name", "f4");
expected.put("schema.4.data-type", "DECIMAL(10, 3)");
expected.put("schema.watermark.0.rowtime", "f1.q2");
expected.put("schema.watermark.0.strategy.expr", "`f1`.`q2` - INTERVAL '5' SECOND");
expected.put("schema.watermark.0.strategy.data-type", "TIMESTAMP(3)");
{code}
However, this design does not follow a proper hierarchy as both an array and watermarks are stored under a common `schema.` key. Thus, it is impossible to express those properties in JSON or YAML file:
{code}
tables:
- name: PROD_RATES_HISTORY
type: source
update-mode: append
schema:
- name: RS_TIMESTAMP
data-type: TIMESTAMP(3)
- name: RS_SYMBOL
data-type: STRING
- name: DOUBLE
data-type: RS_RATE
???watermark:
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
