[jira] [Created] (FLINK-18758) Support debezium data type in debezium format

[Shang Yuanchun (Jira)]

Leonard Xu created FLINK-18758:
----------------------------------

             Summary: Support debezium data type in debezium format
                 Key: FLINK-18758
                 URL: https://issues.apache.org/jira/browse/FLINK-18758
             Project: Flink
          Issue Type: Improvement
          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
            Reporter: Leonard Xu


Currently debezium json format  wrapper the json format to serialize/deserialize the debezuim json format data.

But debezium json format has its own data type which consists of Literal Type and Semantic Type[1], i.g. Date type in debezium is the an integer which represents the number of days since epoch rather than a string with 'yyyy-MM-dd' pattern.  
{code:java}
{   "schema":{      "fields":[         {            "fields":[               {                 
                 "type":"int32",
                  "optional":false,
                  "name":"io.debezium.time.Date",
                  "version":1,
                  "field":"order_date"
               },
               {                  "type":"int32",
                  "optional":false,
                  "field":"quantity"
               }
            ]
         }
      ]
   },
   "payload":{ 
      "before":null,
      "after":{ 
         "order_date":16852,
         "quantity":1
      },
      "op":"c",
      "ts_ms":1596081813474
   }
} {code}
 

I think we need obtain the debezuim data type from schema information and then serialize/deserialize the data in payload.

[1][https://debezium.io/documentation/reference/1.2/connectors/mysql.html]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)