[jira] [Created] (FLINK-16865) 【Flink Kafka Connector】Restore from Savepoint,if add new Kafka topic,Flink will consume the new topic from earlist,it may cause duplicate data sink

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-16865) 【Flink Kafka Connector】Restore from Savepoint,if add new Kafka topic,Flink will consume the new topic from earlist,it may cause duplicate data sink

Shang Yuanchun (Jira)
zhisheng created FLINK-16865:
--------------------------------

             Summary: 【Flink Kafka Connector】Restore from Savepoint,if add new Kafka topic,Flink will consume the new topic from earlist,it may cause duplicate data sink
                 Key: FLINK-16865
                 URL: https://issues.apache.org/jira/browse/FLINK-16865
             Project: Flink
          Issue Type: Bug
            Reporter: zhisheng
         Attachments: image-2020-03-30-19-57-42-451.png

h3. 【Flink Kafka Connector】

If the job adds another Kafka topic when the job start from Savepoint, it will start to consume from the earlist of that topic, it may cause duplicate data sink.

I found the config is in , maybe it can be 

I found that the configuration is already written in the code FlinkKafkaConsumerBase#open(), maybe it can be Configurable.

 

!image-2020-03-30-19-57-42-451.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)