flink Kafka connector

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

flink Kafka connector

Arun Balan
Hi, I am trying to use the flink-kafka-connector and I notice that every time I restart my application it re-reads the last message on the kafka topic. So if the latest offset on the topic is 10, then when the application is restarted, kafka will re-read message 10. Why is this the behavior? I would assume that the last message has already been read and offset committed. I require that messages that are already processed from the topic not be reprocessed. Any insight would be helpful.

Thanks
Arun Balan