Kafka Checkpointing weird behavior.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Kafka Checkpointing weird behavior.

Dominik Wosiński
Hey,
I just want to understand something, because I am observing weird behavior
of Kafka Consumer > 0.8 .

So the idea is, if we enable the checkpointing and enable the commit
offsets on checkpoint, which AFAIK is enabled by default, then for versions
of Kafka > 0.8 we should see the changes in the topics where broker keeps
the offsets for the particular groups am I correct here ?

And do I understand it correctly, that we can keep the offsets for Flink
between job restarts In two ways:
1) Cancel with savepoint and start the job with savepoint again.
2) When using group offsets Flink should be able to determine the offset
for particular group by using the Kafka offsets ?

Is my understanding correct?

Thanks in advance,
Best Regards.
Dom.