[jira] [Created] (FLINK-18865) Missing documentation of FlinkKafkaConsumer.setStartFromEarliest behavior when a partition offset becomes out of range

[Shang Yuanchun (Jira)]

Vladimirs Kotovs created FLINK-18865:
----------------------------------------

             Summary: Missing documentation of FlinkKafkaConsumer.setStartFromEarliest behavior when a partition offset becomes out of range
                 Key: FLINK-18865
                 URL: https://issues.apache.org/jira/browse/FLINK-18865
             Project: Flink
          Issue Type: Improvement
          Components: Documentation
    Affects Versions: 1.12.0
            Reporter: Vladimirs Kotovs


Flink documentation is missing a clear explanation of the Kafka consumer behavior configured with "setStartFromEarliest()" when a partition offset becomes out of range.

We see the following log messages when running Flink application with Kafka topics with a configured retention period and Kafka consumer configured with "setStartFromEarliest()".
{code:java}
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:999)
org.apache.kafka.clients.consumer.internals.Fetcher
[Consumer clientId=consumer-3, groupId=some-consumer] Fetch offset 12956961 is out of range for partition some_topic-80, resetting offset ...]{code}
Affected partition offset is being reset according to "auto.offset.reset" setting in the properties with "latest" as a default value that may contradict expectations when using "setStartFromEarliest()" configuration method.

Flink documentation should provide a clear explanation for this behavior.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)