[jira] [Created] (FLINK-16112) KafkaDeserializationSchema#isEndOfStream is ill-defined

[Shang Yuanchun (Jira)]

Tzu-Li (Gordon) Tai created FLINK-16112:
-------------------------------------------

             Summary: KafkaDeserializationSchema#isEndOfStream is ill-defined
                 Key: FLINK-16112
                 URL: https://issues.apache.org/jira/browse/FLINK-16112
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Kafka
            Reporter: Tzu-Li (Gordon) Tai


Motivated by this email thread: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/KafkaFetcher-closed-before-end-of-stream-is-received-for-all-partitions-td32827.html

In general, the {{isEndOfStream}} method is ill-defined. An possible improvement is to redefine the semantics and method signature, to:

{code}
boolean isEndOfPartition(T nextElement, String topic, int partition);
{code}

so that it indicates end of a Kafka partition. With this, the Kafka consumer is able to properly define end-of-subtask to be when all of its assigned partitions have signaled EOF.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)