[jira] [Created] (FLINK-21320) Stream from FlinkKafkaShuffle.persistentKeyBy does not end if upstream ends

[Shang Yuanchun (Jira)]

Kezhu Wang created FLINK-21320:
----------------------------------

             Summary: Stream from FlinkKafkaShuffle.persistentKeyBy does not end if upstream ends
                 Key: FLINK-21320
                 URL: https://issues.apache.org/jira/browse/FLINK-21320
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
    Affects Versions: 1.13.0
            Reporter: Kezhu Wang


I pushed a [test case|https://github.com/kezhuw/flink/commit/5793083c370241717c485b5663a10f72884e028b] in my repository for evaluation, that test will hang after all input consumed.

I think we could do something(such as setting {{running}} to {{false}} upon {{Watermark.MAX_WATERMARK}}) in {{KafkaShuffleFetcher.partitionConsumerRecordsHandler}} to end kafka consumer.

cc [~ym]  [~sewen] [~AHeise] [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)