[jira] [Created] (FLINK-22918) StreamingFileSink does not commit partition when no message is sent between checkpoints

[Shang Yuanchun (Jira)]

lihe ma created FLINK-22918:
-------------------------------

             Summary: StreamingFileSink does not commit partition when no message is sent between checkpoints
                 Key: FLINK-22918
                 URL: https://issues.apache.org/jira/browse/FLINK-22918
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem
    Affects Versions: 1.11.3
            Reporter: lihe ma


FileSink extract the partitions from messages , then send partition list to committer. In  dynamic partition case, if one partition contains no data  between two checkpoints. then the partition wont commit.

like this:
date=0101/hour=01/key=a/part1-file  (write at 01-01 01:20:00)

if there is no data with key=a between 01:20 and 02:10 (a time when watermark passed) , the committer will not receive this partition and it will never commit.
 

  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)