[jira] [Created] (FLINK-18479) can not commit partition when set partition time

[Shang Yuanchun (Jira)]

Jun Zhang created FLINK-18479:
---------------------------------

             Summary: can not commit partition when set partition time
                 Key: FLINK-18479
                 URL: https://issues.apache.org/jira/browse/FLINK-18479
             Project: Flink
          Issue Type: Bug
          Components: FileSystems
    Affects Versions: 1.11.0
            Reporter: Jun Zhang
             Fix For: 1.11.1


when we write streaming data to filesystem, and select the 'partition time' , we can not commit the partition when write finished.

 
{code:java}

LocalDateTime partTime = extractor.extract(
      partitionKeys, extractPartitionValues(new Path(partition)));
if (watermark > toMills(partTime) + commitDelay) {
   needCommit.add(partition);
   iter.remove();
}
{code}
when we set a not UTC zone, and submit the partition, the method 'toMills' will get the UTC mills ，for example ，in UTC/GMT+08:00 ，the watermark will less than the toMills , so we can not commit the partition forever.

if we use a local time , not utc , it will be ok in UTC zone and other zone.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)