[jira] [Created] (FLINK-18303) Filesystem connector doesn't flush part files after rolling interval

[Shang Yuanchun (Jira)]

Jark Wu created FLINK-18303:
-------------------------------

             Summary: Filesystem connector doesn't flush part files after rolling interval
                 Key: FLINK-18303
                 URL: https://issues.apache.org/jira/browse/FLINK-18303
             Project: Flink
          Issue Type: Bug
          Components: Connectors / FileSystem, Table SQL / Ecosystem
            Reporter: Jark Wu
            Assignee: Jark Wu
             Fix For: 1.11.0


I have set "execution.checkpointing.interval" to "5s" and "sink.rolling-policy.time-interval" to "2s". However, it still take 60 seconds to see the first part file.

This can be reproduced by the following code in SQL CLI:

{code:sql}
CREATE TABLE CsvTable (
  event_timestamp STRING,
  `user` STRING,
  message STRING,
  duplicate_count BIGINT,
  constant STRING
) WITH (
  'connector' = 'filesystem',
  'path' = '$RESULT',
  'format' = 'csv',
  'sink.rolling-policy.time-interval' = '2s'
);

INSERT INTO CsvTable -- read from Kafka Avro, and write into Filesystem Csv
SELECT AvroTable.*, RegReplace('Test constant folding.', 'Test', 'Success') AS constant
FROM AvroTable;
{code}

This is found when I migrate SQLClientKafkaITCase to use DDL (FLINK-18086).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)