[jira] [Created] (FLINK-13376) FileInputFormat can not be used in blink-planner

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-13376) FileInputFormat can not be used in blink-planner

Shang Yuanchun (Jira)
Jingsong Lee created FLINK-13376:
------------------------------------

             Summary: FileInputFormat can not be used in blink-planner
                 Key: FLINK-13376
                 URL: https://issues.apache.org/jira/browse/FLINK-13376
             Project: Flink
          Issue Type: Sub-task
          Components: Connectors / FileSystem, Runtime / Task, Table SQL / Runtime
            Reporter: Jingsong Lee
             Fix For: 1.9.0, 1.10.0


FileInputFormat use ContinuousFileReaderOperator in StreamExecutionEnvironment, and ContinuousFileReaderOperator join the async thread in close, So before closing, the data is incomplete.

But batch operators use endInput to do some work, this lead to incomplete records in endInput, and lead to wrong results.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)