[jira] [Created] (FLINK-12008) Support read a whole directory or multiple input data files for read apis of HadoopInputs

[Shang Yuanchun (Jira)]

vinoyang created FLINK-12008:
--------------------------------

             Summary: Support read a whole directory or multiple input data files for read apis of HadoopInputs
                 Key: FLINK-12008
                 URL: https://issues.apache.org/jira/browse/FLINK-12008
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Hadoop Compatibility
            Reporter: vinoyang
            Assignee: vinoyang


Currently, the read APIs provided by {{HadoopInputs}} only can read one path. I think it's not strong enough. We should support read a whole directory or multiple input files.

Hadoop provides {{org.apache.hadoop.mapred.FileInputFormat.setInputPaths()}} to support this requirement. 

Spark's {{sequenceFile}} API calls [this API|[https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L1049].] 

Flink calls {{org.apache.hadoop.mapred.FileInputFormat.addInputPath}} which only  supports one path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)