[jira] [Created] (FLINK-19481) Add support for a flink native GCS FileSystem

[Shang Yuanchun (Jira)]

Ben Augarten created FLINK-19481:
------------------------------------

             Summary: Add support for a flink native GCS FileSystem
                 Key: FLINK-19481
                 URL: https://issues.apache.org/jira/browse/FLINK-19481
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem, FileSystems
    Affects Versions: 1.12.0
            Reporter: Ben Augarten
             Fix For: 1.12.0


Currently, GCS is supported but only by using the hadoop connector[1]

 

The objective of this improvement is to add support for checkpointing to Google Cloud Storage with the Flink File System,

 

This would allow the `gs://` scheme to be used for savepointing and checkpointing. Long term, it would be nice if we could use the GCS FileSystem as a source and sink in flink jobs as well. 

 

Long term, I hope that implementing a flink native GCS FileSystem will simplify usage of GCS because the hadoop FileSystem ends up bringing in many unshaded dependencies.

 

[1] [https://github.com/GoogleCloudDataproc/hadoop-connectors|https://github.com/GoogleCloudDataproc/hadoop-connectors)]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)