[jira] [Created] (FLINK-17593) Support arbitrary recovery mechanism for PartFileWriter

[Shang Yuanchun (Jira)]

Yun Gao created FLINK-17593:
-------------------------------

             Summary: Support arbitrary recovery mechanism for PartFileWriter
                 Key: FLINK-17593
                 URL: https://issues.apache.org/jira/browse/FLINK-17593
             Project: Flink
          Issue Type: New Feature
          Components: Connectors / FileSystem
            Reporter: Yun Gao
             Fix For: 1.11.0


Currently Bucket relies directly on _RecoverableOutputStream_ provided by FileSystem to achieve snapshotting and recovery the in-progress part file for all the PartFileWriter implementations. This would require that the PartFileWriter must be based on the OutputStream.

To support the path-based PartFileWriter required by the Hive Sink, we will first need to abstract the snapshotting mechanism of the PartFileWriter and make RecoverableOutputStream to be one type of implementation, thus we could decouple PartFileWriter with the output streams.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)