[jira] [Created] (FLINK-13768) Update documentation regarding `path style access` for S3 filesystem implementations

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-13768) Update documentation regarding `path style access` for S3 filesystem implementations

Shang Yuanchun (Jira)
Achyuth Narayan Samudrala created FLINK-13768:
-------------------------------------------------

             Summary: Update documentation regarding `path style access` for S3 filesystem implementations
                 Key: FLINK-13768
                 URL: https://issues.apache.org/jira/browse/FLINK-13768
             Project: Flink
          Issue Type: New Feature
          Components: Documentation
            Reporter: Achyuth Narayan Samudrala


The documentation related to various properties that can be provided for the s3 sink is not very informative. According to the code in flink-s3-fs-base/flink-s3-fs-hadoop, any property specified as s3.<some-property> is transformed to fs.s3a.<some-property>.

 

When interacting with s3 compatible file systems such as CEPH/Minio, default configuration properties might not be sufficient. One such property is the fs.s3a.path.style.access. This property enables different modes of access to the s3 buckets. By default if this property is not set, virtual host style addressing is used. The documentation should mention how this property can be passed on as a flink conf property.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)