Chesnay Schepler created FLINK-13602:
----------------------------------------
Summary: Presto S3 filesystem effectively does not support credential providers
Key: FLINK-13602
URL:
https://issues.apache.org/jira/browse/FLINK-13602 Project: Flink
Issue Type: Bug
Components: Connectors / FileSystem
Affects Versions: 1.9.0
Reporter: Chesnay Schepler
To provide credentials to S3 users may configure a credentials provider. For providers from amazon (which are relocated) we allow users to configure the original class name, and relocate it manually in the S3 filesystem factories.
However, none of the amazon provided credential providers can be used with the Presto filesystem, since it _additionally_ requires them to have a constructor accepting a hadoop configuration. (
https://prestodb.github.io/docs/current/connector/hive.html#amazon-s3-configuration)
{{hadoop-aws}} _does_ include a number of credential providers that have this constructor, however these use configuration keys that aren't mirrored from the flink config. (they expect {{fs.s3a}} as a key-prefix), not to mention that users would have to configure the relocated class (since the S3 factory only manually relocates amazon classes).
Finally, a custom implementation of the credentials provider can effectively be ruled out since they too would have to be implemented against the relocated amazon/hadoop classes, which we can't really expect users to do.
In summary, amazon providers aren't working since they don't have a constructor that presto requires, hadoop providers don't work since we don't mirror the required configuration keys, and custom providers are unreasonable as they'd have to be implemented against relocated classes.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
Locked
