[jira] [Created] (FLINK-17359) Entropy key is not resolved if flink-s3-fs-hadoop is added as a plugin

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-17359) Entropy key is not resolved if flink-s3-fs-hadoop is added as a plugin

Shang Yuanchun (Jira)
Prem Santosh created FLINK-17359:
------------------------------------

             Summary: Entropy key is not resolved if flink-s3-fs-hadoop is added as a plugin
                 Key: FLINK-17359
                 URL: https://issues.apache.org/jira/browse/FLINK-17359
             Project: Flink
          Issue Type: Bug
            Reporter: Prem Santosh


Using flink 1.10

I added theĀ flink-s3-fs-hadoop jar in plugins dir but I am seeing the checkpoints paths like {{s3://my_app/__ENTROPY__/app_name-staging/flink/checkpoints/e10f47968ae74934bd833108d2272419/chk-3071}} which means the entropy injection key is not being resolved. After some debugging I found that in the [EntropyInjector|https://github.com/apache/flink/blob/release-1.10.0/flink-core/src/main/java/org/apache/flink/core/fs/EntropyInjector.java#L97] we check if the given fileSystem is of type {{SafetyNetWrapperFileSystem}} and if so we check if the delegate is of type {{EntropyInjectingFileSystem}} but don't check for {{[ClassLoaderFixingFileSystem |https://github.com/apache/flink/blob/release-1.10.0/flink-core/src/main/java/org/apache/flink/core/fs/PluginFileSystemFactory.java#L65]}} which would be the type if S3 file system dependencies are added as a plugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)