Chesnay Schepler created FLINK-20804:
----------------------------------------
Summary: Add SwiftFS end-to-end test
Key: FLINK-20804
URL:
https://issues.apache.org/jira/browse/FLINK-20804 Project: Flink
Issue Type: Improvement
Components: FileSystems, Tests
Reporter: Chesnay Schepler
Our swift filesystem is closely aligned with the S3 filesystems in that it follows the same packaging scheme as flink-fs-hadoop-shaded (used by the S3 filesystems), and thus ideally is kept in sync with these filesystems, for example in regards to dependencies.
As it stands, this is not the case. While the S3 modules work against Hadoop 3, swift is still sitting on 2.8.1 . Presumable this difference is due to us having no infrastructure for actually testing our swift filesystem.
Unfortunately it appears that Switft is rather annoying to setup, but I was able find a [project|
https://github.com/swiftstack/docker-swift] that tries to provide a docker image containing everything you need. Maybe we could use this to bootstrap some end-to-end-test, allowing us to finally touch this module again without fear of completely breaking it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
