[jira] [Created] (FLINK-5013) Flink Kinesis connector doesn't work on old EMR versions

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-5013) Flink Kinesis connector doesn't work on old EMR versions

Shang Yuanchun (Jira)
Robert Metzger created FLINK-5013:
-------------------------------------

             Summary: Flink Kinesis connector doesn't work on old EMR versions
                 Key: FLINK-5013
                 URL: https://issues.apache.org/jira/browse/FLINK-5013
             Project: Flink
          Issue Type: Bug
          Components: Kinesis Connector
            Reporter: Robert Metzger


A user reported on the mailing list that our Kinesis connector doesn't work with EMR 4.4.0: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kinesis-Connector-Dependency-Problems-td9790.html

The problem seems to be that Flink is loading older libraries from the "YARN container classpath", which on EMR contains the default Amazon libraries.

We should try to shade kinesis and its amazon dependencies into a different namespace.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)