[jira] [Created] (FLINK-13291) Cannot Privide AWS Direct Connect ENDPOINT

[Shang Yuanchun (Jira)]

Shakir Poolakkal Mukkath created FLINK-13291:
------------------------------------------------

             Summary: Cannot Privide AWS Direct Connect ENDPOINT
                 Key: FLINK-13291
                 URL: https://issues.apache.org/jira/browse/FLINK-13291
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kinesis
    Affects Versions: 1.8.1, 1.8.0
            Reporter: Shakir Poolakkal Mukkath
             Fix For: 1.10.0


The issue is at two places,

First, *KinesisConfigUtil.validateConsumerConfiguration((Properties config))* limit providing both REGION and ENDPOINT in properties. 

Second, *AWSUtil.createKinesisClient(Properties configProps, ClientConfiguration awsClientConfig)* is passing REGION as null when ENDPOINT is provided. 

The above cases will not work when an AWS DIRECT CONNECT ENDPOINT is used. A sample direct connect endpoint for east region is _kinesis-ae1.hdw.r53.feap.pv_  So this does not follow the convention of kinesis.us-east-1.amazonaws.com where the first word after _"-" is_ region_._
And using the above endpoint will results error  __  

_org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.model.AmazonKinesisException: Credential should be scoped to a valid region, not 'ae1'. (Service: AmazonKinesis; Status Code: 400; Error Code: InvalidSignatureException; Request ID: ee678308-0c88-ca77-bbc0-54322258672d)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1632)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2388)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2364)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.executeListShards(AmazonKinesisClient.java:1337)_
 _at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.listShards(AmazonKinesisClient.java:1312)_
 _at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.listShards(KinesisProxy.java:442)_
 _at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardsOfStream(KinesisProxy.java:392)_
 _at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardList(KinesisProxy.java:282)_
 _at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.discoverNewShardsToSubscribe(KinesisDataFetcher.java:550)_
 _at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:270)_
 _at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)_
 _at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)_
 _at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)_
 _at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)_
 _at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)_
 _at java.lang.Thread.run(Thread.java:748)_

 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)