Kinesis Connector RuntimeException

[PoolakkalMukkath, Shakir]

Hi Flink Team,



Getting the below error when using Kinesis Connector, FlinkKinesisProducer. Can you help ?



I am using Flink 1.8 version Scala 2.11, both Flows and cluster. And this is on a standalone zookeeper ha cluster.





java.lang.RuntimeException: Could not copy native binaries to temp directory /tmp/amazon-kinesis-producer-native-binaries

       at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:907)

       at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.<init>(KinesisProducer.java:245)

       at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.getKinesisProducer(FlinkKinesisProducer.java:344)

       at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.open(FlinkKinesisProducer.java:212)

       at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)

       at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)

       at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)

       at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)

       at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)

       at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)

       at java.lang.Thread.run(Thread.java:748)

Caused by: java.io.IOException: Permission denied

       at java.io.UnixFileSystem.createFileExclusively(Native Method)

       at java.io.File.createTempFile(File.java:2024)

       at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.HashedFileCopier.copyFileFrom(HashedFileCopier.java:46)

       at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:893)

       ... 10 more

Thanks,
Shakir

[Thomas Weise]

The error message says it all: Permission denied:
tmp/amazon-kinesis-producer-native-binaries

The producer as such is working, we run it on Flink 1.8 at scale.

Thomas

On Wed, May 22, 2019 at 8:36 AM PoolakkalMukkath, Shakir <
[hidden email]> wrote:

> Hi Flink Team,
>
>
>
> Getting the below error when using Kinesis Connector,
> FlinkKinesisProducer. Can you help ?
>
>
>
> I am using Flink 1.8 version Scala 2.11, both Flows and cluster. And this
> is on a standalone zookeeper ha cluster.
>
>
>
>
>
> java.lang.RuntimeException: Could not copy native binaries to temp
> directory /tmp/amazon-kinesis-producer-native-binaries
>
>        at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:907)
>
>        at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.<init>(KinesisProducer.java:245)
>
>        at
> org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.getKinesisProducer(FlinkKinesisProducer.java:344)
>
>        at
> org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.open(FlinkKinesisProducer.java:212)
>
>        at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
>
>        at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
>
>        at
> org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
>
>        at
> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
>
>        at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
>
>        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
>
>        at java.lang.Thread.run(Thread.java:748)
>
> Caused by: java.io.IOException: Permission denied
>
>        at java.io.UnixFileSystem.createFileExclusively(Native Method)
>
>        at java.io.File.createTempFile(File.java:2024)
>
>        at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.HashedFileCopier.copyFileFrom(HashedFileCopier.java:46)
>
>        at
> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:893)
>
>        ... 10 more
>
> Thanks,
> Shakir
>