Kafka Consumer Deserializer Exception on application mode

[han guoguo]

Hi,
 Kafka source may has some issues on application mode

when i run it with application mode on  flink 1.11.2 it can't startup
the detail Excetion is:
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:789)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:643)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
    at org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
    at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
    at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
    at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer
    at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:688)
    ... 15 more

I has create open it on

https://issues.apache.org/jira/browse/FLINK-20617

[Aljoscha Krettek-2]

I believe this is caused by dependency conflicts/mismatch. I also
commented this on the Jira issue.

Best,
Aljoscha

On 16.12.20 07:39, han guoguo wrote:

> Hi,
>   Kafka source may has some issues on application mode
>
> when i run it with application mode on  flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
>      at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:789)
>      at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:643)
>      at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
>      at org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
>      at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
>      at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
>      at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
>      at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
>      at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
>      at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
>      at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
>      at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
>      at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
>      at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
>      at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
>      at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer
>      at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
>      at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:688)
>      ... 15 more
>
> I has create open it on
>
> https://issues.apache.org/jira/browse/FLINK-20617
>