[jira] [Created] (FLINK-16289) Missing serialVersionUID blocks running in Kubernetes.

[Shang Yuanchun (Jira)]

Niels Basjes created FLINK-16289:
------------------------------------

             Summary: Missing serialVersionUID blocks running in Kubernetes.
                 Key: FLINK-16289
                 URL: https://issues.apache.org/jira/browse/FLINK-16289
             Project: Flink
          Issue Type: Bug
          Components: Deployment / Kubernetes, Table SQL / Runtime
            Reporter: Niels Basjes


I have written a Flink 1.10 job that reads a file (using the S3 Presto client), applies an SQL statement on that (with [Yauaa|https://yauaa.basjes.nl/UDF-ApacheFlinkTable.html] as a UDF) and then tries to write it to ElasticSearch.

The problem is that when submit this into the native Kubernetes cluster I get this exception (full stack trace below):

{code:java}java.io.InvalidClassException: org.apache.flink.table.codegen.GeneratedAggregationsFunction; local class incompatible: stream classdesc serialVersionUID = 1538379512770243128, local class serialVersionUID = -5485442333060060467 {code}

According to [this stack overflow answer|https://stackoverflow.com/a/10378907/114196] page this error stems from the JVM automatically generating a serialVersionUID in case it is missing, which can be JDK/JRE version dependent.

On my local machine (Ubuntu 16.04 LTS) I use the openjdk-9-jdk.

Apparently Flink docker image uses JRE 1.8
{code}
....KubernetesTaskExecutorRunner  - --------------------------------------------------------------------------------
....KubernetesTaskExecutorRunner  -  Starting Kubernetes TaskExecutor runner (Version: 1.10.0, Rev:aa4eb8f, Date:07.02.2020 @ 19:18:19 CET)
....KubernetesTaskExecutorRunner  -  OS current user: root
....KubernetesTaskExecutorRunner  -  Current Hadoop/Kerberos user: <no hadoop dependency found>
....KubernetesTaskExecutorRunner  -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.242-b08
{code}

I have tried doing the same with JDK 1.8 on my machine but that still does not work (apperently there is still a too big a difference in the Java versions).

When I remove the "group by" (i.e. the Aggregation) from my SQL statement this passes (and right now fails on missing dependencies ... different problem).
 
{code:java}
2020-02-26 11:40:48,303 INFO org.apache.flink.runtime.taskmanager.Task - groupBy: (useragent, DeviceClass, AgentNameVersionMajor), window: (TumblingGroupWindow('w$, 'EventTime, 3600000.millis)), select: (useragent, DeviceClass, AgentNameVersionMajor, SUM(clicks) AS clicks, SUM(visitors) AS visitors, start('w$) AS w$start, end('w$) AS w$end, rowtime('w$) AS w$rowtime, proctime('w$) AS w$proctime) -> select: (w$start AS wStart, useragent, DeviceClass, AgentNameVersionMajor, clicks, visitors) -> to: Row -> Sink: Unnamed (11/32) (db5cb408a1b286e705a2e3e30ac8131e) switched from RUNNING to FAILED.
 org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user function.
 at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:269)
 at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:115)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:433)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461)
 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
 at java.lang.Thread.run(Thread.java:748)
 Caused by: java.io.InvalidClassException: org.apache.flink.table.codegen.GeneratedAggregationsFunction; local class incompatible: stream classdesc serialVersionUID = 1538379512770243128, local class serialVersionUID = -5485442333060060467
 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699)
 at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1940)
 at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1806)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2097)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1625)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2342)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2266)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2124)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1625)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2342)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2266)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2124)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1625)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2342)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2266)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2124)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1625)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2342)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2266)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2124)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1625)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:465)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:423)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:576)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:562)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:550)
 at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:511)
 at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:254)
 ... 6 more
 {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)