[jira] [Created] (FLINK-16589) Flink Table SQL fails/crashes with big queries with lots of fields

[Shang Yuanchun (Jira)]

Viet Pham created FLINK-16589:
---------------------------------

             Summary: Flink Table SQL fails/crashes with big queries with lots of fields
                 Key: FLINK-16589
                 URL: https://issues.apache.org/jira/browse/FLINK-16589
             Project: Flink
          Issue Type: Bug
          Components: Table SQL / API
    Affects Versions: 1.10.0
            Reporter: Viet Pham


Hi,

My use case is a streaming application with a few streaming tables.

I was trying to build a SELECT query (and registering it as a temporary view) with about 200 fields/expressions out of another streaming table. The application is successfully submitted to Flink cluster. However the worker processes keep crashing, with the exception as quoted below. 

It clearly mentioned in the log that this is a bug, so I fire this ticket. By the way, if I lower the number of fields down to 100 then it works nicely.

Please advice.

Thanks a lot for all the efforts bring Flink up. It is really amazing!
{code:java}
java.lang.RuntimeException: Could not instantiate generated class 'GroupAggsHandler$9687'    at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:57)    at org.apache.flink.table.runtime.operators.aggregate.MiniBatchGroupAggFunction.open(MiniBatchGroupAggFunction.java:136)    at org.apache.flink.table.runtime.operators.bundle.AbstractMapBundleOperator.open(AbstractMapBundleOperator.java:84)    at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeStateAndOpen(StreamTask.java:1007)    at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:454)    at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:94)    at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:449)    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461)    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)    at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.    at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:68)    at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:78)    at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:52)    ... 10 moreCaused by: org.apache.flink.shaded.guava18.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)    at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:66)    ... 12 moreCaused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.    at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:81)    at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$1(CompileUtils.java:66)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)    at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)    ... 15 moreCaused by: org.codehaus.janino.InternalCompilerException: Compiling "GroupAggsHandler$9687": Code of method "retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class "GroupAggsHandler$9687" grows beyond 64 KB    at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382)    at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)    at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)    at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)    at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)    at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)    at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)    at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)    ... 21 moreCaused by: org.codehaus.janino.InternalCompilerException: Code of method "retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class "GroupAggsHandler$9687" grows beyond 64 KB    at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009)    at org.codehaus.janino.CodeContext.write(CodeContext.java:901)    at org.codehaus.janino.CodeContext.writeShort(CodeContext.java:1026)    at org.codehaus.janino.UnitCompiler.writeConstantLongInfo(UnitCompiler.java:12274)    at org.codehaus.janino.UnitCompiler.pushConstant(UnitCompiler.java:10679)    at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4936)    at org.codehaus.janino.UnitCompiler.access$8400(UnitCompiler.java:215)    at org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4414)    at org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4394)    at org.codehaus.janino.Java$UnaryOperation.accept(Java.java:4719)    at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394)    at org.codehaus.janino.UnitCompiler.fakeCompile(UnitCompiler.java:3719)    at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5569)    at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2580)    at org.codehaus.janino.UnitCompiler.access$2700(UnitCompiler.java:215)    at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1503)    at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1487)    at org.codehaus.janino.Java$LocalVariableDeclarationStatement.accept(Java.java:3511)    at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)    at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)    at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)    at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)    at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)    at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)    at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)    at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)    at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)    at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)    at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)    at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)    at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)    ... 28 more
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)