[jira] [Created] (FLINK-19363) Code of split method grows beyond 64 KB

[Shang Yuanchun (Jira)]

hailong wang created FLINK-19363:
------------------------------------

             Summary: Code of split method grows beyond 64 KB
                 Key: FLINK-19363
                 URL: https://issues.apache.org/jira/browse/FLINK-19363
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / Planner
    Affects Versions: 1.11.0
            Reporter: hailong wang
             Fix For: 1.12.0


For now, If the total size beyond 64kb, we will split one field into one field. But when sql is complicated, one field size will grow 64kb.
{code:java}
Caused by: org.codehaus.janino.InternalCompilerException: Compiling "StreamExecCalc$4436": Code of method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436" grows beyond 64 KBCaused by: org.codehaus.janino.InternalCompilerException: Compiling "StreamExecCalc$4436": Code of method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436" grows beyond 64 KB at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237) at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78) ... 24 moreCaused by: org.codehaus.janino.InternalCompilerException: Code of method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436" grows beyond 64 KB at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009) at org.codehaus.janino.CodeContext.write(CodeContext.java:918) at org.codehaus.janino.CodeContext.writeBranch(CodeContext.java:1038)
{code}
By this case, we should split one field to multi functions.

My solution is as follow,
{code:java}
override def visitCall(call: RexCall): GeneratedExpression = {

  val resultType = FlinkTypeFactory.toLogicalType(call.getType)

  // convert operands and help giving untyped NULL literals a type
  val operands = call.getOperands.zipWithIndex.map {

    // this helps e.g. for AS(null)
    // we might need to extend this logic in case some rules do not create typed NULLs
    case (operandLiteral: RexLiteral, 0) if
    operandLiteral.getType.getSqlTypeName == SqlTypeName.NULL &&
      call.getOperator.getReturnTypeInference == ReturnTypes.ARG0 =>
      generateNullLiteral(resultType, ctx.nullCheck)

    case (o@_, _) => o.accept(this)
  }
  // when function is too large, we split it.
  val exp = generateCallExpression(ctx, call.getOperator, operands, resultType)
  if (exp.code.length > maxGeneratedCodeLength) {
    ctx.setCodeSplit()
    val methodName = newName("callSplit")
    val resultTypeTerm = boxedTypeTermForType(exp.resultType)
    val callResultTerm = ctx.addReusableLocalVariable(resultTypeTerm, "callSplitResultTerm")
    val callNullResultTerm = ctx.addReusableLocalVariable("boolean", "callSplitNullResultTerm")

    val method =
      s"""
         |private void $methodName() throws Exception {
         |  ${exp.code}
         |  ${callResultTerm}= ${exp.resultTerm};
         |  ${callNullResultTerm}= ${exp.nullTerm};
         |}
         |""".stripMargin
    val methodCode =
      s"""
         |$methodName();
         |""".stripMargin
    ctx.addReusableMember(method)
    return new GeneratedExpression(callResultTerm, callNullResultTerm, methodCode, exp.resultType)
  }
  exp
}
{code}
When function beyong maxGeneratedCodeLength, we split it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)