[jira] [Created] (FLINK-15172) optimize the operator algorithm to lazily allocate memory

[Shang Yuanchun (Jira)]

Jingsong Lee created FLINK-15172:
------------------------------------

             Summary: optimize the operator algorithm to lazily allocate memory
                 Key: FLINK-15172
                 URL: https://issues.apache.org/jira/browse/FLINK-15172
             Project: Flink
          Issue Type: Sub-task
          Components: Table SQL / Runtime
            Reporter: Jingsong Lee
             Fix For: 1.11.0


Now after FLINK-14063 , operators will get all manage memory of TaskManager,  The cost of over allocate memory is very high, lead to performance regression of small batch sql jobs:
 * Allocate memory will have the cost of memory management algorithm.
 * Allocate memory will have the cost of memory initialization, will set all memory to zero. And this initialization will require the operating system to actually allocate physical memory.

 * Over allocate memory will squash the file cache too.

We can optimize the operator algorithm, apply lazy allocation, and avoid meaningless memory allocation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)