[jira] [Created] (FLINK-11882) Introduce BytesHashMap to batch hash agg

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-11882) Introduce BytesHashMap to batch hash agg

Shang Yuanchun (Jira)
Jingsong Lee created FLINK-11882:
------------------------------------

             Summary: Introduce BytesHashMap to batch hash agg
                 Key: FLINK-11882
                 URL: https://issues.apache.org/jira/browse/FLINK-11882
             Project: Flink
          Issue Type: New Feature
          Components: Runtime / Operators
            Reporter: Jingsong Lee
            Assignee: Jingsong Lee


Introduce bytes based hash table.
It can be used for performing aggregations where the aggregated values are fixed-width.
Because the data is stored in continuous memory, AggBuffer of variable length cannot be applied to this HashMap. The KeyValue form in hash map is designed to reduce the cost of key fetching in lookup.

Add a test to do a complete hash agg. When HashMap has enough memory, pure hash AGG is performed; when memory is insufficient, it degenerates into sort agg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)