Stephan Ewen created FLINK-5590:
-----------------------------------
Summary: Create a proper internal state hierarchy
Key: FLINK-5590
URL:
https://issues.apache.org/jira/browse/FLINK-5590 Project: Flink
Issue Type: Improvement
Components: State Backends, Checkpointing
Affects Versions: 1.2.0
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 1.3.0
Currently, the state interfaces (like {{ListState}}, {{ValueState}}, {{ReducingState}}) are very sparse and contain only methods exposed to the users. That makes sense to keep the public stable API minimal
At the same time, the runtime needs more methods for its internal interaction with state, such as:
- setting namespaces
- accessing raw values
- merging namespaces
These are currently realized by re-creating or re-obtaining the state objects from the KeyedStateBackend. That method causes quite an overhead for each access to the state
The KeyedStateBackend tries to do some tricks to reduce that overhead, but does it only partially and induces other overhead in the course.
The root cause of all these issues is a problem in the design: There is no proper "internal state abstraction" in a similar way as there is an external state abstraction (the public state API).
We should add a similar hierarchy of states for the internal methods. It would look like in the example below:
{code}
* State
* |
* +-------------------InternalKvState
* | |
* MergingState |
* | |
* +-----------------InternalMergingState
* | |
* +--------+------+ |
* | | |
* ReducingState ListState +-----+-----------------+
* | | | |
* +-----------+ +----------- -----------------InternalListState
* | |
* +---------InternalReducingState
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
