[jira] [Created] (FLINK-13326) Support asynchronous writing to raw operator (and raw keyed) state

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-13326) Support asynchronous writing to raw operator (and raw keyed) state

Shang Yuanchun (Jira)
Igal Shilman created FLINK-13326:
------------------------------------

             Summary: Support asynchronous writing to raw operator (and raw keyed) state
                 Key: FLINK-13326
                 URL: https://issues.apache.org/jira/browse/FLINK-13326
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / State Backends
            Reporter: Igal Shilman
             Fix For: 1.10.0


Currently in AbstractStreamOperator, it is expected that the implementors of snapshotState, that use either raw operator state or raw keyed operator state, would complete the snapshotting of their state in the synchronous part of the checkpoint (before the snapshotState method complete)
But sometimes it is desirable to snapshot custom data structure asynchronously (i.e. out side of the snapshotState method) in a similar way that the other state backends allow.

This issue is about adding the ability to complete the raw states outside of the snapshotState method, and hence effectively support async snapshots for raw states.

Flink snapshotting logic already supports splitting the snapshot to sync and async parts for the various state backends, therefore the expected changes mainly to reuse that logic for the raw keyed/ raw operator states.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)