[jira] [Created] (FLINK-4940) Add Support for Broadcast/Global State

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-4940) Add Support for Broadcast/Global State

Shang Yuanchun (Jira)
Aljoscha Krettek created FLINK-4940:
---------------------------------------

             Summary: Add Support for Broadcast/Global State
                 Key: FLINK-4940
                 URL: https://issues.apache.org/jira/browse/FLINK-4940
             Project: Flink
          Issue Type: Sub-task
          Components: Streaming
            Reporter: Aljoscha Krettek
             Fix For: 1.2.0


We need this to support the pattern where you have one input of a two-input operator keyed and the other input carries messages that should be broadcast to all operator instances. This other (side) input can contain configuration updates, models, or what have you that are relevant for all elements in the other stream.

We should add something akin to {{KeyedStateBackend}} where state is kept by operator. When checkpointing, we only checkpoint the state of the first operator instance because we assume the state to be the same across all instances. When restoring we restore the one snapshot to all operator instances.

Current thinking is that this global/broadcast state backend should allow access by key and possibly also iteration over all stored keys.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)