[jira] [Created] (FLINK-20491) Support Broadcast State in BATCH execution mode

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-20491) Support Broadcast State in BATCH execution mode

Shang Yuanchun (Jira)
Aljoscha Krettek created FLINK-20491:
----------------------------------------

             Summary: Support Broadcast State in BATCH execution mode
                 Key: FLINK-20491
                 URL: https://issues.apache.org/jira/browse/FLINK-20491
             Project: Flink
          Issue Type: Improvement
          Components: API / DataStream
            Reporter: Aljoscha Krettek
            Assignee: Aljoscha Krettek


Right now, we don't support {{DataStream.connect(BroadcastStream)}} in {{BATCH}} execution mode. I believe we can add support for this with not too much work.

The key insight is that we can process the broadcast side before the non-broadcast side. Initially, we were shying away from this because of concerns about {{ctx.applyToKeyedState()}} which allows the broadcast side of the user function to access/iterate over state from the keyed side. We thought that we couldn't support this. However, since we know that we process the broadcast side first we know that the keyed side will always be empty when doing so. We can thus just make this "keyed iteration" call a no-op, instead of throwing an exception as we do now.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)