Stephan Ewen created FLINK-3514:
-----------------------------------
Summary: Add support for slowly changing streaming broadcast variables
Key: FLINK-3514
URL:
https://issues.apache.org/jira/browse/FLINK-3514 Project: Flink
Issue Type: New Feature
Components: Streaming
Reporter: Stephan Ewen
With "slow changing inputs", I mean auxiliary data sets that change over time. Think about dictionaries, set of rules, etc that are updates at slow rates and then applied onto another stream.
They are a bit like broadcast variables in the DataSet API, but not constant over the entire lifetime of the job.
One can currently approximate this via connected streams:
{code}
DataStream<Rule> rules = ...
DataStream<Event> events = ...
rules.broadcast().connect(events).flatMap(...);
{code}
The disadvantage is that one has no control that the broadcasted "rules" input is materialized in the function before the "events" are delivered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
