Tzu-Li (Gordon) Tai created FLINK-18316:
-------------------------------------------
Summary: Add a dynamic state registration primitive for Stateful Functions
Key: FLINK-18316
URL:
https://issues.apache.org/jira/browse/FLINK-18316 Project: Flink
Issue Type: New Feature
Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
Currently, using the {{PersistedValue}} / {{PersistedTable}} / {{PersistedAppendingBuffer}} primitives, the user can only eagerly define states prior to function instance activation using the {{Persisted}} field annotation.
We propose to add a primitive that allows them to register states dynamically after activation (i.e. during runtime), along the lines of:
{code}
public MyStateFn implements StatefulFunction {
@Persisted
private final PersistedStateProvider provider = new PersistedStateProvider();
public MyStateFn() {
PersistedValue<String> valueState = provider.getValue(...);
}
void invoke(Object input) {
PersistedValue<String> anotherValueState = provider.getValue(...);
}
}
{code}
Note how you can register state during instantiation (in the constructor) and in the invoke method. Both registrations should be picked up by the runtime and bound to Flink state.
This will be useful for a few scenarios:
- Could enable us to get rid of eager state spec definitions in the YAML modules for remote functions in the future.
- Will allow new state to be registered in remote functions, without shutting down the StateFun cluster.
- Moreover, this approach allows us to differentiate which functions have dynamic state and which ones have only eager state, which might be handy in the future in case there is a need to differentiate.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
