[jira] [Created] (FLINK-20265) Extend invocation protocol to allow functions to indicate incomplete state context

[Shang Yuanchun (Jira)]

Tzu-Li (Gordon) Tai created FLINK-20265:
-------------------------------------------

             Summary: Extend invocation protocol to allow functions to indicate incomplete state context
                 Key: FLINK-20265
                 URL: https://issues.apache.org/jira/browse/FLINK-20265
             Project: Flink
          Issue Type: Sub-task
          Components: Stateful Functions
            Reporter: Tzu-Li (Gordon) Tai
            Assignee: Tzu-Li (Gordon) Tai
             Fix For: statefun-2.3.0


Currently, users declare the states a function will access with a module YAML definition file. The modules are loaded once when starting a StateFun cluster, meaning that the state specifications remain static throughout the cluster's execution lifetime.

We propose that state specifications should be declared by the function themselves via the language SDKs, instead of being declared in the module YAMLs.

The state specifications, now living in the functions, can be made discoverable by the StateFun runtime through the invocation request-reply protocol.

Brief simplified sketch of the extended protocol:
- StateFun dispatches an invocation request, with states {A, B}.

- Function receives request, but since it requires {A, B, C, D}, it responds with a IncompleteInvocationContext response indicating that state values for C, D is missing

- StateFun receives this response, and registers new Flink state handles for {C, D}.

- Then, a new invocation request with the same input messages, but "patched" with new states to contain all values for {A, B, C, D} is resent to the function.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)