Kezhu Wang created FLINK-14793:
----------------------------------
Summary: Add SPI interface to discovery state backend and establish well-known names
Key: FLINK-14793
URL:
https://issues.apache.org/jira/browse/FLINK-14793 Project: Flink
Issue Type: New Feature
Components: Runtime / State Backends
Reporter: Kezhu Wang
Currently, {{StateBackendLoader.loadStateBackendFromConfig}} uses hybrid of hardcoded state backend name and class path to construct {{StateBackendFactory}}.
I think we can provide a SPI interface, say \{{StateBackendProvider}}, to accomplish this.
{code:title=StateBackendProvider.java}
/**
* SPI interface for {@link StateBackend} discovery and naming.
*
* @param <T> StateBackend
*/
@PublicEvolving
public interface StateBackendProvider<T extends StateBackend> extends StateBackendFactory<T> {
/**
* Name of provided {@link StateBackend}.
*
* @return name of provided {@link StateBackend}
*/
@Nonnull
String name();
/**
* Aliases for provided {@link StateBackend}. Implementations can use this for deprecated names.
*
* @return aliases for provided {@link StateBackend}
*/
default Set<String> aliases() {
return Collections.emptySet();
}
}
{code}
With above spi, we gain several benefits over current hardcode and class path approach:
* Explicit and established state backend naming and discovery instead of hardcoded. We can know exactly which state backends are avaiable and whether they are provided by user jars or not.
* In state backend configuration, we can simply use established state backend name while not tied to fragile class path which is easily refactored out without great caution. These established state backend names could be flink community level, such as `jobmanager`, `filesystem` and `rocksdb`, and corporation level, such as `corporation-a.product.state-backend-basename`. Together with FLINK-14790, we can switch among state backends without knowledge of concrete implementation of target state backends in source code level.
* More compatible with java module shipped in java 9 and above.
Discovery of state backend can be restricted in {{StateBackendLoader}}, and we can fallback to class path approach if no matching state backend found in discovered state backends.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
