Posted by
curcur on
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/DISCUSS-Statebackend-Runtime-Changelog-Statebackend-Configuration-Proposal-tp50981.html
Hey all,
We would like to start a discussion on how to enable/config Changelog
Statebakcend.
As part of FLIP-158[1], Changelog state backend wraps on top of existing
state backend (HashMapStateBackend, EmbeddedRocksDBStateBackend and may
expect more) and delegates state changes to the underlying state backends.
This thread is to discuss the problem of how Changelog StateBackend should
be enabled and configured.
Proposed options to enable/config state changelog is listed below:
Option 1: Enable Changelog Statebackend through a Boolean Flag
Option 2: Enable Changelog Statebackend through a Boolean Flag + a Special
Case
Option 3: Enable Changelog Statebackend through a Boolean Flag + W/O
ChangelogStateBackend Exposed
Option 4: Explicit Nested Configuration + “changelog.inner” prefix for
inner backend
Option 5: Explicit Nested Configuration + inner state backend configuration
unchanged
Option 6: Config Changelog and Inner Statebackend All-Together
Details of each option can be found here:
https://docs.google.com/document/d/13AaCf5fczYTDHZ4G1mgYL685FqbnoEhgo0cdwuJlZmw/edit?usp=sharingWhen considering these options, please consider these four dimensions:
1 Consistency
API/config should follow a consistent model and should not have
contradicted logic beneath
2 Simplicity
API should be easy to use and not introduce too much burden on users
3. Explicity
API/config should not contain implicit assumptions and should be intuitive
to users
4. Extensibility
With foreseen future, whether the current setting can be easily extended
Please let us know what do you think and please keep the discussion in this
mailing thread.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-158%3A+Generalized+incremental+checkpointsBest
Yuan