Steve Bairos created FLINK-18038:
------------------------------------
Summary: StateBackendLoader logs application-defined state before it is fully configured
Key: FLINK-18038
URL:
https://issues.apache.org/jira/browse/FLINK-18038 Project: Flink
Issue Type: Bug
Components: Runtime / State Backends
Affects Versions: 1.9.1
Reporter: Steve Bairos
In the [StateBackendLoader|[
https://github.com/apache/flink/blob/bb46756b84940a6134910e74406bfaff4f2f37e9/flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java#L201]], there's this log line:
{code:java}
logger.info("Using application-defined state backend: {}", fromApplication); {code}
It seems like this is inaccurate though because immediately after logging this, if fromApplication is a ConfigurableStateBackend, we call the .configure() function and it is replaced by a newly configured StateBackend.
To me, it seems like it would be better if we logged the state backend after it was fully configured. In the current setup, we get confusing logs like this:
{code:java}
2020-05-29 21:39:44,387 INFO org.apache.flink.streaming.runtime.tasks.StreamTask - Using application-defined state backend: RocksDBStateBackend{checkpointStreamBackend=File State Backend (checkpoints: 's3://pinterest-montreal/checkpoints/xenon-dev-001-20191210/Xenon/BasicJavaStream', savepoints: 'null', asynchronous: UNDEFINED, fileStateThreshold: -1), localRocksDbDirectories=null, enableIncrementalCheckpointing=UNDEFINED, numberOfTransferingThreads=-1}2020-05-29 21:39:44,387 INFO org.apache.flink.streaming.runtime.tasks.StreamTask - Configuring application-defined state backend with job/cluster config{code}
Which makes it ambiguous whether or not settings in our flink-conf.yaml like "state.backend.incremental: true" are being applied properly or not.
I can make a diff for the change if there aren't any objections
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
