[jira] [Created] (FLINK-11947) Support MapState key / value schema evolution for RocksDB

[Shang Yuanchun (Jira)]

Tzu-Li (Gordon) Tai created FLINK-11947:
-------------------------------------------

             Summary: Support MapState key / value schema evolution for RocksDB
                 Key: FLINK-11947
                 URL: https://issues.apache.org/jira/browse/FLINK-11947
             Project: Flink
          Issue Type: Improvement
          Components: API / Type Serialization System, Runtime / State Backends
            Reporter: Tzu-Li (Gordon) Tai


Currently, we do not attempt to perform state schema evolution if the key or value's schema of a user {{MapState}} has changed when using {{RocksDB}}:

https://github.com/apache/flink/blob/953a5ffcbdae4115f7d525f310723cf8770779df/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L542

This was disallowed in the initial support for state schema evolution because the way we did state evolution in the RocksDB state backend was simply overwriting values.

For {{MapState}} key evolution, only overwriting RocksDB values does not work, since RocksDB entries for {{MapState}} uses a composite key containing the map state key. This means that when evolving {{MapState}} in this case with an evolved key schema, we will have new entries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)