Tzu-Li (Gordon) Tai created FLINK-11327:
-------------------------------------------
Summary: Migrate all remaining Flink serializers' snapshot classes to the new TypeSerializerSnapshot / TypeSerializerSchemaCompatibility abstractions
Key: FLINK-11327
URL:
https://issues.apache.org/jira/browse/FLINK-11327 Project: Flink
Issue Type: Task
Components: State Backends, Checkpointing, Type Serialization System
Reporter: Tzu-Li (Gordon) Tai
Fix For: 1.8.0
There are still a few serializers in Flink that needs to be upgraded to use the new serialization compatibility APIs (`TypeSerializerSnapshot` and `TypeSerializerSchemaCompatibility`).
By doing this as soon as possible (ideally all are migrated for the 1.8 release), state serializers would no longer be written via Java serialization into savepoints, and in general allowing serializer upgrades more future-proof.
To split up the efforts, the remaining serializers can be categorized as follows.
They are categorized because some are fairly straightforward, while some are more complex and involves reconfiguration / migration cases.
We should have an independent sub-task JIRA for each category:
* Parameterless serializers / subclasses of `TypeSerializerSingleton`
* Simple composite serializers that contain nested serializers
* Scala-macro generated serializers
* `POJOSerializer`
* Kryo-related serializers
* Enum serializers
After upgrading a serializers' snapshot, the following should be achieved to consider the migration completed:
* The serializer should now return a `TypeSerializerSnapshot`, and not a subclass of the legacy `TypeSerializerConfigSnapshot` as its snapshot.
* Check whether or not the legacy `TypeSerializer#ensureCompatibility` method can be removed from the serializer class.
* There is a corresponding test specification for the migrated serializer that uses the `TypeSerializerSnapshotMigrationTestBase`.
Ideally, to complete this JIRA, we should also have some utilities / meta-tests:
* A test that verifies ALL serializers in the Flink codebase have a corresponding migration test that uses the `TypeSerializerSnapshotMigrationTestBase`.
* A utility script that generates test data and snapshots of all serializers in the Flink codebase, to be used by the migration tests. This script should be used every time we release a Flink major version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
