[jira] [Created] (FLINK-21452) FLIP-27 sources cannot reliably downscale

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-21452) FLIP-27 sources cannot reliably downscale

Shang Yuanchun (Jira)
Arvid Heise created FLINK-21452:
-----------------------------------

             Summary: FLIP-27 sources cannot reliably downscale
                 Key: FLINK-21452
                 URL: https://issues.apache.org/jira/browse/FLINK-21452
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Common
    Affects Versions: 1.12.1, 1.13.0
            Reporter: Arvid Heise
            Assignee: Arvid Heise


Sources currently store their registered readers into the snapshot. However, when downscaling, we have unmatched readers that we violate a couple of invariants.

The solution is to not store registered readers - they are re-registered anyways on restart.

To keep it backward compatible, the best option is to always store an empty set of readers while writing the snapshot and discard any recovered readers from the snapshot.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)