[jira] [Created] (FLINK-4155) Get Kafka producer partition info in open method instead of constructor

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-4155) Get Kafka producer partition info in open method instead of constructor

Shang Yuanchun (Jira)
Gyula Fora created FLINK-4155:
---------------------------------

             Summary: Get Kafka producer partition info in open method instead of constructor
                 Key: FLINK-4155
                 URL: https://issues.apache.org/jira/browse/FLINK-4155
             Project: Flink
          Issue Type: Bug
          Components: Kafka Connector
    Affects Versions: 1.0.3, 1.1.0
            Reporter: Gyula Fora


Currently the Flink Kafka producer does not really do any error handling if something is wrong with the partition metadata as it is serialized with the user function.

This means that in some cases the job can go into an error loop when using the checkpoints. Getting the partition info in the open method would solve this problem (like restarting from a savepoint which re-runs the constructor).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)