[jira] [Created] (FLINK-6022) Improve support for Avro GenericRecord

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-6022) Improve support for Avro GenericRecord

Shang Yuanchun (Jira)
Robert Metzger created FLINK-6022:
-------------------------------------

             Summary: Improve support for Avro GenericRecord
                 Key: FLINK-6022
                 URL: https://issues.apache.org/jira/browse/FLINK-6022
             Project: Flink
          Issue Type: Improvement
          Components: Type Serialization System
            Reporter: Robert Metzger


Currently, Flink is serializing the schema for each Avro GenericRecord in the stream.
This leads to a lot of overhead over the wire/disk + high serialization costs.

Therefore, I'm proposing to improve the support for GenericRecord in Flink by shipping the schema to each serializer  through the AvroTypeInformation.
Then, we can only support GenericRecords with the same type per stream, but the performance will be much better.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)