John created FLINK-12304:
----------------------------
Summary: AvroInputFormat should support schema evolution
Key: FLINK-12304
URL:
https://issues.apache.org/jira/browse/FLINK-12304 Project: Flink
Issue Type: Bug
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.8.0
Reporter: John
From the avro spec:
_A reader of Avro data, whether from an RPC or a file, can always parse that data because its schema is provided. But that schema may not be exactly the schema that was expected. For example, if the data was written with a different version of the software than it is read, then records may have had fields added or removed._
The AvroInputFormat should allow the application to supply a reader's schema to support cases where data was written with an old version of a schema and needs to be read with a newer version. The reader's schema can have addition fields with defaults so that the old schema can be adapted to the new. The underlying avro java library supports schema resolution, so adding support in AvroInputFormat should be straight forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
