[jira] [Created] (FLINK-13538) Give field names in deserializers thrown exceptions

[Shang Yuanchun (Jira)]

François Lacombe created FLINK-13538:
----------------------------------------

             Summary: Give field names in deserializers thrown exceptions
                 Key: FLINK-13538
                 URL: https://issues.apache.org/jira/browse/FLINK-13538
             Project: Flink
          Issue Type: Improvement
          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
    Affects Versions: 1.8.0
            Reporter: François Lacombe


Deserializers like JsonRowDeserializerSchema parse JSON strings according to a TypeInformation<?> object.

Types mistakes can occur and it usually rise a IOException caused by a IllegalStateException. Here I try to parse "field":"blabla" described with Type.INT

 
{code:java}
java.io.IOException: Failed to deserialize JSON object.
    at org.apache.flink.formats.json.JsonRowDeserializationSchema.deserialize(JsonRowDeserializationSchema.java:97)
    at com.dcbrain.etl.inputformat.JsonInputFormat.nextRecord(JsonInputFormat.java:96)
    at com.dcbrain.etl.inputformat.JsonInputFormat.nextRecord(JsonInputFormat.java:1)
    at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:192)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Unsupported type information 'Integer' for node: "blabla"
    at org.apache.flink.formats.json.JsonRowDeserializationSchema.convert(JsonRowDeserializationSchema.java:191)
    at org.apache.flink.formats.json.JsonRowDeserializationSchema.convertRow(JsonRowDeserializationSchema.java:212)
    at org.apache.flink.formats.json.JsonRowDeserializationSchema.deserialize(JsonRowDeserializationSchema.java:95)
    ... 5 common frames omitted{code}
 

The message nor the exception objects contains reference to field causing this error which require time to inspect complex input data to find where the error really is.

Could it be possible to improve messages or even Exceptions objects thrown by Serializers/Deserializers to get which field is responsible of the error please?

JsonRowDeserializerSchema isn't the only one touched by such issues.

 

This will allow to produce more useful logs to be read by users or administrators.

 

All the best



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)