[jira] [Created] (FLINK-2631) StreamFold operator does not respect returns type and stores non serializable values

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-2631) StreamFold operator does not respect returns type and stores non serializable values

Shang Yuanchun (Jira)
Till Rohrmann created FLINK-2631:
------------------------------------

             Summary: StreamFold operator does not respect returns type and stores non serializable values
                 Key: FLINK-2631
                 URL: https://issues.apache.org/jira/browse/FLINK-2631
             Project: Flink
          Issue Type: Bug
            Reporter: Till Rohrmann


The {{StreamFold}} operator stores the initial value of the fold operation for the task deployment. This value does not necessarily have to be serializable. Thus, using the fold operation with a non-serializable initial value will fail the job.

Moreover, the {{StreamFold}} operator needs to know the output type in order to create a {{TypeSerializer}}. For {{StreamGraphs}} where the output type is not know when the operator is created, as it is the case for the Scala DataStream API which directly sets the output type after creating the operator via the {{returns}} method, this approach will fail. The reason is that the {{StreamFold}} operator does receive the type information set by the {{returns}} method. Therefore, the job will fail at runtime because the operator tries to create a serializer from a {{MissingTypeInformation}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)