Scala Tuple2 not recognized as a Scala Tuple

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Scala Tuple2 not recognized as a Scala Tuple

Pieter-Jan Van Aeken
From the Java API, I get an instance of_ DataSet_​ which I wrap with
the method available in _org.apache.flink.api.scala.package. _This
works fine when I print out the result, but when I try to write as
CSV, I get an exception:

java.lang.IllegalArgumentException: requirement failed: CSV output can
only be used with Tuple DataSets.

It seems that an instance of _DataSet[scala.Tuple2[Long, Long]]_ is
somehow considered different from an instance of_ DataSet[(Long,
Long)]_

Are there any suggestions on how to fix this problem? Note that I
cannot hardcode the conversion as the tuple size may vary or it might
be an entirely different custom class.

Regards,

Pieter-Jan

Reply | Threaded
Open this post in threaded view
|

Re: Scala Tuple2 not recognized as a Scala Tuple

Pieter-Jan Van Aeken
Found the solution: Creating the TypeInformation in Scala and then
passing it to Java where it is used explicitly in the returns method
resolved the issue.

Op Maandag, 01/06/2015 om 16:59 schreef Pieter-Jan Van Aeken:

From the Java API, I get an instance of_ DataSet_​ which I wrap with
the method available in _org.apache.flink.api.scala.package. _This
works fine when I print out the result, but when I try to write as
CSV, I get an exception:

java.lang.IllegalArgumentException: requirement failed: CSV output can
only be used with Tuple DataSets.

It seems that an instance of _DataSet[scala.Tuple2[Long, Long]]_ is
somehow considered different from an instance of_ DataSet[(Long,
Long)]_

Are there any suggestions on how to fix this problem? Note that I
cannot hardcode the conversion as the tuple size may vary or it might
be an entirely different custom class.

Regards,

Pieter-Jan


Reply | Threaded
Open this post in threaded view
|

Re: Scala Tuple2 not recognized as a Scala Tuple

Aljoscha Krettek-2
Great, good to hear!
On Jun 1, 2015 5:39 PM, "Pieter-Jan Van Aeken" <
[hidden email]> wrote:

> Found the solution: Creating the TypeInformation in Scala and then
> passing it to Java where it is used explicitly in the returns method
> resolved the issue.
>
> Op Maandag, 01/06/2015 om 16:59 schreef Pieter-Jan Van Aeken:
>
> From the Java API, I get an instance of_ DataSet_​ which I wrap with
> the method available in _org.apache.flink.api.scala.package. _This
> works fine when I print out the result, but when I try to write as
> CSV, I get an exception:
>
> java.lang.IllegalArgumentException: requirement failed: CSV output can
> only be used with Tuple DataSets.
>
> It seems that an instance of _DataSet[scala.Tuple2[Long, Long]]_ is
> somehow considered different from an instance of_ DataSet[(Long,
> Long)]_
>
> Are there any suggestions on how to fix this problem? Note that I
> cannot hardcode the conversion as the tuple size may vary or it might
> be an entirely different custom class.
>
> Regards,
>
> Pieter-Jan
>
>
>