Assign a unique id to each line of a dataset

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Assign a unique id to each line of a dataset

Thomas FOURNIER
Hello,

Is it possible with the current Flink-API to give a unique id to each line
of a dataset ?

More precisely, I've globally sorted my Dataset with partitionByRange and
I'd like to perform a kind of "zipWithIndex" operation, so that I can
retrieve a Map (such as a collectAsMap with Spark).

Thanks
Regards

Thomas
Reply | Threaded
Open this post in threaded view
|

Re: Assign a unique id to each line of a dataset

Fabian Hueske-2
Hi Thomas,

have a look at DataSetUtils [1].

Best, Fabian

[1]
https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/utils/DataSetUtils.java

2016-11-02 13:17 GMT+01:00 Thomas FOURNIER <[hidden email]>:

> Hello,
>
> Is it possible with the current Flink-API to give a unique id to each line
> of a dataset ?
>
> More precisely, I've globally sorted my Dataset with partitionByRange and
> I'd like to perform a kind of "zipWithIndex" operation, so that I can
> retrieve a Map (such as a collectAsMap with Spark).
>
> Thanks
> Regards
>
> Thomas
>