Use case

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Use case

Kevin Jacobs
Hi all,

I am trying to keep track of the biggest value in a stream. I do this by
using the iterative step mechanism of Apache Flink. However, I get an
exception that checkpointing is not supported for iterative jobs. Why
can't this be enabled? My iterative stream is also quite small: only one
value. Is there a better way to keep track of a largest value in a stream?

Regards,
Kevin

Reply | Threaded
Open this post in threaded view
|

Re: Use case

Timo Walther-2
Hi Kevin,

I don't know what your entire program is doing but wouldn't be a
FlatMapFunction containing a state with your biggest value sufficient?
Your stream goes through your FlatMapper and compares with the last
saved biggest value. You can then emit something if the value has increased.

I hope that helps.

Timo

Am 28/07/16 um 17:00 schrieb Kevin Jacobs:

> Hi all,
>
> I am trying to keep track of the biggest value in a stream. I do this
> by using the iterative step mechanism of Apache Flink. However, I get
> an exception that checkpointing is not supported for iterative jobs.
> Why can't this be enabled? My iterative stream is also quite small:
> only one value. Is there a better way to keep track of a largest value
> in a stream?
>
> Regards,
> Kevin


--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr