watermark in fault tolerance

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

watermark in fault tolerance

lonely Feb
I wonder if stream source task which extract timestamps and watermark from
source element fail and restart with fault tolerance, what kind of
watermark it would emit?
Reply | Threaded
Open this post in threaded view
|

Re: watermark in fault tolerance

Aljoscha Krettek-2
Hi,
the current watermark position is not persisted. The sources will start
emitting based on the elements that they are seeing. So if the source is
reset to some previous reading position (say with Kafka). Then it will emit
the watermarks based on those earlier elements again.

Cheers,
Aljoscha

On Sat, 26 Mar 2016 at 09:16 lonely Feb <[hidden email]> wrote:

> I wonder if stream source task which extract timestamps and watermark from
> source element fail and restart with fault tolerance, what kind of
> watermark it would emit?
>