Fwd: Re: Flink Watermark and timing

[Timo Walther-2]

Hi Björn,

the behavior of borderlines is defined clearly by the API: "start
timestamp (inclusive) and an end timestamp (exclusive)". So it is always
[0-4] [5-9]. You could increase the interval by one millisecond to
include 5.


Regards,

Timo



-------- Weitergeleitete Nachricht --------
Betreff: Re: Flink Watermark and timing
Datum: Tue, 3 Oct 2017 06:37:13 +0200
Von: Björn Zachrisson <[hidden email]>
An: Timo Walther <[hidden email]>



Hi Timo,

One more question regarding that to clarify.
Where do i specify in which window a event that arrives on the exact
window-borderline, window sizes [0-5] [5-10] and the event arrives at
exactly 5
Where should the event go and can i control this?

Regards
Björn

2017-10-02 19:28 GMT+02:00 Timo Walther <[hidden email]
<mailto:[hidden email]>>:

    Hi Björn,


    I don't know if I get your example correctly, but I think your
    explanation "All events up to and equal to watermark should be
    handled in the prevoius window" is not 100% correct. Watermarks just
    indicate the progress ("until here we have seen all events with
    lower timestamp than X") and trigger the evaluation of a window. The
    assignment of events to windows is based on the timestamp not the
    watermark. The documentation will be improved for the upcoming release:

    https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/windows.html#window-assigners
    <https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/windows.html#window-assigners>

    "Time-based windows have a start timestamp (inclusive) and an end
    timestamp (exclusive) that together describe the size of the window. "

    I hope this helps.

    Regards,
    Timo


    Am 10/2/17 um 1:06 PM schrieb Björn Zachrisson:

>     Hi,
>
>     I have a question regarding timing of events.
>
>     According to;
>     https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/event_time.html#event-time-and-watermarks
>     <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/event_time.html#event-time-and-watermarks>
>
>     All events up to and equal to watermark should be handled in "the
>     prevoius window".
>
>     In my case I use event-timestamp.
>
>
>     I'm testing the timing out.
>
>     The case is events from 2000-01-01 02:00:00 and up to 2000-01-01
>     02:20:00 where eavh event is 2 minutes apart. I try to group the
>     events in 5 minute windows
>
>     2000-01-01 02:00:00 => 2000-01-01 02:05:00
>     2000-01-01 02:05:00 => 2000-01-01 02:10:00
>     2000-01-01 02:10:00 => 2000-01-01 02:15:00
>     2000-01-01 02:15:00 => 2000-01-01 02:20:00
>
>     How ever, events at the exakt time 02:10:00 (946692600000) is put
>     in the Window "2000-01-01 02:10:00 => 2000-01-01 02:15:00" which
>     is not according to what i can read on the wiki.
>
>     This is the exakt result;
>     2000-01-01 02:00:00, 946692000000
>     2000-01-01 02:02:00, 946692120000
>     2000-01-01 02:04:00, 946692240000
>
>     2000-01-01 02:06:00, 946692360000
>     2000-01-01 02:08:00, 946692480000
>
>     2000-01-01 02:10:00, 946692600000
>     2000-01-01 02:12:00, 946692720000
>     2000-01-01 02:14:00, 946692840000
>
>     2000-01-01 02:16:00, 946692960000
>     2000-01-01 02:18:00, 946693080000
>
>     2000-01-01 02:20:00, 946693200000
>
>     Is this due to that I'm using event time extractor or what might
>     be the case?
>
>     Regards
>     Björn
>
>