Which window function to use to start a window at anytime

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Which window function to use to start a window at anytime

Bowen Li
Hi guys,

I sent this email to Flink user email alias but didn't get any feedbacks.
I'm sending it here.

We are trying use Flink to count millions of keyed items of an hour window
hourly as `time(SlidingEventTimeWindows.of(1hour, 1hour))`. According to
the sliding window doc
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/windows.html#sliding-windows>,
all windows are aligned with epoch and always start at integral hours like
1:00:00.000 - 1:59:59.999.

But we actually want to start an hour window whenever an new item arrives.
For example,
- for item A, the first event arrives at 1:12:24.123, so the window would
be 1:12:24.123 - 2:12:24.122, and the next window would be 2:12:24.123 -
3:12:24.122, and so on
- for item B, the first event arrives at 1:10:20:321, so the window would
be 1:10:20:321 - 2:10:20:320, and the next window would be 2:10:20:321 -
3:10:20:320, and so on.

Do you have any insights on how to achieve it? Thanks!

Bowen