Splitting input stream in to multiple windows and process individually

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Splitting input stream in to multiple windows and process individually

madhairsilence
I have a Datastream . I have to apply window function as 1 hour, 2 hour, 3 hour...24 hour. And each split has its own window function to be done

How do I split the window in an efficient way.

The dirtiest way I can think of is

for(int i=1; i < 24 ;i++){
   inputStream.keyBy("id").timeWindow(Time.hours(i)).apply(..);
}
Reply | Threaded
Open this post in threaded view
|

Re: Splitting input stream in to multiple windows and process individually

Ufuk Celebi-2
Hey! I think this question is better suited for the user mailing list.
Your solution should work, but will require a lot of memory if you
can't use an incrementally aggregating window (e.g. fold/reduce/etc.).

– Ufuk

On Thu, Apr 13, 2017 at 7:39 AM, madhairsilence <[hidden email]> wrote:

> I have a Datastream . I have to apply window function as 1 hour, 2 hour, 3
> hour...24 hour. And each split has its own window function to be done
>
> How do I split the window in an efficient way.
>
> The dirtiest way I can think of is
>
> for(int i=1; i < 24 ;i++){
>    inputStream.keyBy("id").timeWindow(Time.hours(i)).apply(..);
> }
>
>
>
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Splitting-input-stream-in-to-multiple-windows-and-process-individually-tp17010.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.