(DEPRECATED) Apache Flink Mailing List archive.

Streaming windowing rework

Classic

List

Threaded

25 messages Options

Márton Balassi

Streaming windowing rework

Hey,

As you may know in some European countries Santa Claus comes on the 6th of
December, Saint Nicolas Day - from Finland.

As Flink is not yet very active in Finland our Sweden team stepped in and
gave us a big present with a huge rework on the windowing semantics - this
is the implementation of the ideas that were posed the Stockholm hackathon
in October.

Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed intro to
the new features here soon.

Marton

Stephan Ewen

Re: Streaming windowing rework

Wow, good news :-)

Saint Nicolas had the help of his Finish elves Parisson, Gyulafred, and
Jonasson ?

On Fri, Dec 5, 2014 at 6:34 PM, Márton Balassi <[hidden email]>
wrote:

> Hey,
>
> As you may know in some European countries Santa Claus comes on the 6th of
> December, Saint Nicolas Day - from Finland.
>
> As Flink is not yet very active in Finland our Sweden team stepped in and
> gave us a big present with a huge rework on the windowing semantics - this
> is the implementation of the ideas that were posed the Stockholm hackathon
> in October.
>
> Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed intro to
> the new features here soon.
>
> Marton
>

Vasiliki Kalavri

Re: Streaming windowing rework

Yooohooo! Grattis Swedish team ^^

On 5 December 2014 at 18:54, Stephan Ewen <[hidden email]> wrote:

> Wow, good news :-)
>
> Saint Nicolas had the help of his Finish elves Parisson, Gyulafred, and
> Jonasson ?
>
>
>
> On Fri, Dec 5, 2014 at 6:34 PM, Márton Balassi <[hidden email]>
> wrote:
>
> > Hey,
> >
> > As you may know in some European countries Santa Claus comes on the 6th
> of
> > December, Saint Nicolas Day - from Finland.
> >
> > As Flink is not yet very active in Finland our Sweden team stepped in and
> > gave us a big present with a huge rework on the windowing semantics -
> this
> > is the implementation of the ideas that were posed the Stockholm
> hackathon
> > in October.
> >
> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed intro
> to
> > the new features here soon.
> >
> > Marton
> >
>

Henry Saputra

Re: Streaming windowing rework

In reply to this post by Márton Balassi

Nice! Looking forward to it

On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <[hidden email]> wrote:

Kostas Tzoumas-2

Re: Streaming windowing rework

Super!

Paris and Jonas have been doing a lot of "silent" work (= not very visible
in the mailing list) recently. Looking forward to seeing this coming in.

On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <[hidden email]>
wrote:

> Nice! Looking forward to it
>
> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <[hidden email]>
> wrote:
> > Hey,
> >
> > As you may know in some European countries Santa Claus comes on the 6th
> of
> > December, Saint Nicolas Day - from Finland.
> >
> > As Flink is not yet very active in Finland our Sweden team stepped in and
> > gave us a big present with a huge rework on the windowing semantics -
> this
> > is the implementation of the ideas that were posed the Stockholm
> hackathon
> > in October.
> >
> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed intro
> to
> > the new features here soon.
> >
> > Marton
>

Henry Saputra

Re: Streaming windowing rework

We actually need to bring these work come to light early so we could
give feedback before getting to deep.

Marton, could you encourage Paris and Jonas to publish or share their
thoughts about the work they are doing?

Would love to give some feedbacks.

- Henry

On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]> wrote:

> Super!
>
> Paris and Jonas have been doing a lot of "silent" work (= not very visible
> in the mailing list) recently. Looking forward to seeing this coming in.
>
> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <[hidden email]>
> wrote:
>
>> Nice! Looking forward to it
>>
>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <[hidden email]>
>> wrote:
>> > Hey,
>> >
>> > As you may know in some European countries Santa Claus comes on the 6th
>> of
>> > December, Saint Nicolas Day - from Finland.
>> >
>> > As Flink is not yet very active in Finland our Sweden team stepped in and
>> > gave us a big present with a huge rework on the windowing semantics -
>> this
>> > is the implementation of the ideas that were posed the Stockholm
>> hackathon
>> > in October.
>> >
>> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed intro
>> to
>> > the new features here soon.
>> >
>> > Marton
>>

Jonas Traub

Re: Streaming windowing rework

Hello to everybody!

Sorry for the delay, but here comes a short introduction for the new
features!

We are glad to announce that we've introduced newly developed and high
flexible windowing semantics in flink-streaming.

Flink-streaming allows now to define windows using policies. There are two
types of policies, Trigger- and Eviction-policies. The trigger policy
specifies when the reduce function gets executed on the current element
buffer (respectively the current window) and the result gets emitted. The
eviction policy specifies when elements get removed from the buffer (window
size).

We provide a wide range of pre-defined policies, such as count-based,
time-based, punctuation-based and delta-based.
Punctuation allows to specify windows externally by putting flags at some
elements in the stream. Delta-policies can apply a distance measure to
decide when to trigger or evict.

Using the provided set of predefined policies users can already define a
huge variate of windows. For example:

- Hold elements of the last 5 seconds, while the user defined
aggregation/reduce is executed on the windows every second (sliding the
window by 1 second):
- dataStream.window(Time.of(5, TimeUnit.SECONDS)).every(Time.of(1,
TimeUnit.SECONDS))

- Downsample our stream: Take the latest 100 elements of our stream
every minute:
- dataStream.window(Count.of(100)).every(Time.of(1, TimeUnit.MINUTES))

- Using combinations of multiple different policies, we can go even
further:
- Every 5 minutes give me the aggregations of the last 1 mio.
elements but never include elements older than 10 minutes.
- Emit every 5 minutes, but immediately in case the current value
differs more than X% from the last one.

Additionally, policies can be implemented as UDFs, which gives the user the
highest possible flexibility for the specification of windows.

The new policy based windowing can also be used for grouped streams.
For example: To get the maximal value by key on the last 100 elements we
use the following approach:

-
dataStream.window(Count.of(100)).every(…).groupBy(groupingField).max(field);

To create fixed size windows for every key we need to reverse the order of
the groupBy call. So to take the max for the last 100 elements in Each
group:

-
dataStream.groupBy(groupingField).window(Count.of(100)).every(…).max(field);

This will create separate windows for different keys and apply the trigger
and eviction policies on a per group basis.

We extended the streaming guide[1] with a documentation about the new
features. Additionally you can find several examples in the windowing
package[2].

Kind Regards
Jonas Traub, Gyula Fóra, Paris Carbone

[1]
https://github.com/apache/incubator-flink/blob/master/docs/streaming_guide.md
[2]
https://github.com/apache/incubator-flink/tree/master/flink-addons/flink-streaming/flink-streaming-examples/src/main/java/org/apache/flink/streaming/examples/windowing

2014-12-06 0:56 GMT+01:00 Henry Saputra <[hidden email]>:

> We actually need to bring these work come to light early so we could
> give feedback before getting to deep.
>
> Marton, could you encourage Paris and Jonas to publish or share their
> thoughts about the work they are doing?
>
> Would love to give some feedbacks.
>
> - Henry
>
> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]>
> wrote:
> > Super!
> >
> > Paris and Jonas have been doing a lot of "silent" work (= not very
> visible
> > in the mailing list) recently. Looking forward to seeing this coming in.
> >
> > On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <[hidden email]>
> > wrote:
> >
> >> Nice! Looking forward to it
> >>
> >> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> [hidden email]>
> >> wrote:
> >> > Hey,
> >> >
> >> > As you may know in some European countries Santa Claus comes on the
> 6th
> >> of
> >> > December, Saint Nicolas Day - from Finland.
> >> >
> >> > As Flink is not yet very active in Finland our Sweden team stepped in
> and
> >> > gave us a big present with a huge rework on the windowing semantics -
> >> this
> >> > is the implementation of the ideas that were posed the Stockholm
> >> hackathon
> >> > in October.
> >> >
> >> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
> intro
> >> to
> >> > the new features here soon.
> >> >
> >> > Marton
> >>
>

Gyula Fóra

Re: Streaming windowing rework

In reply to this post by Henry Saputra

Hey Henry,

(I am resending this mail because it seems it didnt make it through the
mail system)

I suggest to have a public hangout sometimes on the current work we are
doing and our plans on streaming.

We have actually put together a list of topics that we want to work on next
year as we will have a quite large Flink team at SICS from January: Paris,
Jonas, Me and later Vasia.

Basically the topics are:
- Windowing semantics and multiquery optimisations (thats mainly Jonas'
thesis topic)
- Different fault tolerance approaches
- Connecting batch and streaming api ('lambda')
- Streaming graph algorithms
- Samoa for Flink (and streaming machine learning in general)

So lets start a series of public hangout calls to discuss these topics, I
would suggest one sometimes next week when it is suitable.

Regards,
Gyula

On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <[hidden email]>
wrote:

Kostas Tzoumas-2

Re: Streaming windowing rework

+1 great idea

On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]> wrote:

> Hey Henry,
>
> (I am resending this mail because it seems it didnt make it through the
> mail system)
>
> I suggest to have a public hangout sometimes on the current work we are
> doing and our plans on streaming.
>
> We have actually put together a list of topics that we want to work on next
> year as we will have a quite large Flink team at SICS from January: Paris,
> Jonas, Me and later Vasia.
>
> Basically the topics are:
> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
> thesis topic)
> - Different fault tolerance approaches
> - Connecting batch and streaming api ('lambda')
> - Streaming graph algorithms
> - Samoa for Flink (and streaming machine learning in general)
>
> So lets start a series of public hangout calls to discuss these topics, I
> would suggest one sometimes next week when it is suitable.
>
> Regards,
> Gyula
>
> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <[hidden email]>
> wrote:
>
> > We actually need to bring these work come to light early so we could
> > give feedback before getting to deep.
> >
> > Marton, could you encourage Paris and Jonas to publish or share their
> > thoughts about the work they are doing?
> >
> > Would love to give some feedbacks.
> >
> > - Henry
> >
> > On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]>
> > wrote:
> > > Super!
> > >
> > > Paris and Jonas have been doing a lot of "silent" work (= not very
> > visible
> > > in the mailing list) recently. Looking forward to seeing this coming
> in.
> > >
> > > On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <[hidden email]
> >
> > > wrote:
> > >
> > >> Nice! Looking forward to it
> > >>
> > >> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> > [hidden email]>
> > >> wrote:
> > >> > Hey,
> > >> >
> > >> > As you may know in some European countries Santa Claus comes on the
> > 6th
> > >> of
> > >> > December, Saint Nicolas Day - from Finland.
> > >> >
> > >> > As Flink is not yet very active in Finland our Sweden team stepped
> in
> > and
> > >> > gave us a big present with a huge rework on the windowing semantics
> -
> > >> this
> > >> > is the implementation of the ideas that were posed the Stockholm
> > >> hackathon
> > >> > in October.
> > >> >
> > >> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
> > intro
> > >> to
> > >> > the new features here soon.
> > >> >
> > >> > Marton
> > >>
> >
>

Paris Carbone

RE: Streaming windowing rework

+1

I agree it will be nice to discuss our upcoming directions and ideas with anyone that is interested to give feedback.
One suggested time for the hangout is next Friday 12 Dec. 17:30pm (GMT+1). I hope it is cross-atlantic friendly enough. Is that generally ok for the rest?

Paris

PS: thanks for the hilarious introduction of the windowing semantics everyone :)

________________________________________
From: Kostas Tzoumas [[hidden email]]
Sent: Sunday, December 07, 2014 1:23 PM
To: [hidden email]
Subject: Re: Streaming windowing rework

+1 great idea

On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]> wrote:

Márton Balassi

Re: Streaming windowing rework

Friday late afternoon works for me. :)

On Sun, Dec 7, 2014 at 8:30 PM, Paris Carbone <[hidden email]> wrote:

> +1
>
> I agree it will be nice to discuss our upcoming directions and ideas with
> anyone that is interested to give feedback.
> One suggested time for the hangout is next Friday 12 Dec. 17:30pm (GMT+1).
> I hope it is cross-atlantic friendly enough. Is that generally ok for the
> rest?
>
> Paris
>
> PS: thanks for the hilarious introduction of the windowing semantics
> everyone :)
>
> ________________________________________
> From: Kostas Tzoumas [[hidden email]]
> Sent: Sunday, December 07, 2014 1:23 PM
> To: [hidden email]
> Subject: Re: Streaming windowing rework
>
> +1 great idea
>
> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]> wrote:
>
> > Hey Henry,
> >
> > (I am resending this mail because it seems it didnt make it through the
> > mail system)
> >
> > I suggest to have a public hangout sometimes on the current work we are
> > doing and our plans on streaming.
> >
> > We have actually put together a list of topics that we want to work on
> next
> > year as we will have a quite large Flink team at SICS from January:
> Paris,
> > Jonas, Me and later Vasia.
> >
> > Basically the topics are:
> > - Windowing semantics and multiquery optimisations (thats mainly Jonas'
> > thesis topic)
> > - Different fault tolerance approaches
> > - Connecting batch and streaming api ('lambda')
> > - Streaming graph algorithms
> > - Samoa for Flink (and streaming machine learning in general)
> >
> > So lets start a series of public hangout calls to discuss these topics, I
> > would suggest one sometimes next week when it is suitable.
> >
> > Regards,
> > Gyula
> >
> > On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <[hidden email]>
> > wrote:
> >
> > > We actually need to bring these work come to light early so we could
> > > give feedback before getting to deep.
> > >
> > > Marton, could you encourage Paris and Jonas to publish or share their
> > > thoughts about the work they are doing?
> > >
> > > Would love to give some feedbacks.
> > >
> > > - Henry
> > >
> > > On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]>
> > > wrote:
> > > > Super!
> > > >
> > > > Paris and Jonas have been doing a lot of "silent" work (= not very
> > > visible
> > > > in the mailing list) recently. Looking forward to seeing this coming
> > in.
> > > >
> > > > On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > >> Nice! Looking forward to it
> > > >>
> > > >> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> > > [hidden email]>
> > > >> wrote:
> > > >> > Hey,
> > > >> >
> > > >> > As you may know in some European countries Santa Claus comes on
> the
> > > 6th
> > > >> of
> > > >> > December, Saint Nicolas Day - from Finland.
> > > >> >
> > > >> > As Flink is not yet very active in Finland our Sweden team stepped
> > in
> > > and
> > > >> > gave us a big present with a huge rework on the windowing
> semantics
> > -
> > > >> this
> > > >> > is the implementation of the ideas that were posed the Stockholm
> > > >> hackathon
> > > >> > in October.
> > > >> >
> > > >> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
> > > intro
> > > >> to
> > > >> > the new features here soon.
> > > >> >
> > > >> > Marton
> > > >>
> > >
> >
>

Vasiliki Kalavri

Re: Streaming windowing rework

+1 I'm in for Friday too!

On 7 December 2014 at 20:42, Márton Balassi <[hidden email]>
wrote:

> Friday late afternoon works for me. :)
>
> On Sun, Dec 7, 2014 at 8:30 PM, Paris Carbone <[hidden email]> wrote:
>
> > +1
> >
> > I agree it will be nice to discuss our upcoming directions and ideas with
> > anyone that is interested to give feedback.
> > One suggested time for the hangout is next Friday 12 Dec. 17:30pm
> (GMT+1).
> > I hope it is cross-atlantic friendly enough. Is that generally ok for the
> > rest?
> >
> > Paris
> >
> > PS: thanks for the hilarious introduction of the windowing semantics
> > everyone :)
> >
> > ________________________________________
> > From: Kostas Tzoumas [[hidden email]]
> > Sent: Sunday, December 07, 2014 1:23 PM
> > To: [hidden email]
> > Subject: Re: Streaming windowing rework
> >
> > +1 great idea
> >
> > On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]>
> wrote:
> >
> > > Hey Henry,
> > >
> > > (I am resending this mail because it seems it didnt make it through the
> > > mail system)
> > >
> > > I suggest to have a public hangout sometimes on the current work we are
> > > doing and our plans on streaming.
> > >
> > > We have actually put together a list of topics that we want to work on
> > next
> > > year as we will have a quite large Flink team at SICS from January:
> > Paris,
> > > Jonas, Me and later Vasia.
> > >
> > > Basically the topics are:
> > > - Windowing semantics and multiquery optimisations (thats mainly Jonas'
> > > thesis topic)
> > > - Different fault tolerance approaches
> > > - Connecting batch and streaming api ('lambda')
> > > - Streaming graph algorithms
> > > - Samoa for Flink (and streaming machine learning in general)
> > >
> > > So lets start a series of public hangout calls to discuss these
> topics, I
> > > would suggest one sometimes next week when it is suitable.
> > >
> > > Regards,
> > > Gyula
> > >
> > > On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
> [hidden email]>
> > > wrote:
> > >
> > > > We actually need to bring these work come to light early so we could
> > > > give feedback before getting to deep.
> > > >
> > > > Marton, could you encourage Paris and Jonas to publish or share their
> > > > thoughts about the work they are doing?
> > > >
> > > > Would love to give some feedbacks.
> > > >
> > > > - Henry
> > > >
> > > > On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
> >
> > > > wrote:
> > > > > Super!
> > > > >
> > > > > Paris and Jonas have been doing a lot of "silent" work (= not very
> > > > visible
> > > > > in the mailing list) recently. Looking forward to seeing this
> coming
> > > in.
> > > > >
> > > > > On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > >> Nice! Looking forward to it
> > > > >>
> > > > >> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> > > > [hidden email]>
> > > > >> wrote:
> > > > >> > Hey,
> > > > >> >
> > > > >> > As you may know in some European countries Santa Claus comes on
> > the
> > > > 6th
> > > > >> of
> > > > >> > December, Saint Nicolas Day - from Finland.
> > > > >> >
> > > > >> > As Flink is not yet very active in Finland our Sweden team
> stepped
> > > in
> > > > and
> > > > >> > gave us a big present with a huge rework on the windowing
> > semantics
> > > -
> > > > >> this
> > > > >> > is the implementation of the ideas that were posed the Stockholm
> > > > >> hackathon
> > > > >> > in October.
> > > > >> >
> > > > >> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a
> detailed
> > > > intro
> > > > >> to
> > > > >> > the new features here soon.
> > > > >> >
> > > > >> > Marton
> > > > >>
> > > >
> > >
> >
>

Gyula Fóra

Re: Streaming windowing rework

Good for me as well :)

Gyula

> On 07 Dec 2014, at 21:14, Vasiliki Kalavri <[hidden email]> wrote:
>
> +1 I'm in for Friday too!
>
> On 7 December 2014 at 20:42, Márton Balassi <[hidden email]>
> wrote:
>
>> Friday late afternoon works for me. :)
>>
>> On Sun, Dec 7, 2014 at 8:30 PM, Paris Carbone <[hidden email]> wrote:
>>
>>> +1
>>>
>>> I agree it will be nice to discuss our upcoming directions and ideas with
>>> anyone that is interested to give feedback.
>>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
>> (GMT+1).
>>> I hope it is cross-atlantic friendly enough. Is that generally ok for the
>>> rest?
>>>
>>> Paris
>>>
>>> PS: thanks for the hilarious introduction of the windowing semantics
>>> everyone :)
>>>
>>> ________________________________________
>>> From: Kostas Tzoumas [[hidden email]]
>>> Sent: Sunday, December 07, 2014 1:23 PM
>>> To: [hidden email]
>>> Subject: Re: Streaming windowing rework
>>>
>>> +1 great idea
>>>
>>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]>
>> wrote:
>>>
>>>> Hey Henry,
>>>>
>>>> (I am resending this mail because it seems it didnt make it through the
>>>> mail system)
>>>>
>>>> I suggest to have a public hangout sometimes on the current work we are
>>>> doing and our plans on streaming.
>>>>
>>>> We have actually put together a list of topics that we want to work on
>>> next
>>>> year as we will have a quite large Flink team at SICS from January:
>>> Paris,
>>>> Jonas, Me and later Vasia.
>>>>
>>>> Basically the topics are:
>>>> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
>>>> thesis topic)
>>>> - Different fault tolerance approaches
>>>> - Connecting batch and streaming api ('lambda')
>>>> - Streaming graph algorithms
>>>> - Samoa for Flink (and streaming machine learning in general)
>>>>
>>>> So lets start a series of public hangout calls to discuss these
>> topics, I
>>>> would suggest one sometimes next week when it is suitable.
>>>>
>>>> Regards,
>>>> Gyula
>>>>
>>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
>> [hidden email]>
>>>> wrote:
>>>>
>>>>> We actually need to bring these work come to light early so we could
>>>>> give feedback before getting to deep.
>>>>>
>>>>> Marton, could you encourage Paris and Jonas to publish or share their
>>>>> thoughts about the work they are doing?
>>>>>
>>>>> Would love to give some feedbacks.
>>>>>
>>>>> - Henry
>>>>>
>>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
>>>
>>>>> wrote:
>>>>>> Super!
>>>>>>
>>>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
>>>>> visible
>>>>>> in the mailing list) recently. Looking forward to seeing this
>> coming
>>>> in.
>>>>>>
>>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
>>> [hidden email]
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Nice! Looking forward to it
>>>>>>>
>>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
>>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>> Hey,
>>>>>>>>
>>>>>>>> As you may know in some European countries Santa Claus comes on
>>> the
>>>>> 6th
>>>>>>> of
>>>>>>>> December, Saint Nicolas Day - from Finland.
>>>>>>>>
>>>>>>>> As Flink is not yet very active in Finland our Sweden team
>> stepped
>>>> in
>>>>> and
>>>>>>>> gave us a big present with a huge rework on the windowing
>>> semantics
>>>> -
>>>>>>> this
>>>>>>>> is the implementation of the ideas that were posed the Stockholm
>>>>>>> hackathon
>>>>>>>> in October.
>>>>>>>>
>>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a
>> detailed
>>>>> intro
>>>>>>> to
>>>>>>>> the new features here soon.
>>>>>>>>
>>>>>>>> Marton
>>>>>>>
>>>>>
>>>>
>>>
>>

Henry Saputra

Re: Streaming windowing rework

In reply to this post by Paris Carbone

Thanks Gyula,

Just checking it will be 8:30am PST?

To make it more productive please have some kind of simple design doc or
proposal with preso style to help us discuss each proposed work.

Following the apache way, we then need to create Jira ticket and attach it
for other whom cannot attend the hangout to review and chime in.
Any Final decisions need to happen in dev list via emails or comments in
Jira.

On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:

> +1
>
> I agree it will be nice to discuss our upcoming directions and ideas with
> anyone that is interested to give feedback.
> One suggested time for the hangout is next Friday 12 Dec. 17:30pm (GMT+1).
> I hope it is cross-atlantic friendly enough. Is that generally ok for the
> rest?
>
> Paris
>
> PS: thanks for the hilarious introduction of the windowing semantics
> everyone :)
>
> ________________________________________
> From: Kostas Tzoumas [[hidden email] <javascript:;>]
> Sent: Sunday, December 07, 2014 1:23 PM
> To: [hidden email] <javascript:;>
> Subject: Re: Streaming windowing rework
>
> +1 great idea
>
> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
> <javascript:;>> wrote:
>
> > Hey Henry,
> >
> > (I am resending this mail because it seems it didnt make it through the
> > mail system)
> >
> > I suggest to have a public hangout sometimes on the current work we are
> > doing and our plans on streaming.
> >
> > We have actually put together a list of topics that we want to work on
> next
> > year as we will have a quite large Flink team at SICS from January:
> Paris,
> > Jonas, Me and later Vasia.
> >
> > Basically the topics are:
> > - Windowing semantics and multiquery optimisations (thats mainly Jonas'
> > thesis topic)
> > - Different fault tolerance approaches
> > - Connecting batch and streaming api ('lambda')
> > - Streaming graph algorithms
> > - Samoa for Flink (and streaming machine learning in general)
> >
> > So lets start a series of public hangout calls to discuss these topics, I
> > would suggest one sometimes next week when it is suitable.
> >
> > Regards,
> > Gyula
> >
> > On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <[hidden email]
> <javascript:;>>
> > wrote:
> >
> > > We actually need to bring these work come to light early so we could
> > > give feedback before getting to deep.
> > >
> > > Marton, could you encourage Paris and Jonas to publish or share their
> > > thoughts about the work they are doing?
> > >
> > > Would love to give some feedbacks.
> > >
> > > - Henry
> > >
> > > On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
> <javascript:;>>
> > > wrote:
> > > > Super!
> > > >
> > > > Paris and Jonas have been doing a lot of "silent" work (= not very
> > > visible
> > > > in the mailing list) recently. Looking forward to seeing this coming
> > in.
> > > >
> > > > On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> [hidden email] <javascript:;>
> > >
> > > > wrote:
> > > >
> > > >> Nice! Looking forward to it
> > > >>
> > > >> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> > > [hidden email] <javascript:;>>
> > > >> wrote:
> > > >> > Hey,
> > > >> >
> > > >> > As you may know in some European countries Santa Claus comes on
> the
> > > 6th
> > > >> of
> > > >> > December, Saint Nicolas Day - from Finland.
> > > >> >
> > > >> > As Flink is not yet very active in Finland our Sweden team stepped
> > in
> > > and
> > > >> > gave us a big present with a huge rework on the windowing
> semantics
> > -
> > > >> this
> > > >> > is the implementation of the ideas that were posed the Stockholm
> > > >> hackathon
> > > >> > in October.
> > > >> >
> > > >> > Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
> > > intro
> > > >> to
> > > >> > the new features here soon.
> > > >> >
> > > >> > Marton
> > > >>
> > >
> >
>

Asterios Katsifodimos

Re: Streaming windowing rework

In reply to this post by Paris Carbone

Great work, +1 for Friday!

Cheers,
Asterios

> On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]> wrote:
>
> Thanks Gyula,
>
> Just checking it will be 8:30am PST?
>
> To make it more productive please have some kind of simple design doc or
> proposal with preso style to help us discuss each proposed work.
>
> Following the apache way, we then need to create Jira ticket and attach it
> for other whom cannot attend the hangout to review and chime in.
> Any Final decisions need to happen in dev list via emails or comments in
> Jira.
>
> On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
>
>> +1
>>
>> I agree it will be nice to discuss our upcoming directions and ideas with
>> anyone that is interested to give feedback.
>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm (GMT+1).
>> I hope it is cross-atlantic friendly enough. Is that generally ok for the
>> rest?
>>
>> Paris
>>
>> PS: thanks for the hilarious introduction of the windowing semantics
>> everyone :)
>>
>> ________________________________________
>> From: Kostas Tzoumas [[hidden email] <javascript:;>]
>> Sent: Sunday, December 07, 2014 1:23 PM
>> To: [hidden email] <javascript:;>
>> Subject: Re: Streaming windowing rework
>>
>> +1 great idea
>>
>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
>> <javascript:;>> wrote:
>>
>>> Hey Henry,
>>>
>>> (I am resending this mail because it seems it didnt make it through the
>>> mail system)
>>>
>>> I suggest to have a public hangout sometimes on the current work we are
>>> doing and our plans on streaming.
>>>
>>> We have actually put together a list of topics that we want to work on
>> next
>>> year as we will have a quite large Flink team at SICS from January:
>> Paris,
>>> Jonas, Me and later Vasia.
>>>
>>> Basically the topics are:
>>> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
>>> thesis topic)
>>> - Different fault tolerance approaches
>>> - Connecting batch and streaming api ('lambda')
>>> - Streaming graph algorithms
>>> - Samoa for Flink (and streaming machine learning in general)
>>>
>>> So lets start a series of public hangout calls to discuss these topics, I
>>> would suggest one sometimes next week when it is suitable.
>>>
>>> Regards,
>>> Gyula
>>>
>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <[hidden email]
>> <javascript:;>>
>>> wrote:
>>>
>>>> We actually need to bring these work come to light early so we could
>>>> give feedback before getting to deep.
>>>>
>>>> Marton, could you encourage Paris and Jonas to publish or share their
>>>> thoughts about the work they are doing?
>>>>
>>>> Would love to give some feedbacks.
>>>>
>>>> - Henry
>>>>
>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
>> <javascript:;>>
>>>> wrote:
>>>>> Super!
>>>>>
>>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
>>>> visible
>>>>> in the mailing list) recently. Looking forward to seeing this coming
>>> in.
>>>>>
>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
>> [hidden email] <javascript:;>
>>>>
>>>>> wrote:
>>>>>
>>>>>> Nice! Looking forward to it
>>>>>>
>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
>>>> [hidden email] <javascript:;>>
>>>>>> wrote:
>>>>>>> Hey,
>>>>>>>
>>>>>>> As you may know in some European countries Santa Claus comes on
>> the
>>>> 6th
>>>>>> of
>>>>>>> December, Saint Nicolas Day - from Finland.
>>>>>>>
>>>>>>> As Flink is not yet very active in Finland our Sweden team stepped
>>> in
>>>> and
>>>>>>> gave us a big present with a huge rework on the windowing
>> semantics
>>> -
>>>>>> this
>>>>>>> is the implementation of the ideas that were posed the Stockholm
>>>>>> hackathon
>>>>>>> in October.
>>>>>>>
>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
>>>> intro
>>>>>> to
>>>>>>> the new features here soon.
>>>>>>>
>>>>>>> Marton
>>>>>>
>>>>
>>>
>>

Gianmarco De Francisci Morales

Re: Streaming windowing rework

Hi,
I'm one of the developers of SAMOA and I'd be happy to participate as well.
How should I connect?

Thanks,

--
Gianmarco

On 8 December 2014 at 09:28, Asterios Katsifodimos <
[hidden email]> wrote:

> Great work, +1 for Friday!
>
> Cheers,
> Asterios
> > On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]> wrote:
> >
> > Thanks Gyula,
> >
> > Just checking it will be 8:30am PST?
> >
> > To make it more productive please have some kind of simple design doc or
> > proposal with preso style to help us discuss each proposed work.
> >
> > Following the apache way, we then need to create Jira ticket and attach
> it
> > for other whom cannot attend the hangout to review and chime in.
> > Any Final decisions need to happen in dev list via emails or comments in
> > Jira.
> >
> > On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
> >
> >> +1
> >>
> >> I agree it will be nice to discuss our upcoming directions and ideas
> with
> >> anyone that is interested to give feedback.
> >> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
> (GMT+1).
> >> I hope it is cross-atlantic friendly enough. Is that generally ok for
> the
> >> rest?
> >>
> >> Paris
> >>
> >> PS: thanks for the hilarious introduction of the windowing semantics
> >> everyone :)
> >>
> >> ________________________________________
> >> From: Kostas Tzoumas [[hidden email] <javascript:;>]
> >> Sent: Sunday, December 07, 2014 1:23 PM
> >> To: [hidden email] <javascript:;>
> >> Subject: Re: Streaming windowing rework
> >>
> >> +1 great idea
> >>
> >> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
> >> <javascript:;>> wrote:
> >>
> >>> Hey Henry,
> >>>
> >>> (I am resending this mail because it seems it didnt make it through the
> >>> mail system)
> >>>
> >>> I suggest to have a public hangout sometimes on the current work we are
> >>> doing and our plans on streaming.
> >>>
> >>> We have actually put together a list of topics that we want to work on
> >> next
> >>> year as we will have a quite large Flink team at SICS from January:
> >> Paris,
> >>> Jonas, Me and later Vasia.
> >>>
> >>> Basically the topics are:
> >>> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
> >>> thesis topic)
> >>> - Different fault tolerance approaches
> >>> - Connecting batch and streaming api ('lambda')
> >>> - Streaming graph algorithms
> >>> - Samoa for Flink (and streaming machine learning in general)
> >>>
> >>> So lets start a series of public hangout calls to discuss these
> topics, I
> >>> would suggest one sometimes next week when it is suitable.
> >>>
> >>> Regards,
> >>> Gyula
> >>>
> >>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
> [hidden email]
> >> <javascript:;>>
> >>> wrote:
> >>>
> >>>> We actually need to bring these work come to light early so we could
> >>>> give feedback before getting to deep.
> >>>>
> >>>> Marton, could you encourage Paris and Jonas to publish or share their
> >>>> thoughts about the work they are doing?
> >>>>
> >>>> Would love to give some feedbacks.
> >>>>
> >>>> - Henry
> >>>>
> >>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
> >> <javascript:;>>
> >>>> wrote:
> >>>>> Super!
> >>>>>
> >>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
> >>>> visible
> >>>>> in the mailing list) recently. Looking forward to seeing this coming
> >>> in.
> >>>>>
> >>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> >> [hidden email] <javascript:;>
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Nice! Looking forward to it
> >>>>>>
> >>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> >>>> [hidden email] <javascript:;>>
> >>>>>> wrote:
> >>>>>>> Hey,
> >>>>>>>
> >>>>>>> As you may know in some European countries Santa Claus comes on
> >> the
> >>>> 6th
> >>>>>> of
> >>>>>>> December, Saint Nicolas Day - from Finland.
> >>>>>>>
> >>>>>>> As Flink is not yet very active in Finland our Sweden team stepped
> >>> in
> >>>> and
> >>>>>>> gave us a big present with a huge rework on the windowing
> >> semantics
> >>> -
> >>>>>> this
> >>>>>>> is the implementation of the ideas that were posed the Stockholm
> >>>>>> hackathon
> >>>>>>> in October.
> >>>>>>>
> >>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
> >>>> intro
> >>>>>> to
> >>>>>>> the new features here soon.
> >>>>>>>
> >>>>>>> Marton
> >>>>>>
> >>>>
> >>>
> >>
>
>

Gyula Fóra

Re: Streaming windowing rework

Hey,

I’m glad that you want to join. We will post a hangout link here later :)

Can someone please help me with setting these things up?

Cheers,
Gyula

> On 08 Dec 2014, at 11:16, Gianmarco De Francisci Morales <[hidden email]> wrote:
>
> Hi,
> I'm one of the developers of SAMOA and I'd be happy to participate as well.
> How should I connect?
>
> Thanks,
>
>
> --
> Gianmarco
>
> On 8 December 2014 at 09:28, Asterios Katsifodimos <
> [hidden email]> wrote:
>
>> Great work, +1 for Friday!
>>
>> Cheers,
>> Asterios
>>> On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]> wrote:
>>>
>>> Thanks Gyula,
>>>
>>> Just checking it will be 8:30am PST?
>>>
>>> To make it more productive please have some kind of simple design doc or
>>> proposal with preso style to help us discuss each proposed work.
>>>
>>> Following the apache way, we then need to create Jira ticket and attach
>> it
>>> for other whom cannot attend the hangout to review and chime in.
>>> Any Final decisions need to happen in dev list via emails or comments in
>>> Jira.
>>>
>>> On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
>>>
>>>> +1
>>>>
>>>> I agree it will be nice to discuss our upcoming directions and ideas
>> with
>>>> anyone that is interested to give feedback.
>>>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
>> (GMT+1).
>>>> I hope it is cross-atlantic friendly enough. Is that generally ok for
>> the
>>>> rest?
>>>>
>>>> Paris
>>>>
>>>> PS: thanks for the hilarious introduction of the windowing semantics
>>>> everyone :)
>>>>
>>>> ________________________________________
>>>> From: Kostas Tzoumas [[hidden email] <javascript:;>]
>>>> Sent: Sunday, December 07, 2014 1:23 PM
>>>> To: [hidden email] <javascript:;>
>>>> Subject: Re: Streaming windowing rework
>>>>
>>>> +1 great idea
>>>>
>>>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
>>>> <javascript:;>> wrote:
>>>>
>>>>> Hey Henry,
>>>>>
>>>>> (I am resending this mail because it seems it didnt make it through the
>>>>> mail system)
>>>>>
>>>>> I suggest to have a public hangout sometimes on the current work we are
>>>>> doing and our plans on streaming.
>>>>>
>>>>> We have actually put together a list of topics that we want to work on
>>>> next
>>>>> year as we will have a quite large Flink team at SICS from January:
>>>> Paris,
>>>>> Jonas, Me and later Vasia.
>>>>>
>>>>> Basically the topics are:
>>>>> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
>>>>> thesis topic)
>>>>> - Different fault tolerance approaches
>>>>> - Connecting batch and streaming api ('lambda')
>>>>> - Streaming graph algorithms
>>>>> - Samoa for Flink (and streaming machine learning in general)
>>>>>
>>>>> So lets start a series of public hangout calls to discuss these
>> topics, I
>>>>> would suggest one sometimes next week when it is suitable.
>>>>>
>>>>> Regards,
>>>>> Gyula
>>>>>
>>>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
>> [hidden email]
>>>> <javascript:;>>
>>>>> wrote:
>>>>>
>>>>>> We actually need to bring these work come to light early so we could
>>>>>> give feedback before getting to deep.
>>>>>>
>>>>>> Marton, could you encourage Paris and Jonas to publish or share their
>>>>>> thoughts about the work they are doing?
>>>>>>
>>>>>> Would love to give some feedbacks.
>>>>>>
>>>>>> - Henry
>>>>>>
>>>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
>>>> <javascript:;>>
>>>>>> wrote:
>>>>>>> Super!
>>>>>>>
>>>>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
>>>>>> visible
>>>>>>> in the mailing list) recently. Looking forward to seeing this coming
>>>>> in.
>>>>>>>
>>>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
>>>> [hidden email] <javascript:;>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Nice! Looking forward to it
>>>>>>>>
>>>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
>>>>>> [hidden email] <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>> Hey,
>>>>>>>>>
>>>>>>>>> As you may know in some European countries Santa Claus comes on
>>>> the
>>>>>> 6th
>>>>>>>> of
>>>>>>>>> December, Saint Nicolas Day - from Finland.
>>>>>>>>>
>>>>>>>>> As Flink is not yet very active in Finland our Sweden team stepped
>>>>> in
>>>>>> and
>>>>>>>>> gave us a big present with a huge rework on the windowing
>>>> semantics
>>>>> -
>>>>>>>> this
>>>>>>>>> is the implementation of the ideas that were posed the Stockholm
>>>>>>>> hackathon
>>>>>>>>> in October.
>>>>>>>>>
>>>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
>>>>>> intro
>>>>>>>> to
>>>>>>>>> the new features here soon.
>>>>>>>>>
>>>>>>>>> Marton
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>

Paris Carbone-2

Re: Streaming windowing rework

Hey,

@Gyula I can help with the discussion content doc
@Gianmarco It will be great to have you! We will also keep you posted on the Samoa integration.
@Henry, correct, it’s 8:30am PST.

cheers
Paris

> On 08 Dec 2014, at 11:34, Gyula Fora <[hidden email]> wrote:
>
> Hey,
>
> I’m glad that you want to join. We will post a hangout link here later :)
>
> Can someone please help me with setting these things up?
>
> Cheers,
> Gyula
>
>> On 08 Dec 2014, at 11:16, Gianmarco De Francisci Morales <[hidden email]> wrote:
>>
>> Hi,
>> I'm one of the developers of SAMOA and I'd be happy to participate as well.
>> How should I connect?
>>
>> Thanks,
>>
>>
>> --
>> Gianmarco
>>
>> On 8 December 2014 at 09:28, Asterios Katsifodimos <
>> [hidden email]> wrote:
>>
>>> Great work, +1 for Friday!
>>>
>>> Cheers,
>>> Asterios
>>>> On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]> wrote:
>>>>
>>>> Thanks Gyula,
>>>>
>>>> Just checking it will be 8:30am PST?
>>>>
>>>> To make it more productive please have some kind of simple design doc or
>>>> proposal with preso style to help us discuss each proposed work.
>>>>
>>>> Following the apache way, we then need to create Jira ticket and attach
>>> it
>>>> for other whom cannot attend the hangout to review and chime in.
>>>> Any Final decisions need to happen in dev list via emails or comments in
>>>> Jira.
>>>>
>>>> On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> I agree it will be nice to discuss our upcoming directions and ideas
>>> with
>>>>> anyone that is interested to give feedback.
>>>>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
>>> (GMT+1).
>>>>> I hope it is cross-atlantic friendly enough. Is that generally ok for
>>> the
>>>>> rest?
>>>>>
>>>>> Paris
>>>>>
>>>>> PS: thanks for the hilarious introduction of the windowing semantics
>>>>> everyone :)
>>>>>
>>>>> ________________________________________
>>>>> From: Kostas Tzoumas [[hidden email] <javascript:;>]
>>>>> Sent: Sunday, December 07, 2014 1:23 PM
>>>>> To: [hidden email] <javascript:;>
>>>>> Subject: Re: Streaming windowing rework
>>>>>
>>>>> +1 great idea
>>>>>
>>>>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
>>>>> <javascript:;>> wrote:
>>>>>
>>>>>> Hey Henry,
>>>>>>
>>>>>> (I am resending this mail because it seems it didnt make it through the
>>>>>> mail system)
>>>>>>
>>>>>> I suggest to have a public hangout sometimes on the current work we are
>>>>>> doing and our plans on streaming.
>>>>>>
>>>>>> We have actually put together a list of topics that we want to work on
>>>>> next
>>>>>> year as we will have a quite large Flink team at SICS from January:
>>>>> Paris,
>>>>>> Jonas, Me and later Vasia.
>>>>>>
>>>>>> Basically the topics are:
>>>>>> - Windowing semantics and multiquery optimisations (thats mainly Jonas'
>>>>>> thesis topic)
>>>>>> - Different fault tolerance approaches
>>>>>> - Connecting batch and streaming api ('lambda')
>>>>>> - Streaming graph algorithms
>>>>>> - Samoa for Flink (and streaming machine learning in general)
>>>>>>
>>>>>> So lets start a series of public hangout calls to discuss these
>>> topics, I
>>>>>> would suggest one sometimes next week when it is suitable.
>>>>>>
>>>>>> Regards,
>>>>>> Gyula
>>>>>>
>>>>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
>>> [hidden email]
>>>>> <javascript:;>>
>>>>>> wrote:
>>>>>>
>>>>>>> We actually need to bring these work come to light early so we could
>>>>>>> give feedback before getting to deep.
>>>>>>>
>>>>>>> Marton, could you encourage Paris and Jonas to publish or share their
>>>>>>> thoughts about the work they are doing?
>>>>>>>
>>>>>>> Would love to give some feedbacks.
>>>>>>>
>>>>>>> - Henry
>>>>>>>
>>>>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <[hidden email]
>>>>> <javascript:;>>
>>>>>>> wrote:
>>>>>>>> Super!
>>>>>>>>
>>>>>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
>>>>>>> visible
>>>>>>>> in the mailing list) recently. Looking forward to seeing this coming
>>>>>> in.
>>>>>>>>
>>>>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
>>>>> [hidden email] <javascript:;>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Nice! Looking forward to it
>>>>>>>>>
>>>>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>> Hey,
>>>>>>>>>>
>>>>>>>>>> As you may know in some European countries Santa Claus comes on
>>>>> the
>>>>>>> 6th
>>>>>>>>> of
>>>>>>>>>> December, Saint Nicolas Day - from Finland.
>>>>>>>>>>
>>>>>>>>>> As Flink is not yet very active in Finland our Sweden team stepped
>>>>>> in
>>>>>>> and
>>>>>>>>>> gave us a big present with a huge rework on the windowing
>>>>> semantics
>>>>>> -
>>>>>>>>> this
>>>>>>>>>> is the implementation of the ideas that were posed the Stockholm
>>>>>>>>> hackathon
>>>>>>>>>> in October.
>>>>>>>>>>
>>>>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a detailed
>>>>>>> intro
>>>>>>>>> to
>>>>>>>>>> the new features here soon.
>>>>>>>>>>
>>>>>>>>>> Marton
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>>
>

Stephan Ewen

Re: Streaming windowing rework

I have to see whether I can make it, I might still be on my way back from
Amsterdam (giving a talk at the Netherlands Hadoop User Group). I might be
a bit late and only join at 6pm (CET) / 9am (PST)

Greetings,
Stephan

On Mon, Dec 8, 2014 at 12:22 PM, Paris Carbone <[hidden email]>
wrote:

> Hey,
>
> @Gyula I can help with the discussion content doc
> @Gianmarco It will be great to have you! We will also keep you posted on
> the Samoa integration.
> @Henry, correct, it’s 8:30am PST.
>
> cheers
> Paris
>
>
> > On 08 Dec 2014, at 11:34, Gyula Fora <[hidden email]> wrote:
> >
> > Hey,
> >
> > I’m glad that you want to join. We will post a hangout link here later :)
> >
> > Can someone please help me with setting these things up?
> >
> > Cheers,
> > Gyula
> >
> >> On 08 Dec 2014, at 11:16, Gianmarco De Francisci Morales <
> [hidden email]> wrote:
> >>
> >> Hi,
> >> I'm one of the developers of SAMOA and I'd be happy to participate as
> well.
> >> How should I connect?
> >>
> >> Thanks,
> >>
> >>
> >> --
> >> Gianmarco
> >>
> >> On 8 December 2014 at 09:28, Asterios Katsifodimos <
> >> [hidden email]> wrote:
> >>
> >>> Great work, +1 for Friday!
> >>>
> >>> Cheers,
> >>> Asterios
> >>>> On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]>
> wrote:
> >>>>
> >>>> Thanks Gyula,
> >>>>
> >>>> Just checking it will be 8:30am PST?
> >>>>
> >>>> To make it more productive please have some kind of simple design doc
> or
> >>>> proposal with preso style to help us discuss each proposed work.
> >>>>
> >>>> Following the apache way, we then need to create Jira ticket and
> attach
> >>> it
> >>>> for other whom cannot attend the hangout to review and chime in.
> >>>> Any Final decisions need to happen in dev list via emails or comments
> in
> >>>> Jira.
> >>>>
> >>>> On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
> >>>>
> >>>>> +1
> >>>>>
> >>>>> I agree it will be nice to discuss our upcoming directions and ideas
> >>> with
> >>>>> anyone that is interested to give feedback.
> >>>>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
> >>> (GMT+1).
> >>>>> I hope it is cross-atlantic friendly enough. Is that generally ok for
> >>> the
> >>>>> rest?
> >>>>>
> >>>>> Paris
> >>>>>
> >>>>> PS: thanks for the hilarious introduction of the windowing semantics
> >>>>> everyone :)
> >>>>>
> >>>>> ________________________________________
> >>>>> From: Kostas Tzoumas [[hidden email] <javascript:;>]
> >>>>> Sent: Sunday, December 07, 2014 1:23 PM
> >>>>> To: [hidden email] <javascript:;>
> >>>>> Subject: Re: Streaming windowing rework
> >>>>>
> >>>>> +1 great idea
> >>>>>
> >>>>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
> >>>>> <javascript:;>> wrote:
> >>>>>
> >>>>>> Hey Henry,
> >>>>>>
> >>>>>> (I am resending this mail because it seems it didnt make it through
> the
> >>>>>> mail system)
> >>>>>>
> >>>>>> I suggest to have a public hangout sometimes on the current work we
> are
> >>>>>> doing and our plans on streaming.
> >>>>>>
> >>>>>> We have actually put together a list of topics that we want to work
> on
> >>>>> next
> >>>>>> year as we will have a quite large Flink team at SICS from January:
> >>>>> Paris,
> >>>>>> Jonas, Me and later Vasia.
> >>>>>>
> >>>>>> Basically the topics are:
> >>>>>> - Windowing semantics and multiquery optimisations (thats mainly
> Jonas'
> >>>>>> thesis topic)
> >>>>>> - Different fault tolerance approaches
> >>>>>> - Connecting batch and streaming api ('lambda')
> >>>>>> - Streaming graph algorithms
> >>>>>> - Samoa for Flink (and streaming machine learning in general)
> >>>>>>
> >>>>>> So lets start a series of public hangout calls to discuss these
> >>> topics, I
> >>>>>> would suggest one sometimes next week when it is suitable.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Gyula
> >>>>>>
> >>>>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
> >>> [hidden email]
> >>>>> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> We actually need to bring these work come to light early so we
> could
> >>>>>>> give feedback before getting to deep.
> >>>>>>>
> >>>>>>> Marton, could you encourage Paris and Jonas to publish or share
> their
> >>>>>>> thoughts about the work they are doing?
> >>>>>>>
> >>>>>>> Would love to give some feedbacks.
> >>>>>>>
> >>>>>>> - Henry
> >>>>>>>
> >>>>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <
> [hidden email]
> >>>>> <javascript:;>>
> >>>>>>> wrote:
> >>>>>>>> Super!
> >>>>>>>>
> >>>>>>>> Paris and Jonas have been doing a lot of "silent" work (= not very
> >>>>>>> visible
> >>>>>>>> in the mailing list) recently. Looking forward to seeing this
> coming
> >>>>>> in.
> >>>>>>>>
> >>>>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> >>>>> [hidden email] <javascript:;>
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Nice! Looking forward to it
> >>>>>>>>>
> >>>>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> >>>>>>> [hidden email] <javascript:;>>
> >>>>>>>>> wrote:
> >>>>>>>>>> Hey,
> >>>>>>>>>>
> >>>>>>>>>> As you may know in some European countries Santa Claus comes on
> >>>>> the
> >>>>>>> 6th
> >>>>>>>>> of
> >>>>>>>>>> December, Saint Nicolas Day - from Finland.
> >>>>>>>>>>
> >>>>>>>>>> As Flink is not yet very active in Finland our Sweden team
> stepped
> >>>>>> in
> >>>>>>> and
> >>>>>>>>>> gave us a big present with a huge rework on the windowing
> >>>>> semantics
> >>>>>> -
> >>>>>>>>> this
> >>>>>>>>>> is the implementation of the ideas that were posed the Stockholm
> >>>>>>>>> hackathon
> >>>>>>>>>> in October.
> >>>>>>>>>>
> >>>>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a
> detailed
> >>>>>>> intro
> >>>>>>>>> to
> >>>>>>>>>> the new features here soon.
> >>>>>>>>>>
> >>>>>>>>>> Marton
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >
>
>

Jonas Traub

Re: Streaming windowing rework

The proposed time on Friday is fine for me as well. Both 9 PST and 8:30 PST
would work for me.

Greetings
Jonas

2014-12-08 15:58 GMT+01:00 Stephan Ewen <[hidden email]>:

> I have to see whether I can make it, I might still be on my way back from
> Amsterdam (giving a talk at the Netherlands Hadoop User Group). I might be
> a bit late and only join at 6pm (CET) / 9am (PST)
>
> Greetings,
> Stephan
>
> On Mon, Dec 8, 2014 at 12:22 PM, Paris Carbone <[hidden email]>
> wrote:
>
> > Hey,
> >
> > @Gyula I can help with the discussion content doc
> > @Gianmarco It will be great to have you! We will also keep you posted on
> > the Samoa integration.
> > @Henry, correct, it’s 8:30am PST.
> >
> > cheers
> > Paris
> >
> >
> > > On 08 Dec 2014, at 11:34, Gyula Fora <[hidden email]> wrote:
> > >
> > > Hey,
> > >
> > > I’m glad that you want to join. We will post a hangout link here later
> :)
> > >
> > > Can someone please help me with setting these things up?
> > >
> > > Cheers,
> > > Gyula
> > >
> > >> On 08 Dec 2014, at 11:16, Gianmarco De Francisci Morales <
> > [hidden email]> wrote:
> > >>
> > >> Hi,
> > >> I'm one of the developers of SAMOA and I'd be happy to participate as
> > well.
> > >> How should I connect?
> > >>
> > >> Thanks,
> > >>
> > >>
> > >> --
> > >> Gianmarco
> > >>
> > >> On 8 December 2014 at 09:28, Asterios Katsifodimos <
> > >> [hidden email]> wrote:
> > >>
> > >>> Great work, +1 for Friday!
> > >>>
> > >>> Cheers,
> > >>> Asterios
> > >>>> On 08 Dec 2014, at 00:31, Henry Saputra <[hidden email]>
> > wrote:
> > >>>>
> > >>>> Thanks Gyula,
> > >>>>
> > >>>> Just checking it will be 8:30am PST?
> > >>>>
> > >>>> To make it more productive please have some kind of simple design
> doc
> > or
> > >>>> proposal with preso style to help us discuss each proposed work.
> > >>>>
> > >>>> Following the apache way, we then need to create Jira ticket and
> > attach
> > >>> it
> > >>>> for other whom cannot attend the hangout to review and chime in.
> > >>>> Any Final decisions need to happen in dev list via emails or
> comments
> > in
> > >>>> Jira.
> > >>>>
> > >>>> On Sunday, December 7, 2014, Paris Carbone <[hidden email]> wrote:
> > >>>>
> > >>>>> +1
> > >>>>>
> > >>>>> I agree it will be nice to discuss our upcoming directions and
> ideas
> > >>> with
> > >>>>> anyone that is interested to give feedback.
> > >>>>> One suggested time for the hangout is next Friday 12 Dec. 17:30pm
> > >>> (GMT+1).
> > >>>>> I hope it is cross-atlantic friendly enough. Is that generally ok
> for
> > >>> the
> > >>>>> rest?
> > >>>>>
> > >>>>> Paris
> > >>>>>
> > >>>>> PS: thanks for the hilarious introduction of the windowing
> semantics
> > >>>>> everyone :)
> > >>>>>
> > >>>>> ________________________________________
> > >>>>> From: Kostas Tzoumas [[hidden email] <javascript:;>]
> > >>>>> Sent: Sunday, December 07, 2014 1:23 PM
> > >>>>> To: [hidden email] <javascript:;>
> > >>>>> Subject: Re: Streaming windowing rework
> > >>>>>
> > >>>>> +1 great idea
> > >>>>>
> > >>>>> On Sun, Dec 7, 2014 at 10:35 AM, Gyula Fóra <[hidden email]
> > >>>>> <javascript:;>> wrote:
> > >>>>>
> > >>>>>> Hey Henry,
> > >>>>>>
> > >>>>>> (I am resending this mail because it seems it didnt make it
> through
> > the
> > >>>>>> mail system)
> > >>>>>>
> > >>>>>> I suggest to have a public hangout sometimes on the current work
> we
> > are
> > >>>>>> doing and our plans on streaming.
> > >>>>>>
> > >>>>>> We have actually put together a list of topics that we want to
> work
> > on
> > >>>>> next
> > >>>>>> year as we will have a quite large Flink team at SICS from
> January:
> > >>>>> Paris,
> > >>>>>> Jonas, Me and later Vasia.
> > >>>>>>
> > >>>>>> Basically the topics are:
> > >>>>>> - Windowing semantics and multiquery optimisations (thats mainly
> > Jonas'
> > >>>>>> thesis topic)
> > >>>>>> - Different fault tolerance approaches
> > >>>>>> - Connecting batch and streaming api ('lambda')
> > >>>>>> - Streaming graph algorithms
> > >>>>>> - Samoa for Flink (and streaming machine learning in general)
> > >>>>>>
> > >>>>>> So lets start a series of public hangout calls to discuss these
> > >>> topics, I
> > >>>>>> would suggest one sometimes next week when it is suitable.
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> Gyula
> > >>>>>>
> > >>>>>> On Sat, Dec 6, 2014 at 12:56 AM, Henry Saputra <
> > >>> [hidden email]
> > >>>>> <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> We actually need to bring these work come to light early so we
> > could
> > >>>>>>> give feedback before getting to deep.
> > >>>>>>>
> > >>>>>>> Marton, could you encourage Paris and Jonas to publish or share
> > their
> > >>>>>>> thoughts about the work they are doing?
> > >>>>>>>
> > >>>>>>> Would love to give some feedbacks.
> > >>>>>>>
> > >>>>>>> - Henry
> > >>>>>>>
> > >>>>>>> On Fri, Dec 5, 2014 at 12:38 PM, Kostas Tzoumas <
> > [hidden email]
> > >>>>> <javascript:;>>
> > >>>>>>> wrote:
> > >>>>>>>> Super!
> > >>>>>>>>
> > >>>>>>>> Paris and Jonas have been doing a lot of "silent" work (= not
> very
> > >>>>>>> visible
> > >>>>>>>> in the mailing list) recently. Looking forward to seeing this
> > coming
> > >>>>>> in.
> > >>>>>>>>
> > >>>>>>>> On Fri, Dec 5, 2014 at 8:00 PM, Henry Saputra <
> > >>>>> [hidden email] <javascript:;>
> > >>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Nice! Looking forward to it
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Dec 5, 2014 at 9:34 AM, Márton Balassi <
> > >>>>>>> [hidden email] <javascript:;>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>> Hey,
> > >>>>>>>>>>
> > >>>>>>>>>> As you may know in some European countries Santa Claus comes
> on
> > >>>>> the
> > >>>>>>> 6th
> > >>>>>>>>> of
> > >>>>>>>>>> December, Saint Nicolas Day - from Finland.
> > >>>>>>>>>>
> > >>>>>>>>>> As Flink is not yet very active in Finland our Sweden team
> > stepped
> > >>>>>> in
> > >>>>>>> and
> > >>>>>>>>>> gave us a big present with a huge rework on the windowing
> > >>>>> semantics
> > >>>>>> -
> > >>>>>>>>> this
> > >>>>>>>>>> is the implementation of the ideas that were posed the
> Stockholm
> > >>>>>>>>> hackathon
> > >>>>>>>>>> in October.
> > >>>>>>>>>>
> > >>>>>>>>>> Big thanks to Jonas, Paris and Gyula. Jonas is posting a
> > detailed
> > >>>>>>> intro
> > >>>>>>>>> to
> > >>>>>>>>>> the new features here soon.
> > >>>>>>>>>>
> > >>>>>>>>>> Marton
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >
> >
> >
>