Motivation for Delta Iterations

[Trevor Grant]

I was wondering where the idea for delta iterations came from.

Many of the features in Flink have some sort of academic foundation, maybe
someone could point me to a paper?

I checked the JIRA and mailing list but couldn't find any relevant
discussion, so I apologize if I missed it.

Thanks in advance,

tg

Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*

[Till Rohrmann]

Hi Trevor,

delta iterations are described in this paper [1].

[1]
http://stratosphere.eu/assets/papers/spinningFastIterativeDataFlows_12.pdf

Cheers,
Till

On Thu, May 26, 2016 at 1:57 AM, Trevor Grant <[hidden email]>
wrote:

> I was wondering where the idea for delta iterations came from.
>
> Many of the features in Flink have some sort of academic foundation, maybe
> someone could point me to a paper?
>
> I checked the JIRA and mailing list but couldn't find any relevant
> discussion, so I apologize if I missed it.
>
> Thanks in advance,
>
> tg
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>

[Slim Baltagi]

Hi Trevor

A summary of this 12 pages academic paper: 'Spinning Fast Iterative Data Flows’ by Ewen and al published in 2012 http://vldb.org/pvldb/vol5/p1268_stephanewen_vldb2012.pdf <http://vldb.org/pvldb/vol5/p1268_stephanewen_vldb2012.pdf>
is done by Adrian Colyer and published on June 18, 2015  https://blog.acolyer.org/2015/06/18/spinning-fast-iterative-dataflows/ <https://blog.acolyer.org/2015/06/18/spinning-fast-iterative-dataflows/>

For a practical introduction to iterations and delta iterations in Flink, I’d like to suggest this presentation by Sameer Wadkar at our 2nd Washington DC Flink meetup on November 19, 2015
http://www.slideshare.net/wadkarsameer/flink-batchanditerationsv2 <http://www.slideshare.net/wadkarsameer/flink-batchanditerationsv2>

Good luck for your talk today May 26, 2016 at our Chicago Flink Meetup! http://www.meetup.com/Chicago-Apache-Flink-Meetup/events/231080374/ <http://www.meetup.com/Chicago-Apache-Flink-Meetup/events/231080374/>

I hope this helps.

Slim

> On May 26, 2016, at 2:46 AM, Till Rohrmann <[hidden email]> wrote:
>
> Hi Trevor,
>
> delta iterations are described in this paper [1].
>
> [1]
> http://stratosphere.eu/assets/papers/spinningFastIterativeDataFlows_12.pdf
>
> Cheers,
> Till
>
> On Thu, May 26, 2016 at 1:57 AM, Trevor Grant <[hidden email]>
> wrote:
>
>> I was wondering where the idea for delta iterations came from.
>>
>> Many of the features in Flink have some sort of academic foundation, maybe
>> someone could point me to a paper?
>>
>> I checked the JIRA and mailing list but couldn't find any relevant
>> discussion, so I apologize if I missed it.
>>
>> Thanks in advance,
>>
>> tg
>>
>> Trevor Grant
>> Data Scientist
>> https://github.com/rawkintrevo
>> http://stackexchange.com/users/3002022/rawkintrevo
>> http://trevorgrant.org
>>
>> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>>

[Trevor Grant]

Thanks Till and Slim.

Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*


On Thu, May 26, 2016 at 3:10 AM, Slim Baltagi <[hidden email]> wrote:

> Hi Trevor
>
> A summary of this 12 pages academic paper: 'Spinning Fast Iterative Data
> Flows’ by Ewen and al published in 2012
> http://vldb.org/pvldb/vol5/p1268_stephanewen_vldb2012.pdf <
> http://vldb.org/pvldb/vol5/p1268_stephanewen_vldb2012.pdf>
> is done by Adrian Colyer and published on June 18, 2015
> https://blog.acolyer.org/2015/06/18/spinning-fast-iterative-dataflows/ <
> https://blog.acolyer.org/2015/06/18/spinning-fast-iterative-dataflows/>
>
> For a practical introduction to iterations and delta iterations in Flink,
> I’d like to suggest this presentation by Sameer Wadkar at our 2nd
> Washington DC Flink meetup on November 19, 2015
> http://www.slideshare.net/wadkarsameer/flink-batchanditerationsv2 <
> http://www.slideshare.net/wadkarsameer/flink-batchanditerationsv2>
>
> Good luck for your talk today May 26, 2016 at our Chicago Flink Meetup!
> http://www.meetup.com/Chicago-Apache-Flink-Meetup/events/231080374/ <
> http://www.meetup.com/Chicago-Apache-Flink-Meetup/events/231080374/>
>
> I hope this helps.
>
> Slim
>
>
> > On May 26, 2016, at 2:46 AM, Till Rohrmann <[hidden email]> wrote:
> >
> > Hi Trevor,
> >
> > delta iterations are described in this paper [1].
> >
> > [1]
> >
> http://stratosphere.eu/assets/papers/spinningFastIterativeDataFlows_12.pdf
> >
> > Cheers,
> > Till
> >
> > On Thu, May 26, 2016 at 1:57 AM, Trevor Grant <[hidden email]>
> > wrote:
> >
> >> I was wondering where the idea for delta iterations came from.
> >>
> >> Many of the features in Flink have some sort of academic foundation,
> maybe
> >> someone could point me to a paper?
> >>
> >> I checked the JIRA and mailing list but couldn't find any relevant
> >> discussion, so I apologize if I missed it.
> >>
> >> Thanks in advance,
> >>
> >> tg
> >>
> >> Trevor Grant
> >> Data Scientist
> >> https://github.com/rawkintrevo
> >> http://stackexchange.com/users/3002022/rawkintrevo
> >> http://trevorgrant.org
> >>
> >> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
> >>
>
>