Hello everyone,
Last week at Strata, I attended a talk about a streaming machine learning framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The project is going through apache incubation application and currently runs on Storm, S4 and Samza. After the talk, I spoke with the guy that presented it and we both thought that it'd be cool to port Samoa on Flink. I'm writing about this here because I want to know your thoughts on this and also because I'm not very familiar with the Streaming-API, so I will need your input :) I have already discussed with Paris and there is interest to pick this up from some students at KTH as well. As far as I understand, Samoa expects an API similar to that of Storm and from there the integration should be pretty straight-forward. So, do you think it would be feasible and -most importantly- worth it? Cheers, Vasia. |
Hey Vasia,
I really like the idea and it seems we have some volunteers from our side already! It is in my opinion a starter task since it requires to create a simple adapter for flink as well as mapping abstractions to our streaming topology API in a similar fashion (in its simplest form just using several flatmaps). It will serve as a suitable common layer to make some comparisons between flink-streaming to storm, s4 and samoa on incremental machine learning tasks. Paris > On 27 Nov 2014, at 13:59, Vasiliki Kalavri <[hidden email]> wrote: > > Hello everyone, > > Last week at Strata, I attended a talk about a streaming machine learning > framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The > project is going through apache incubation application and currently runs > on Storm, S4 and Samza. > > After the talk, I spoke with the guy that presented it and we both thought > that it'd be cool to port Samoa on Flink. > > I'm writing about this here because I want to know your thoughts on this > and also because I'm not very familiar with the Streaming-API, so I will > need your input :) > I have already discussed with Paris and there is interest to pick this up > from some students at KTH as well. > As far as I understand, Samoa expects an API similar to that of Storm and > from there the integration should be pretty straight-forward. > > So, do you think it would be feasible and -most importantly- worth it? > > Cheers, > Vasia. |
Hey,
Thanks Vasia, great idea. When we've started the streaming development the first sketch API we had is was actually really similar to Storm's API a couple of versions back. I've talked to Gyula about the issue when we've first heard about SAMOA ago and we are confident that porting Samoa to Flink streaming shoul be definitely managable and really useful as Paris mentions. Luckily they have ASF2 license so the legal cooperation is clear. The rest is on us and fortunately Paris might have a candidate already to kick off the development. :) Cheers, Marton On Thu, Nov 27, 2014 at 2:05 PM, Paris Carbone <[hidden email]> wrote: > Hey Vasia, > > I really like the idea and it seems we have some volunteers from our side > already! > It is in my opinion a starter task since it requires to create a simple > adapter for flink as well as mapping abstractions to our streaming topology > API in a similar fashion (in its simplest form just using several flatmaps). > > It will serve as a suitable common layer to make some comparisons between > flink-streaming to storm, s4 and samoa on incremental machine learning > tasks. > > Paris > > > On 27 Nov 2014, at 13:59, Vasiliki Kalavri <[hidden email]> > wrote: > > > > Hello everyone, > > > > Last week at Strata, I attended a talk about a streaming machine learning > > framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The > > project is going through apache incubation application and currently runs > > on Storm, S4 and Samza. > > > > After the talk, I spoke with the guy that presented it and we both > thought > > that it'd be cool to port Samoa on Flink. > > > > I'm writing about this here because I want to know your thoughts on this > > and also because I'm not very familiar with the Streaming-API, so I will > > need your input :) > > I have already discussed with Paris and there is interest to pick this up > > from some students at KTH as well. > > As far as I understand, Samoa expects an API similar to that of Storm and > > from there the integration should be pretty straight-forward. > > > > So, do you think it would be feasible and -most importantly- worth it? > > > > Cheers, > > Vasia. > > |
FYI, Samoa was just proposed for Apache Incubation :-)
2014-11-27 15:19 GMT+01:00 Márton Balassi <[hidden email]>: > Hey, > > Thanks Vasia, great idea. > When we've started the streaming development the first sketch API we had is > was actually really similar to Storm's API a couple of versions back. I've > talked to Gyula about the issue when we've first heard about SAMOA ago and > we are confident that porting Samoa to Flink streaming shoul be definitely > managable and really useful as Paris mentions. > > Luckily they have ASF2 license so the legal cooperation is clear. The rest > is on us and fortunately Paris might have a candidate already to kick off > the development. :) > > Cheers, > > Marton > > On Thu, Nov 27, 2014 at 2:05 PM, Paris Carbone <[hidden email]> wrote: > > > Hey Vasia, > > > > I really like the idea and it seems we have some volunteers from our side > > already! > > It is in my opinion a starter task since it requires to create a simple > > adapter for flink as well as mapping abstractions to our streaming > topology > > API in a similar fashion (in its simplest form just using several > flatmaps). > > > > It will serve as a suitable common layer to make some comparisons between > > flink-streaming to storm, s4 and samoa on incremental machine learning > > tasks. > > > > Paris > > > > > On 27 Nov 2014, at 13:59, Vasiliki Kalavri <[hidden email]> > > wrote: > > > > > > Hello everyone, > > > > > > Last week at Strata, I attended a talk about a streaming machine > learning > > > framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The > > > project is going through apache incubation application and currently > runs > > > on Storm, S4 and Samza. > > > > > > After the talk, I spoke with the guy that presented it and we both > > thought > > > that it'd be cool to port Samoa on Flink. > > > > > > I'm writing about this here because I want to know your thoughts on > this > > > and also because I'm not very familiar with the Streaming-API, so I > will > > > need your input :) > > > I have already discussed with Paris and there is interest to pick this > up > > > from some students at KTH as well. > > > As far as I understand, Samoa expects an API similar to that of Storm > and > > > from there the integration should be pretty straight-forward. > > > > > > So, do you think it would be feasible and -most importantly- worth it? > > > > > > Cheers, > > > Vasia. > > > > > |
Great news! Good to be aligned :)
Paris > On 03 Dec 2014, at 10:24, Fabian Hueske <[hidden email]> wrote: > > FYI, Samoa was just proposed for Apache Incubation :-) > > 2014-11-27 15:19 GMT+01:00 Márton Balassi <[hidden email]>: > >> Hey, >> >> Thanks Vasia, great idea. >> When we've started the streaming development the first sketch API we had is >> was actually really similar to Storm's API a couple of versions back. I've >> talked to Gyula about the issue when we've first heard about SAMOA ago and >> we are confident that porting Samoa to Flink streaming shoul be definitely >> managable and really useful as Paris mentions. >> >> Luckily they have ASF2 license so the legal cooperation is clear. The rest >> is on us and fortunately Paris might have a candidate already to kick off >> the development. :) >> >> Cheers, >> >> Marton >> >> On Thu, Nov 27, 2014 at 2:05 PM, Paris Carbone <[hidden email]> wrote: >> >>> Hey Vasia, >>> >>> I really like the idea and it seems we have some volunteers from our side >>> already! >>> It is in my opinion a starter task since it requires to create a simple >>> adapter for flink as well as mapping abstractions to our streaming >> topology >>> API in a similar fashion (in its simplest form just using several >> flatmaps). >>> >>> It will serve as a suitable common layer to make some comparisons between >>> flink-streaming to storm, s4 and samoa on incremental machine learning >>> tasks. >>> >>> Paris >>> >>>> On 27 Nov 2014, at 13:59, Vasiliki Kalavri <[hidden email]> >>> wrote: >>>> >>>> Hello everyone, >>>> >>>> Last week at Strata, I attended a talk about a streaming machine >> learning >>>> framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The >>>> project is going through apache incubation application and currently >> runs >>>> on Storm, S4 and Samza. >>>> >>>> After the talk, I spoke with the guy that presented it and we both >>> thought >>>> that it'd be cool to port Samoa on Flink. >>>> >>>> I'm writing about this here because I want to know your thoughts on >> this >>>> and also because I'm not very familiar with the Streaming-API, so I >> will >>>> need your input :) >>>> I have already discussed with Paris and there is interest to pick this >> up >>>> from some students at KTH as well. >>>> As far as I understand, Samoa expects an API similar to that of Storm >> and >>>> from there the integration should be pretty straight-forward. >>>> >>>> So, do you think it would be feasible and -most importantly- worth it? >>>> >>>> Cheers, >>>> Vasia. >>> >>> >> |
FYI, we'll be working on this here:
https://github.com/senorcarbone/samoa/tree/flink Feel free to provide input :)) Cheers, V. On 3 December 2014 at 11:56, Paris Carbone <[hidden email]> wrote: > Great news! Good to be aligned :) > > Paris > > > On 03 Dec 2014, at 10:24, Fabian Hueske <[hidden email]> wrote: > > > > FYI, Samoa was just proposed for Apache Incubation :-) > > > > 2014-11-27 15:19 GMT+01:00 Márton Balassi <[hidden email]>: > > > >> Hey, > >> > >> Thanks Vasia, great idea. > >> When we've started the streaming development the first sketch API we > had is > >> was actually really similar to Storm's API a couple of versions back. > I've > >> talked to Gyula about the issue when we've first heard about SAMOA ago > and > >> we are confident that porting Samoa to Flink streaming shoul be > definitely > >> managable and really useful as Paris mentions. > >> > >> Luckily they have ASF2 license so the legal cooperation is clear. The > rest > >> is on us and fortunately Paris might have a candidate already to kick > off > >> the development. :) > >> > >> Cheers, > >> > >> Marton > >> > >> On Thu, Nov 27, 2014 at 2:05 PM, Paris Carbone <[hidden email]> wrote: > >> > >>> Hey Vasia, > >>> > >>> I really like the idea and it seems we have some volunteers from our > side > >>> already! > >>> It is in my opinion a starter task since it requires to create a simple > >>> adapter for flink as well as mapping abstractions to our streaming > >> topology > >>> API in a similar fashion (in its simplest form just using several > >> flatmaps). > >>> > >>> It will serve as a suitable common layer to make some comparisons > between > >>> flink-streaming to storm, s4 and samoa on incremental machine learning > >>> tasks. > >>> > >>> Paris > >>> > >>>> On 27 Nov 2014, at 13:59, Vasiliki Kalavri <[hidden email] > > > >>> wrote: > >>>> > >>>> Hello everyone, > >>>> > >>>> Last week at Strata, I attended a talk about a streaming machine > >> learning > >>>> framework from Yahoo labs, SAMOA: https://github.com/yahoo/samoa. The > >>>> project is going through apache incubation application and currently > >> runs > >>>> on Storm, S4 and Samza. > >>>> > >>>> After the talk, I spoke with the guy that presented it and we both > >>> thought > >>>> that it'd be cool to port Samoa on Flink. > >>>> > >>>> I'm writing about this here because I want to know your thoughts on > >> this > >>>> and also because I'm not very familiar with the Streaming-API, so I > >> will > >>>> need your input :) > >>>> I have already discussed with Paris and there is interest to pick this > >> up > >>>> from some students at KTH as well. > >>>> As far as I understand, Samoa expects an API similar to that of Storm > >> and > >>>> from there the integration should be pretty straight-forward. > >>>> > >>>> So, do you think it would be feasible and -most importantly- worth it? > >>>> > >>>> Cheers, > >>>> Vasia. > >>> > >>> > >> > > |
Free forum by Nabble | Edit this page |