Hi,
I see that the last update to the roadmap for FlinkML was some time ago (2016). Is development still active? If so, I would like to contribute some unsupervised clustering algorithms like CLARANS. Would love some pointers. Thanks, Badrul |
HI Badrul,
FlinkML is unfortunately no longer under active development. However, there is some new effort to add a machine learning library to Flink [1]. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs Cheers, Till On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury <[hidden email]> wrote: > Hi, > > I see that the last update to the roadmap for FlinkML was some time ago > (2016). Is development still active? If so, I would like to contribute some > unsupervised clustering algorithms like CLARANS. Would love some pointers. > > Thanks, > Badrul > |
What about Flink-AI [1]? Would you suggest its adoption Till?
[1] https://github.com/alibaba/flink-ai-extended On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email]> wrote: > HI Badrul, > > FlinkML is unfortunately no longer under active development. However, there > is some new effort to add a machine learning library to Flink [1]. > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > Cheers, > Till > > On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > [hidden email]> > wrote: > > > Hi, > > > > I see that the last update to the roadmap for FlinkML was some time ago > > (2016). Is development still active? If so, I would like to contribute > some > > unsupervised clustering algorithms like CLARANS. Would love some > pointers. > > > > Thanks, > > Badrul > > |
Or also https://github.com/alibaba/Alink, I don't know if the 2 are related
somehow.. On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier <[hidden email]> wrote: > What about Flink-AI [1]? Would you suggest its adoption Till? > > [1] https://github.com/alibaba/flink-ai-extended > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email]> wrote: > >> HI Badrul, >> >> FlinkML is unfortunately no longer under active development. However, >> there >> is some new effort to add a machine learning library to Flink [1]. >> >> [1] >> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs >> >> Cheers, >> Till >> >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < >> [hidden email]> >> wrote: >> >> > Hi, >> > >> > I see that the last update to the roadmap for FlinkML was some time ago >> > (2016). Is development still active? If so, I would like to contribute >> some >> > unsupervised clustering algorithms like CLARANS. Would love some >> pointers. >> > >> > Thanks, >> > Badrul >> > > > |
Hi Flavio,
Currently most of the Flink AI efforts are in ecosystem projects. Primarily flink-ai-extended[1] and Alink[2] as you noticed. - flink-ai-extended aims to provide a solution to unify big data and AI (especially deep learning) based on Flink. - Alink is a classic machine learning library. We have seen some unique values of these projects in scenarios such as realtime machine learning, as well as unifying the online/offline systems. That said, because Flink AI is still kind of new and we are still exploring many things. We would like to wait a little bit until the APIs are more stable before putting them into Apache Flink. Thanks, Jiangjie (Becket) Qin [1] https://github.com/alibaba/flink-ai-extended [2] https://github.com/alibaba/Alink On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier <[hidden email]> wrote: > Or also https://github.com/alibaba/Alink, I don't know if the 2 are > related > somehow.. > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier <[hidden email]> > wrote: > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email]> > wrote: > > > >> HI Badrul, > >> > >> FlinkML is unfortunately no longer under active development. However, > >> there > >> is some new effort to add a machine learning library to Flink [1]. > >> > >> [1] > >> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > >> > >> Cheers, > >> Till > >> > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > >> [hidden email]> > >> wrote: > >> > >> > Hi, > >> > > >> > I see that the last update to the roadmap for FlinkML was some time > ago > >> > (2016). Is development still active? If so, I would like to contribute > >> some > >> > unsupervised clustering algorithms like CLARANS. Would love some > >> pointers. > >> > > >> > Thanks, > >> > Badrul > >> > > > > > > |
Thanks all for the pointers! The architecture outlined in FLIP39
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs> seems promising and the community seems to have accepted it- let me look into contributing to that effort. Thanks, Badrul On Thu, Jan 7, 2021 at 4:29 AM Becket Qin <[hidden email]> wrote: > Hi Flavio, > > Currently most of the Flink AI efforts are in ecosystem projects. Primarily > flink-ai-extended[1] and Alink[2] as you noticed. > > - flink-ai-extended aims to provide a solution to unify big data and AI > (especially deep learning) based on Flink. > - Alink is a classic machine learning library. > > We have seen some unique values of these projects in scenarios such as > realtime machine learning, as well as unifying the online/offline systems. > That said, because Flink AI is still kind of new and we are still exploring > many things. We would like to wait a little bit until the APIs are more > stable before putting them into Apache Flink. > > Thanks, > > Jiangjie (Becket) Qin > > [1] https://github.com/alibaba/flink-ai-extended > [2] https://github.com/alibaba/Alink > > > On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier <[hidden email]> > wrote: > > > Or also https://github.com/alibaba/Alink, I don't know if the 2 are > > related > > somehow.. > > > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier <[hidden email]> > > wrote: > > > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email]> > > wrote: > > > > > >> HI Badrul, > > >> > > >> FlinkML is unfortunately no longer under active development. However, > > >> there > > >> is some new effort to add a machine learning library to Flink [1]. > > >> > > >> [1] > > >> > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > >> > > >> Cheers, > > >> Till > > >> > > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > > >> [hidden email]> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > I see that the last update to the roadmap for FlinkML was some time > > ago > > >> > (2016). Is development still active? If so, I would like to > contribute > > >> some > > >> > unsupervised clustering algorithms like CLARANS. Would love some > > >> pointers. > > >> > > > >> > Thanks, > > >> > Badrul > > >> > > > > > > > > > > -- Cheers, Badrul |
Thanks Badrul,
Contribution to the ML pipeline is highly appreciated! Please don't hesitate to reach out if you got any questions. Cheers, Jiangjie (Becket) Qin On Fri, Jan 8, 2021 at 1:02 AM Badrul Chowdhury <[hidden email]> wrote: > Thanks all for the pointers! The architecture outlined in FLIP39 > < > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > seems > promising and the community seems to have accepted it- let me look into > contributing to that effort. > > Thanks, > Badrul > > On Thu, Jan 7, 2021 at 4:29 AM Becket Qin <[hidden email]> wrote: > > > Hi Flavio, > > > > Currently most of the Flink AI efforts are in ecosystem projects. > Primarily > > flink-ai-extended[1] and Alink[2] as you noticed. > > > > - flink-ai-extended aims to provide a solution to unify big data and AI > > (especially deep learning) based on Flink. > > - Alink is a classic machine learning library. > > > > We have seen some unique values of these projects in scenarios such as > > realtime machine learning, as well as unifying the online/offline > systems. > > That said, because Flink AI is still kind of new and we are still > exploring > > many things. We would like to wait a little bit until the APIs are more > > stable before putting them into Apache Flink. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > [1] https://github.com/alibaba/flink-ai-extended > > [2] https://github.com/alibaba/Alink > > > > > > On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier <[hidden email]> > > wrote: > > > > > Or also https://github.com/alibaba/Alink, I don't know if the 2 are > > > related > > > somehow.. > > > > > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier < > [hidden email]> > > > wrote: > > > > > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email]> > > > wrote: > > > > > > > >> HI Badrul, > > > >> > > > >> FlinkML is unfortunately no longer under active development. > However, > > > >> there > > > >> is some new effort to add a machine learning library to Flink [1]. > > > >> > > > >> [1] > > > >> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > >> > > > >> Cheers, > > > >> Till > > > >> > > > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > > > >> [hidden email]> > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > I see that the last update to the roadmap for FlinkML was some > time > > > ago > > > >> > (2016). Is development still active? If so, I would like to > > contribute > > > >> some > > > >> > unsupervised clustering algorithms like CLARANS. Would love some > > > >> pointers. > > > >> > > > > >> > Thanks, > > > >> > Badrul > > > >> > > > > > > > > > > > > > > > > > -- > > Cheers, > Badrul > |
Hi Becket,
I was looking into picking an item to work on and I came across this open PR for the BinarizerMapper: https://issues.apache.org/jira/browse/FLINK-13669 https://github.com/apache/flink/pull/9406 Could you please help me understand what is required for this to be merged? Can I help? Thanks, Badrul On Thu, Jan 7, 2021 at 9:05 AM Becket Qin <[hidden email]> wrote: > Thanks Badrul, > > Contribution to the ML pipeline is highly appreciated! Please don't > hesitate to reach out if you got any questions. > > Cheers, > > Jiangjie (Becket) Qin > > On Fri, Jan 8, 2021 at 1:02 AM Badrul Chowdhury < > [hidden email]> > wrote: > > > Thanks all for the pointers! The architecture outlined in FLIP39 > > < > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > > seems > > promising and the community seems to have accepted it- let me look into > > contributing to that effort. > > > > Thanks, > > Badrul > > > > On Thu, Jan 7, 2021 at 4:29 AM Becket Qin <[hidden email]> wrote: > > > > > Hi Flavio, > > > > > > Currently most of the Flink AI efforts are in ecosystem projects. > > Primarily > > > flink-ai-extended[1] and Alink[2] as you noticed. > > > > > > - flink-ai-extended aims to provide a solution to unify big data and AI > > > (especially deep learning) based on Flink. > > > - Alink is a classic machine learning library. > > > > > > We have seen some unique values of these projects in scenarios such as > > > realtime machine learning, as well as unifying the online/offline > > systems. > > > That said, because Flink AI is still kind of new and we are still > > exploring > > > many things. We would like to wait a little bit until the APIs are more > > > stable before putting them into Apache Flink. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > [2] https://github.com/alibaba/Alink > > > > > > > > > On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier < > [hidden email]> > > > wrote: > > > > > > > Or also https://github.com/alibaba/Alink, I don't know if the 2 are > > > > related > > > > somehow.. > > > > > > > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier < > > [hidden email]> > > > > wrote: > > > > > > > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > > > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann <[hidden email] > > > > > > wrote: > > > > > > > > > >> HI Badrul, > > > > >> > > > > >> FlinkML is unfortunately no longer under active development. > > However, > > > > >> there > > > > >> is some new effort to add a machine learning library to Flink [1]. > > > > >> > > > > >> [1] > > > > >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > >> > > > > >> Cheers, > > > > >> Till > > > > >> > > > > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > > > > >> [hidden email]> > > > > >> wrote: > > > > >> > > > > >> > Hi, > > > > >> > > > > > >> > I see that the last update to the roadmap for FlinkML was some > > time > > > > ago > > > > >> > (2016). Is development still active? If so, I would like to > > > contribute > > > > >> some > > > > >> > unsupervised clustering algorithms like CLARANS. Would love some > > > > >> pointers. > > > > >> > > > > > >> > Thanks, > > > > >> > Badrul > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Cheers, > > Badrul > > > -- Cheers, Badrul |
Hi Badrul,
I think this ticket is a part of the effort of contributing Alink[1] into Flink as its ML library. The ticket became dormant for some time mainly due to the bandwidth of the contributor and reviewer. I'd imagine that Xu would like to resume the ticket some time soon. cc Xu. Thanks, Jiangjie (Becket) Qin [1] https://github.com/alibaba/Alink On Thu, Feb 4, 2021 at 2:54 AM Badrul Chowdhury <[hidden email]> wrote: > Hi Becket, > > I was looking into picking an item to work on and I came across this open > PR for the BinarizerMapper: > > https://issues.apache.org/jira/browse/FLINK-13669 > https://github.com/apache/flink/pull/9406 > > Could you please help me understand what is required for this to be merged? > Can I help? > > Thanks, > Badrul > > On Thu, Jan 7, 2021 at 9:05 AM Becket Qin <[hidden email]> wrote: > > > Thanks Badrul, > > > > Contribution to the ML pipeline is highly appreciated! Please don't > > hesitate to reach out if you got any questions. > > > > Cheers, > > > > Jiangjie (Becket) Qin > > > > On Fri, Jan 8, 2021 at 1:02 AM Badrul Chowdhury < > > [hidden email]> > > wrote: > > > > > Thanks all for the pointers! The architecture outlined in FLIP39 > > > < > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > > > > seems > > > promising and the community seems to have accepted it- let me look into > > > contributing to that effort. > > > > > > Thanks, > > > Badrul > > > > > > On Thu, Jan 7, 2021 at 4:29 AM Becket Qin <[hidden email]> > wrote: > > > > > > > Hi Flavio, > > > > > > > > Currently most of the Flink AI efforts are in ecosystem projects. > > > Primarily > > > > flink-ai-extended[1] and Alink[2] as you noticed. > > > > > > > > - flink-ai-extended aims to provide a solution to unify big data and > AI > > > > (especially deep learning) based on Flink. > > > > - Alink is a classic machine learning library. > > > > > > > > We have seen some unique values of these projects in scenarios such > as > > > > realtime machine learning, as well as unifying the online/offline > > > systems. > > > > That said, because Flink AI is still kind of new and we are still > > > exploring > > > > many things. We would like to wait a little bit until the APIs are > more > > > > stable before putting them into Apache Flink. > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > [2] https://github.com/alibaba/Alink > > > > > > > > > > > > On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier < > > [hidden email]> > > > > wrote: > > > > > > > > > Or also https://github.com/alibaba/Alink, I don't know if the 2 > are > > > > > related > > > > > somehow.. > > > > > > > > > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier < > > > [hidden email]> > > > > > wrote: > > > > > > > > > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > > > > > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann < > [hidden email] > > > > > > > > wrote: > > > > > > > > > > > >> HI Badrul, > > > > > >> > > > > > >> FlinkML is unfortunately no longer under active development. > > > However, > > > > > >> there > > > > > >> is some new effort to add a machine learning library to Flink > [1]. > > > > > >> > > > > > >> [1] > > > > > >> > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > > >> > > > > > >> Cheers, > > > > > >> Till > > > > > >> > > > > > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > > > > > >> [hidden email]> > > > > > >> wrote: > > > > > >> > > > > > >> > Hi, > > > > > >> > > > > > > >> > I see that the last update to the roadmap for FlinkML was some > > > time > > > > > ago > > > > > >> > (2016). Is development still active? If so, I would like to > > > > contribute > > > > > >> some > > > > > >> > unsupervised clustering algorithms like CLARANS. Would love > some > > > > > >> pointers. > > > > > >> > > > > > > >> > Thanks, > > > > > >> > Badrul > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Cheers, > > > Badrul > > > > > > > > -- > > Cheers, > Badrul > |
Hi Becket,
Thanks for sharing more details. Alink seems to be pretty mature, is there a FLIP discussing adopting Alink as Flink's ML library? I wanted to start working on FLIP39 <https://issues.apache.org/jira/browse/FLINK-12470>, but I am not sure it's worth it if it will be replaced by Alink eventually. In that case, I'd rather work on something else (open to suggestions). Thanks, Badrul On Fri, Feb 5, 2021 at 4:43 AM Becket Qin <[hidden email]> wrote: > Hi Badrul, > > I think this ticket is a part of the effort of contributing Alink[1] into > Flink as its ML library. The ticket became dormant for some time mainly due > to the bandwidth of the contributor and reviewer. I'd imagine that Xu would > like to resume the ticket some time soon. > > cc Xu. > > Thanks, > > Jiangjie (Becket) Qin > > [1] https://github.com/alibaba/Alink > > On Thu, Feb 4, 2021 at 2:54 AM Badrul Chowdhury < > [hidden email]> > wrote: > > > Hi Becket, > > > > I was looking into picking an item to work on and I came across this open > > PR for the BinarizerMapper: > > > > https://issues.apache.org/jira/browse/FLINK-13669 > > https://github.com/apache/flink/pull/9406 > > > > Could you please help me understand what is required for this to be > merged? > > Can I help? > > > > Thanks, > > Badrul > > > > On Thu, Jan 7, 2021 at 9:05 AM Becket Qin <[hidden email]> wrote: > > > > > Thanks Badrul, > > > > > > Contribution to the ML pipeline is highly appreciated! Please don't > > > hesitate to reach out if you got any questions. > > > > > > Cheers, > > > > > > Jiangjie (Becket) Qin > > > > > > On Fri, Jan 8, 2021 at 1:02 AM Badrul Chowdhury < > > > [hidden email]> > > > wrote: > > > > > > > Thanks all for the pointers! The architecture outlined in FLIP39 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > > > > > > seems > > > > promising and the community seems to have accepted it- let me look > into > > > > contributing to that effort. > > > > > > > > Thanks, > > > > Badrul > > > > > > > > On Thu, Jan 7, 2021 at 4:29 AM Becket Qin <[hidden email]> > > wrote: > > > > > > > > > Hi Flavio, > > > > > > > > > > Currently most of the Flink AI efforts are in ecosystem projects. > > > > Primarily > > > > > flink-ai-extended[1] and Alink[2] as you noticed. > > > > > > > > > > - flink-ai-extended aims to provide a solution to unify big data > and > > AI > > > > > (especially deep learning) based on Flink. > > > > > - Alink is a classic machine learning library. > > > > > > > > > > We have seen some unique values of these projects in scenarios such > > as > > > > > realtime machine learning, as well as unifying the online/offline > > > > systems. > > > > > That said, because Flink AI is still kind of new and we are still > > > > exploring > > > > > many things. We would like to wait a little bit until the APIs are > > more > > > > > stable before putting them into Apache Flink. > > > > > > > > > > Thanks, > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > [2] https://github.com/alibaba/Alink > > > > > > > > > > > > > > > On Thu, Jan 7, 2021 at 7:53 PM Flavio Pompermaier < > > > [hidden email]> > > > > > wrote: > > > > > > > > > > > Or also https://github.com/alibaba/Alink, I don't know if the 2 > > are > > > > > > related > > > > > > somehow.. > > > > > > > > > > > > On Thu, Jan 7, 2021 at 9:55 AM Flavio Pompermaier < > > > > [hidden email]> > > > > > > wrote: > > > > > > > > > > > > > What about Flink-AI [1]? Would you suggest its adoption Till? > > > > > > > > > > > > > > [1] https://github.com/alibaba/flink-ai-extended > > > > > > > > > > > > > > On Thu, Jan 7, 2021 at 9:38 AM Till Rohrmann < > > [hidden email] > > > > > > > > > > wrote: > > > > > > > > > > > > > >> HI Badrul, > > > > > > >> > > > > > > >> FlinkML is unfortunately no longer under active development. > > > > However, > > > > > > >> there > > > > > > >> is some new effort to add a machine learning library to Flink > > [1]. > > > > > > >> > > > > > > >> [1] > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > > > > > >> > > > > > > >> Cheers, > > > > > > >> Till > > > > > > >> > > > > > > >> On Wed, Jan 6, 2021 at 7:11 PM Badrul Chowdhury < > > > > > > >> [hidden email]> > > > > > > >> wrote: > > > > > > >> > > > > > > >> > Hi, > > > > > > >> > > > > > > > >> > I see that the last update to the roadmap for FlinkML was > some > > > > time > > > > > > ago > > > > > > >> > (2016). Is development still active? If so, I would like to > > > > > contribute > > > > > > >> some > > > > > > >> > unsupervised clustering algorithms like CLARANS. Would love > > some > > > > > > >> pointers. > > > > > > >> > > > > > > > >> > Thanks, > > > > > > >> > Badrul > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Cheers, > > > > Badrul > > > > > > > > > > > > > -- > > > > Cheers, > > Badrul > > > -- Cheers, Badrul |
Free forum by Nabble | Edit this page |