Apache Big Data

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache Big Data

Trevor Grant
Any one who is attending Apache: Big Data in Vancouver May 9-12 stop by and
check out my talk: *Everyone Plays: Collaborative Data Science With
Zeppelin*

I'll obviously be giving some big shout outs to Flink.

Which leads me to my next question: is anyone working on a Flink Streaming
Interpreter for Zeppelin? I've been heads down on writing those LTSM Neural
Nets for Flink (also hoping to have those ready for show casing in time),
but a notebook running Flink Streaming would be quite dope.

Thanks!
tg


Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*
Reply | Threaded
Open this post in threaded view
|

Re: Apache Big Data

Suneel Marthi-2
Same here, will be presenting Distributed ML using Mahout on Flink at
ApacheCon.

Some big shout outs to Flink coming in Vancouver.

On Tue, Mar 1, 2016 at 2:35 PM, Trevor Grant <[hidden email]>
wrote:

> Any one who is attending Apache: Big Data in Vancouver May 9-12 stop by and
> check out my talk: *Everyone Plays: Collaborative Data Science With
> Zeppelin*
>
> I'll obviously be giving some big shout outs to Flink.
>
> Which leads me to my next question: is anyone working on a Flink Streaming
> Interpreter for Zeppelin? I've been heads down on writing those LTSM Neural
> Nets for Flink (also hoping to have those ready for show casing in time),
> but a notebook running Flink Streaming would be quite dope.
>
> Thanks!
> tg
>
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>
Reply | Threaded
Open this post in threaded view
|

Re: Apache Big Data

Till Rohrmann
Great to hear that you two are giving a talk at ApacheCon.

As far as I know, there's nobody working on a streaming interpreter for
Zeppelin. People thought about doing it, but it never got realized so far.
But I think it should not be too difficult to implement. So if you wanna
take the lead there, go ahead. I'm convinced that the community would be
very happy about it.

Cheers,
Till

On Tue, Mar 1, 2016 at 9:15 PM, Suneel Marthi <[hidden email]> wrote:

> Same here, will be presenting Distributed ML using Mahout on Flink at
> ApacheCon.
>
> Some big shout outs to Flink coming in Vancouver.
>
> On Tue, Mar 1, 2016 at 2:35 PM, Trevor Grant <[hidden email]>
> wrote:
>
> > Any one who is attending Apache: Big Data in Vancouver May 9-12 stop by
> and
> > check out my talk: *Everyone Plays: Collaborative Data Science With
> > Zeppelin*
> >
> > I'll obviously be giving some big shout outs to Flink.
> >
> > Which leads me to my next question: is anyone working on a Flink
> Streaming
> > Interpreter for Zeppelin? I've been heads down on writing those LTSM
> Neural
> > Nets for Flink (also hoping to have those ready for show casing in time),
> > but a notebook running Flink Streaming would be quite dope.
> >
> > Thanks!
> > tg
> >
> >
> > Trevor Grant
> > Data Scientist
> > https://github.com/rawkintrevo
> > http://stackexchange.com/users/3002022/rawkintrevo
> > http://trevorgrant.org
> >
> > *"Fortunate is he, who is able to know the causes of things."  -Virgil*
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Apache Big Data

nse sik
Hi guys,

I've integrated streaming in zeppelin for flink.
It works using the scala shell, which I extended to support the streaming
application.
Unfortunately the scala-shell for streaming is not yet included in the
Flink-master, and changed a bit upon public request since I implemented the
zeppelin-flink connection.
It should, however, work with these two versions of Flink and Zeppelin:

https://github.com/nikste/flink/tree/Flink-2522_Scala_shell_streaming_download_linux
https://github.com/nikste/incubator-zeppelin/tree/visualizationDemo

Note that the resubmission of streaming jobs by zeppelin is not yet working.
The running streaming job has to be cancelled by zeppelin, which is
possible by finding out the job-id and sending a cancel message to the task
manager.
I'll continue integrating it with the current versions of Flink and
Zeppelin as soon as I have the time.

cheers,
Nik


2016-03-02 10:27 GMT+01:00 Till Rohrmann <[hidden email]>:

> Great to hear that you two are giving a talk at ApacheCon.
>
> As far as I know, there's nobody working on a streaming interpreter for
> Zeppelin. People thought about doing it, but it never got realized so far.
> But I think it should not be too difficult to implement. So if you wanna
> take the lead there, go ahead. I'm convinced that the community would be
> very happy about it.
>
> Cheers,
> Till
>
> On Tue, Mar 1, 2016 at 9:15 PM, Suneel Marthi <[hidden email]> wrote:
>
> > Same here, will be presenting Distributed ML using Mahout on Flink at
> > ApacheCon.
> >
> > Some big shout outs to Flink coming in Vancouver.
> >
> > On Tue, Mar 1, 2016 at 2:35 PM, Trevor Grant <[hidden email]>
> > wrote:
> >
> > > Any one who is attending Apache: Big Data in Vancouver May 9-12 stop by
> > and
> > > check out my talk: *Everyone Plays: Collaborative Data Science With
> > > Zeppelin*
> > >
> > > I'll obviously be giving some big shout outs to Flink.
> > >
> > > Which leads me to my next question: is anyone working on a Flink
> > Streaming
> > > Interpreter for Zeppelin? I've been heads down on writing those LTSM
> > Neural
> > > Nets for Flink (also hoping to have those ready for show casing in
> time),
> > > but a notebook running Flink Streaming would be quite dope.
> > >
> > > Thanks!
> > > tg
> > >
> > >
> > > Trevor Grant
> > > Data Scientist
> > > https://github.com/rawkintrevo
> > > http://stackexchange.com/users/3002022/rawkintrevo
> > > http://trevorgrant.org
> > >
> > > *"Fortunate is he, who is able to know the causes of things."  -Virgil*
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Apache Big Data

Till Rohrmann
That is cool Nikolaas :-) Looking forward to the scala-shell for streaming
:-)

On Wed, Mar 2, 2016 at 10:53 AM, Nikolaas s <[hidden email]>
wrote:

> Hi guys,
>
> I've integrated streaming in zeppelin for flink.
> It works using the scala shell, which I extended to support the streaming
> application.
> Unfortunately the scala-shell for streaming is not yet included in the
> Flink-master, and changed a bit upon public request since I implemented the
> zeppelin-flink connection.
> It should, however, work with these two versions of Flink and Zeppelin:
>
>
> https://github.com/nikste/flink/tree/Flink-2522_Scala_shell_streaming_download_linux
> https://github.com/nikste/incubator-zeppelin/tree/visualizationDemo
>
> Note that the resubmission of streaming jobs by zeppelin is not yet
> working.
> The running streaming job has to be cancelled by zeppelin, which is
> possible by finding out the job-id and sending a cancel message to the task
> manager.
> I'll continue integrating it with the current versions of Flink and
> Zeppelin as soon as I have the time.
>
> cheers,
> Nik
>
>
> 2016-03-02 10:27 GMT+01:00 Till Rohrmann <[hidden email]>:
>
> > Great to hear that you two are giving a talk at ApacheCon.
> >
> > As far as I know, there's nobody working on a streaming interpreter for
> > Zeppelin. People thought about doing it, but it never got realized so
> far.
> > But I think it should not be too difficult to implement. So if you wanna
> > take the lead there, go ahead. I'm convinced that the community would be
> > very happy about it.
> >
> > Cheers,
> > Till
> >
> > On Tue, Mar 1, 2016 at 9:15 PM, Suneel Marthi <[hidden email]>
> wrote:
> >
> > > Same here, will be presenting Distributed ML using Mahout on Flink at
> > > ApacheCon.
> > >
> > > Some big shout outs to Flink coming in Vancouver.
> > >
> > > On Tue, Mar 1, 2016 at 2:35 PM, Trevor Grant <[hidden email]
> >
> > > wrote:
> > >
> > > > Any one who is attending Apache: Big Data in Vancouver May 9-12 stop
> by
> > > and
> > > > check out my talk: *Everyone Plays: Collaborative Data Science With
> > > > Zeppelin*
> > > >
> > > > I'll obviously be giving some big shout outs to Flink.
> > > >
> > > > Which leads me to my next question: is anyone working on a Flink
> > > Streaming
> > > > Interpreter for Zeppelin? I've been heads down on writing those LTSM
> > > Neural
> > > > Nets for Flink (also hoping to have those ready for show casing in
> > time),
> > > > but a notebook running Flink Streaming would be quite dope.
> > > >
> > > > Thanks!
> > > > tg
> > > >
> > > >
> > > > Trevor Grant
> > > > Data Scientist
> > > > https://github.com/rawkintrevo
> > > > http://stackexchange.com/users/3002022/rawkintrevo
> > > > http://trevorgrant.org
> > > >
> > > > *"Fortunate is he, who is able to know the causes of things."
> -Virgil*
> > > >
> > >
> >
>