Flink shell in Jupyter

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink shell in Jupyter

Gyula Fóra-2
Hey Guys,

Has anyone tried to setup the Flink scala shell with Jupyter? I would
assume the logic is similar to Zeppelin.

The reason I am asking this because we have a Jupyter cluster that runs
python and scala (2.11 I believe) and Spark works on it, so we figured it
would be good to add support for Flink as well so data scientists can then
use that as well.

I am of course willing to help with this, both with development and testing
in a real production environment :)

Cheers,
Gyula
Reply | Threaded
Open this post in threaded view
|

Re: Flink shell in Jupyter

Stephan Ewen
I think Till has done some advanced Pythin / Flink / Zeppelin integration
(to use Python plotting libs) for a talk at some point.

@Till: Do you still have the code? Could you share it with Gyula?

On Wed, Dec 16, 2015 at 4:22 PM, Gyula Fóra <[hidden email]> wrote:

> Hey Guys,
>
> Has anyone tried to setup the Flink scala shell with Jupyter? I would
> assume the logic is similar to Zeppelin.
>
> The reason I am asking this because we have a Jupyter cluster that runs
> python and scala (2.11 I believe) and Spark works on it, so we figured it
> would be good to add support for Flink as well so data scientists can then
> use that as well.
>
> I am of course willing to help with this, both with development and testing
> in a real production environment :)
>
> Cheers,
> Gyula
>
Reply | Threaded
Open this post in threaded view
|

Re: Flink shell in Jupyter

Gyula Fóra
That would be awesome, I think it would be fine to go for Scala 2.11 first
in the notebook, if Python is more trouble.

Gyula

Stephan Ewen <[hidden email]> ezt írta (időpont: 2015. dec. 17., Cs,
14:17):

> I think Till has done some advanced Pythin / Flink / Zeppelin integration
> (to use Python plotting libs) for a talk at some point.
>
> @Till: Do you still have the code? Could you share it with Gyula?
>
> On Wed, Dec 16, 2015 at 4:22 PM, Gyula Fóra <[hidden email]> wrote:
>
> > Hey Guys,
> >
> > Has anyone tried to setup the Flink scala shell with Jupyter? I would
> > assume the logic is similar to Zeppelin.
> >
> > The reason I am asking this because we have a Jupyter cluster that runs
> > python and scala (2.11 I believe) and Spark works on it, so we figured it
> > would be good to add support for Flink as well so data scientists can
> then
> > use that as well.
> >
> > I am of course willing to help with this, both with development and
> testing
> > in a real production environment :)
> >
> > Cheers,
> > Gyula
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Flink shell in Jupyter

Till Rohrmann
The only thing I did was to use the Py4J bridge to ship data between the
Scala and the Python process. That way, I could visualize data coming from
Flink with Python. I couldn't run Flink programs using the Python API,
though.

@Gyula, you can find the code here:
https://github.com/tillrohrmann/incubator-zeppelin/tree/flink

Cheers,
Till

On Thu, Dec 17, 2015 at 2:24 PM, Gyula Fóra <[hidden email]> wrote:

> That would be awesome, I think it would be fine to go for Scala 2.11 first
> in the notebook, if Python is more trouble.
>
> Gyula
>
> Stephan Ewen <[hidden email]> ezt írta (időpont: 2015. dec. 17., Cs,
> 14:17):
>
>> I think Till has done some advanced Pythin / Flink / Zeppelin integration
>> (to use Python plotting libs) for a talk at some point.
>>
>> @Till: Do you still have the code? Could you share it with Gyula?
>>
>> On Wed, Dec 16, 2015 at 4:22 PM, Gyula Fóra <[hidden email]> wrote:
>>
>> > Hey Guys,
>> >
>> > Has anyone tried to setup the Flink scala shell with Jupyter? I would
>> > assume the logic is similar to Zeppelin.
>> >
>> > The reason I am asking this because we have a Jupyter cluster that runs
>> > python and scala (2.11 I believe) and Spark works on it, so we figured
>> it
>> > would be good to add support for Flink as well so data scientists can
>> then
>> > use that as well.
>> >
>> > I am of course willing to help with this, both with development and
>> testing
>> > in a real production environment :)
>> >
>> > Cheers,
>> > Gyula
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Flink shell in Jupyter

Tanya Schlusser
@Gyula if you want to delegate something small this interests me too.

Best,
Tanya

On Thu, Dec 17, 2015 at 9:04 AM, Till Rohrmann <[hidden email]> wrote:

> The only thing I did was to use the Py4J bridge to ship data between the
> Scala and the Python process. That way, I could visualize data coming from
> Flink with Python. I couldn't run Flink programs using the Python API,
> though.
>
> @Gyula, you can find the code here:
> https://github.com/tillrohrmann/incubator-zeppelin/tree/flink
>
> Cheers,
> Till
>
> On Thu, Dec 17, 2015 at 2:24 PM, Gyula Fóra <[hidden email]> wrote:
>
> > That would be awesome, I think it would be fine to go for Scala 2.11
> first
> > in the notebook, if Python is more trouble.
> >
> > Gyula
> >
> > Stephan Ewen <[hidden email]> ezt írta (időpont: 2015. dec. 17., Cs,
> > 14:17):
> >
> >> I think Till has done some advanced Pythin / Flink / Zeppelin
> integration
> >> (to use Python plotting libs) for a talk at some point.
> >>
> >> @Till: Do you still have the code? Could you share it with Gyula?
> >>
> >> On Wed, Dec 16, 2015 at 4:22 PM, Gyula Fóra <[hidden email]> wrote:
> >>
> >> > Hey Guys,
> >> >
> >> > Has anyone tried to setup the Flink scala shell with Jupyter? I would
> >> > assume the logic is similar to Zeppelin.
> >> >
> >> > The reason I am asking this because we have a Jupyter cluster that
> runs
> >> > python and scala (2.11 I believe) and Spark works on it, so we figured
> >> it
> >> > would be good to add support for Flink as well so data scientists can
> >> then
> >> > use that as well.
> >> >
> >> > I am of course willing to help with this, both with development and
> >> testing
> >> > in a real production environment :)
> >> >
> >> > Cheers,
> >> > Gyula
> >> >
> >>
> >
>