A small Project I've been working on

[Aljoscha Krettek-2]

Hi,
I've been working on a little side project in my free time: Ruby on
Flink (RoF). This should finally allow us to tap into the whole web
developer ruby world. The design is also flexible enough to easily
port this to node.js.

The code is not yet in shape to be made public but this is a small
preview of what the API looks like:

input = env.read_text_file("/some/path")

words = input.flatMap do |line|
  parts = line.split(/\W+/)
  parts.each do |word|
    word, 1
  end
end

counts = words.reduce do |left, right|
  left(1), left(1) + right(2)
end

counts.print()

env.execute!()

Cheers,
Aljoscha

[Fabian Hueske-2]

:-D
This is awesome!

Do you have some performance numbers?
On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]> wrote:

> Hi,
> I've been working on a little side project in my free time: Ruby on
> Flink (RoF). This should finally allow us to tap into the whole web
> developer ruby world. The design is also flexible enough to easily
> port this to node.js.
>
> The code is not yet in shape to be made public but this is a small
> preview of what the API looks like:
>
> input = env.read_text_file("/some/path")
>
> words = input.flatMap do |line|
>   parts = line.split(/\W+/)
>   parts.each do |word|
>     word, 1
>   end
> end
>
> counts = words.reduce do |left, right|
>   left(1), left(1) + right(2)
> end
>
> counts.print()
>
> env.execute!()
>
> Cheers,
> Aljoscha
>

[Aljoscha Krettek-2]

Right now, runtime is roughly thrice that of equivalent java programs.
But I plan on bringing that to the same ballpark using code
generation.

On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]> wrote:

> :-D
> This is awesome!
>
> Do you have some performance numbers?
> On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]> wrote:
>
>> Hi,
>> I've been working on a little side project in my free time: Ruby on
>> Flink (RoF). This should finally allow us to tap into the whole web
>> developer ruby world. The design is also flexible enough to easily
>> port this to node.js.
>>
>> The code is not yet in shape to be made public but this is a small
>> preview of what the API looks like:
>>
>> input = env.read_text_file("/some/path")
>>
>> words = input.flatMap do |line|
>>   parts = line.split(/\W+/)
>>   parts.each do |word|
>>     word, 1
>>   end
>> end
>>
>> counts = words.reduce do |left, right|
>>   left(1), left(1) + right(2)
>> end
>>
>> counts.print()
>>
>> env.execute!()
>>
>> Cheers,
>> Aljoscha
>>

[Márton Balassi]

Woot!

On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <[hidden email]>
wrote:

> Right now, runtime is roughly thrice that of equivalent java programs.
> But I plan on bringing that to the same ballpark using code
> generation.
>
> On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]> wrote:
> > :-D
> > This is awesome!
> >
> > Do you have some performance numbers?
> > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]> wrote:
> >
> >> Hi,
> >> I've been working on a little side project in my free time: Ruby on
> >> Flink (RoF). This should finally allow us to tap into the whole web
> >> developer ruby world. The design is also flexible enough to easily
> >> port this to node.js.
> >>
> >> The code is not yet in shape to be made public but this is a small
> >> preview of what the API looks like:
> >>
> >> input = env.read_text_file("/some/path")
> >>
> >> words = input.flatMap do |line|
> >>   parts = line.split(/\W+/)
> >>   parts.each do |word|
> >>     word, 1
> >>   end
> >> end
> >>
> >> counts = words.reduce do |left, right|
> >>   left(1), left(1) + right(2)
> >> end
> >>
> >> counts.print()
> >>
> >> env.execute!()
> >>
> >> Cheers,
> >> Aljoscha
> >>
>

[Ufuk Celebi-2]

Little side projects ftw. Very nice :-)

Can you give some points on how this works internally? Is it making use of
anything generic from the Python API pull request?

On Wednesday, April 1, 2015, Márton Balassi <[hidden email]>
wrote:

> Woot!
>
> On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <[hidden email]
> <javascript:;>>
> wrote:
>
> > Right now, runtime is roughly thrice that of equivalent java programs.
> > But I plan on bringing that to the same ballpark using code
> > generation.
> >
> > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
> <javascript:;>> wrote:
> > > :-D
> > > This is awesome!
> > >
> > > Do you have some performance numbers?
> > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
> <javascript:;>> wrote:
> > >
> > >> Hi,
> > >> I've been working on a little side project in my free time: Ruby on
> > >> Flink (RoF). This should finally allow us to tap into the whole web
> > >> developer ruby world. The design is also flexible enough to easily
> > >> port this to node.js.
> > >>
> > >> The code is not yet in shape to be made public but this is a small
> > >> preview of what the API looks like:
> > >>
> > >> input = env.read_text_file("/some/path")
> > >>
> > >> words = input.flatMap do |line|
> > >>   parts = line.split(/\W+/)
> > >>   parts.each do |word|
> > >>     word, 1
> > >>   end
> > >> end
> > >>
> > >> counts = words.reduce do |left, right|
> > >>   left(1), left(1) + right(2)
> > >> end
> > >>
> > >> counts.print()
> > >>
> > >> env.execute!()
> > >>
> > >> Cheers,
> > >> Aljoscha
> > >>
> >
>

[Till Rohrmann]

Where can I start contributing. I've got the feeling that it will be
inherently faster than everything else which is out there.
On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:

> Little side projects ftw. Very nice :-)
>
> Can you give some points on how this works internally? Is it making use of
> anything generic from the Python API pull request?
>
> On Wednesday, April 1, 2015, Márton Balassi <[hidden email]>
> wrote:
>
> > Woot!
> >
> > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <[hidden email]
> > <javascript:;>>
> > wrote:
> >
> > > Right now, runtime is roughly thrice that of equivalent java programs.
> > > But I plan on bringing that to the same ballpark using code
> > > generation.
> > >
> > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
> > <javascript:;>> wrote:
> > > > :-D
> > > > This is awesome!
> > > >
> > > > Do you have some performance numbers?
> > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
> > <javascript:;>> wrote:
> > > >
> > > >> Hi,
> > > >> I've been working on a little side project in my free time: Ruby on
> > > >> Flink (RoF). This should finally allow us to tap into the whole web
> > > >> developer ruby world. The design is also flexible enough to easily
> > > >> port this to node.js.
> > > >>
> > > >> The code is not yet in shape to be made public but this is a small
> > > >> preview of what the API looks like:
> > > >>
> > > >> input = env.read_text_file("/some/path")
> > > >>
> > > >> words = input.flatMap do |line|
> > > >>   parts = line.split(/\W+/)
> > > >>   parts.each do |word|
> > > >>     word, 1
> > > >>   end
> > > >> end
> > > >>
> > > >> counts = words.reduce do |left, right|
> > > >>   left(1), left(1) + right(2)
> > > >> end
> > > >>
> > > >> counts.print()
> > > >>
> > > >> env.execute!()
> > > >>
> > > >> Cheers,
> > > >> Aljoscha
> > > >>
> > >
> >
>

[Fabian Hueske-2]

My ruby skills are a bit rust(y) but I’d love to contribute.


Can you point me to a repository that I can fork?






From: Till Rohrmann
Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29
To: [hidden email]





Where can I start contributing. I've got the feeling that it will be
inherently faster than everything else which is out there.
On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:

> Little side projects ftw. Very nice :-)
>
> Can you give some points on how this works internally? Is it making use of
> anything generic from the Python API pull request?
>
> On Wednesday, April 1, 2015, Márton Balassi <[hidden email]>
> wrote:
>
> > Woot!
> >
> > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <[hidden email]
> > <javascript:;>>
> > wrote:
> >
> > > Right now, runtime is roughly thrice that of equivalent java programs.
> > > But I plan on bringing that to the same ballpark using code
> > > generation.
> > >
> > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
> > <javascript:;>> wrote:
> > > > :-D
> > > > This is awesome!
> > > >
> > > > Do you have some performance numbers?
> > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
> > <javascript:;>> wrote:
> > > >
> > > >> Hi,
> > > >> I've been working on a little side project in my free time: Ruby on
> > > >> Flink (RoF). This should finally allow us to tap into the whole web
> > > >> developer ruby world. The design is also flexible enough to easily
> > > >> port this to node.js.
> > > >>
> > > >> The code is not yet in shape to be made public but this is a small
> > > >> preview of what the API looks like:
> > > >>
> > > >> input = env.read_text_file("/some/path")
> > > >>
> > > >> words = input.flatMap do |line|
> > > >>   parts = line.split(/\W+/)
> > > >>   parts.each do |word|
> > > >>     word, 1
> > > >>   end
> > > >> end
> > > >>
> > > >> counts = words.reduce do |left, right|
> > > >>   left(1), left(1) + right(2)
> > > >> end
> > > >>
> > > >> counts.print()
> > > >>
> > > >> env.execute!()
> > > >>
> > > >> Cheers,
> > > >> Aljoscha
> > > >>
> > >
> >
>

[Gyula Fóra-2]

Amazing! :)

On Wed, Apr 1, 2015 at 9:41 AM, <[hidden email]> wrote:

> My ruby skills are a bit rust(y) but I’d love to contribute.
>
>
> Can you point me to a repository that I can fork?
>
>
>
>
>
>
> From: Till Rohrmann
> Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29
> To: [hidden email]
>
>
>
>
>
> Where can I start contributing. I've got the feeling that it will be
> inherently faster than everything else which is out there.
> On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:
>
> > Little side projects ftw. Very nice :-)
> >
> > Can you give some points on how this works internally? Is it making use
> of
> > anything generic from the Python API pull request?
> >
> > On Wednesday, April 1, 2015, Márton Balassi <[hidden email]>
> > wrote:
> >
> > > Woot!
> > >
> > > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <[hidden email]
> > > <javascript:;>>
> > > wrote:
> > >
> > > > Right now, runtime is roughly thrice that of equivalent java
> programs.
> > > > But I plan on bringing that to the same ballpark using code
> > > > generation.
> > > >
> > > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
> > > <javascript:;>> wrote:
> > > > > :-D
> > > > > This is awesome!
> > > > >
> > > > > Do you have some performance numbers?
> > > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
> > > <javascript:;>> wrote:
> > > > >
> > > > >> Hi,
> > > > >> I've been working on a little side project in my free time: Ruby
> on
> > > > >> Flink (RoF). This should finally allow us to tap into the whole
> web
> > > > >> developer ruby world. The design is also flexible enough to easily
> > > > >> port this to node.js.
> > > > >>
> > > > >> The code is not yet in shape to be made public but this is a small
> > > > >> preview of what the API looks like:
> > > > >>
> > > > >> input = env.read_text_file("/some/path")
> > > > >>
> > > > >> words = input.flatMap do |line|
> > > > >>   parts = line.split(/\W+/)
> > > > >>   parts.each do |word|
> > > > >>     word, 1
> > > > >>   end
> > > > >> end
> > > > >>
> > > > >> counts = words.reduce do |left, right|
> > > > >>   left(1), left(1) + right(2)
> > > > >> end
> > > > >>
> > > > >> counts.print()
> > > > >>
> > > > >> env.execute!()
> > > > >>
> > > > >> Cheers,
> > > > >> Aljoscha
> > > > >>
> > > >
> > >
> >
>

[Stephan Ewen]

It April, 1st, right?

On Wed, Apr 1, 2015 at 9:47 AM, Gyula Fóra <[hidden email]> wrote:

> Amazing! :)
>
> On Wed, Apr 1, 2015 at 9:41 AM, <[hidden email]> wrote:
>
> > My ruby skills are a bit rust(y) but I’d love to contribute.
> >
> >
> > Can you point me to a repository that I can fork?
> >
> >
> >
> >
> >
> >
> > From: Till Rohrmann
> > Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29
> > To: [hidden email]
> >
> >
> >
> >
> >
> > Where can I start contributing. I've got the feeling that it will be
> > inherently faster than everything else which is out there.
> > On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:
> >
> > > Little side projects ftw. Very nice :-)
> > >
> > > Can you give some points on how this works internally? Is it making use
> > of
> > > anything generic from the Python API pull request?
> > >
> > > On Wednesday, April 1, 2015, Márton Balassi <[hidden email]>
> > > wrote:
> > >
> > > > Woot!
> > > >
> > > > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <
> [hidden email]
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Right now, runtime is roughly thrice that of equivalent java
> > programs.
> > > > > But I plan on bringing that to the same ballpark using code
> > > > > generation.
> > > > >
> > > > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
> > > > <javascript:;>> wrote:
> > > > > > :-D
> > > > > > This is awesome!
> > > > > >
> > > > > > Do you have some performance numbers?
> > > > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
> > > > <javascript:;>> wrote:
> > > > > >
> > > > > >> Hi,
> > > > > >> I've been working on a little side project in my free time: Ruby
> > on
> > > > > >> Flink (RoF). This should finally allow us to tap into the whole
> > web
> > > > > >> developer ruby world. The design is also flexible enough to
> easily
> > > > > >> port this to node.js.
> > > > > >>
> > > > > >> The code is not yet in shape to be made public but this is a
> small
> > > > > >> preview of what the API looks like:
> > > > > >>
> > > > > >> input = env.read_text_file("/some/path")
> > > > > >>
> > > > > >> words = input.flatMap do |line|
> > > > > >>   parts = line.split(/\W+/)
> > > > > >>   parts.each do |word|
> > > > > >>     word, 1
> > > > > >>   end
> > > > > >> end
> > > > > >>
> > > > > >> counts = words.reduce do |left, right|
> > > > > >>   left(1), left(1) + right(2)
> > > > > >> end
> > > > > >>
> > > > > >> counts.print()
> > > > > >>
> > > > > >> env.execute!()
> > > > > >>
> > > > > >> Cheers,
> > > > > >> Aljoscha
> > > > > >>
> > > > >
> > > >
> > >
> >
>

[Stephan Ewen]

For the Ruby interpreter on Flink, I suggest to implement this in Rust.

To implement a system that analyzes data at "web scale", what would be a
better language than one used to implement the engine of a "web browser".

On Wed, Apr 1, 2015 at 10:05 AM, Stephan Ewen <[hidden email]> wrote:

> It April, 1st, right?
>
> On Wed, Apr 1, 2015 at 9:47 AM, Gyula Fóra <[hidden email]> wrote:
>
>> Amazing! :)
>>
>> On Wed, Apr 1, 2015 at 9:41 AM, <[hidden email]> wrote:
>>
>> > My ruby skills are a bit rust(y) but I’d love to contribute.
>> >
>> >
>> > Can you point me to a repository that I can fork?
>> >
>> >
>> >
>> >
>> >
>> >
>> > From: Till Rohrmann
>> > Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29
>> > To: [hidden email]
>> >
>> >
>> >
>> >
>> >
>> > Where can I start contributing. I've got the feeling that it will be
>> > inherently faster than everything else which is out there.
>> > On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:
>> >
>> > > Little side projects ftw. Very nice :-)
>> > >
>> > > Can you give some points on how this works internally? Is it making
>> use
>> > of
>> > > anything generic from the Python API pull request?
>> > >
>> > > On Wednesday, April 1, 2015, Márton Balassi <[hidden email]
>> >
>> > > wrote:
>> > >
>> > > > Woot!
>> > > >
>> > > > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <
>> [hidden email]
>> > > > <javascript:;>>
>> > > > wrote:
>> > > >
>> > > > > Right now, runtime is roughly thrice that of equivalent java
>> > programs.
>> > > > > But I plan on bringing that to the same ballpark using code
>> > > > > generation.
>> > > > >
>> > > > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <[hidden email]
>> > > > <javascript:;>> wrote:
>> > > > > > :-D
>> > > > > > This is awesome!
>> > > > > >
>> > > > > > Do you have some performance numbers?
>> > > > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <[hidden email]
>> > > > <javascript:;>> wrote:
>> > > > > >
>> > > > > >> Hi,
>> > > > > >> I've been working on a little side project in my free time:
>> Ruby
>> > on
>> > > > > >> Flink (RoF). This should finally allow us to tap into the whole
>> > web
>> > > > > >> developer ruby world. The design is also flexible enough to
>> easily
>> > > > > >> port this to node.js.
>> > > > > >>
>> > > > > >> The code is not yet in shape to be made public but this is a
>> small
>> > > > > >> preview of what the API looks like:
>> > > > > >>
>> > > > > >> input = env.read_text_file("/some/path")
>> > > > > >>
>> > > > > >> words = input.flatMap do |line|
>> > > > > >>   parts = line.split(/\W+/)
>> > > > > >>   parts.each do |word|
>> > > > > >>     word, 1
>> > > > > >>   end
>> > > > > >> end
>> > > > > >>
>> > > > > >> counts = words.reduce do |left, right|
>> > > > > >>   left(1), left(1) + right(2)
>> > > > > >> end
>> > > > > >>
>> > > > > >> counts.print()
>> > > > > >>
>> > > > > >> env.execute!()
>> > > > > >>
>> > > > > >> Cheers,
>> > > > > >> Aljoscha
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

[mxm]

I'm in love with the API. So innovative and it feels very Rubyesk, even
though I'm a Python lover. Your WordCount shows some really nice features
which we should port to the Flink Java and Scala API. Perhaps, by embedding
a little Ruby interpreter.

Great idea also to port it to Node.js. Together with V8's stunning
performance, we can bring together Flink with all the great Node.js
libraries out there.

Thanks for all the hard work and dedication. With Ruby support, I feel like
Flink is now complete. Can't wait to announce this on the website. Users
will be thrilled!

On Wed, Apr 1, 2015 at 10:29 AM, Stephan Ewen <[hidden email]> wrote:

> For the Ruby interpreter on Flink, I suggest to implement this in Rust.
>
> To implement a system that analyzes data at "web scale", what would be a
> better language than one used to implement the engine of a "web browser".
>
> On Wed, Apr 1, 2015 at 10:05 AM, Stephan Ewen <[hidden email]> wrote:
>
> > It April, 1st, right?
> >
> > On Wed, Apr 1, 2015 at 9:47 AM, Gyula Fóra <[hidden email]> wrote:
> >
> >> Amazing! :)
> >>
> >> On Wed, Apr 1, 2015 at 9:41 AM, <[hidden email]> wrote:
> >>
> >> > My ruby skills are a bit rust(y) but I’d love to contribute.
> >> >
> >> >
> >> > Can you point me to a repository that I can fork?
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > From: Till Rohrmann
> >> > Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29
> >> > To: [hidden email]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Where can I start contributing. I've got the feeling that it will be
> >> > inherently faster than everything else which is out there.
> >> > On Apr 1, 2015 9:18 AM, "Ufuk Celebi" <[hidden email]> wrote:
> >> >
> >> > > Little side projects ftw. Very nice :-)
> >> > >
> >> > > Can you give some points on how this works internally? Is it making
> >> use
> >> > of
> >> > > anything generic from the Python API pull request?
> >> > >
> >> > > On Wednesday, April 1, 2015, Márton Balassi <
> [hidden email]
> >> >
> >> > > wrote:
> >> > >
> >> > > > Woot!
> >> > > >
> >> > > > On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek <
> >> [hidden email]
> >> > > > <javascript:;>>
> >> > > > wrote:
> >> > > >
> >> > > > > Right now, runtime is roughly thrice that of equivalent java
> >> > programs.
> >> > > > > But I plan on bringing that to the same ballpark using code
> >> > > > > generation.
> >> > > > >
> >> > > > > On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske <
> [hidden email]
> >> > > > <javascript:;>> wrote:
> >> > > > > > :-D
> >> > > > > > This is awesome!
> >> > > > > >
> >> > > > > > Do you have some performance numbers?
> >> > > > > > On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" <
> [hidden email]
> >> > > > <javascript:;>> wrote:
> >> > > > > >
> >> > > > > >> Hi,
> >> > > > > >> I've been working on a little side project in my free time:
> >> Ruby
> >> > on
> >> > > > > >> Flink (RoF). This should finally allow us to tap into the
> whole
> >> > web
> >> > > > > >> developer ruby world. The design is also flexible enough to
> >> easily
> >> > > > > >> port this to node.js.
> >> > > > > >>
> >> > > > > >> The code is not yet in shape to be made public but this is a
> >> small
> >> > > > > >> preview of what the API looks like:
> >> > > > > >>
> >> > > > > >> input = env.read_text_file("/some/path")
> >> > > > > >>
> >> > > > > >> words = input.flatMap do |line|
> >> > > > > >>   parts = line.split(/\W+/)
> >> > > > > >>   parts.each do |word|
> >> > > > > >>     word, 1
> >> > > > > >>   end
> >> > > > > >> end
> >> > > > > >>
> >> > > > > >> counts = words.reduce do |left, right|
> >> > > > > >>   left(1), left(1) + right(2)
> >> > > > > >> end
> >> > > > > >>
> >> > > > > >> counts.print()
> >> > > > > >>
> >> > > > > >> env.execute!()
> >> > > > > >>
> >> > > > > >> Cheers,
> >> > > > > >> Aljoscha
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>