Storm compatibility layer currently does not support Storm's SimpleJoin example

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Storm compatibility layer currently does not support Storm's SimpleJoin example

Szabó Péter
Hey everyone,

I experimented with the Storm compatibility layer Matthias wrote, and ran
some Storm examples on Flink. I found that Storm's SimpleJoin example does
not work. I suppose it is because of the multiple input streams. I'm
willing to add another example instead.
Right now, I'm getting it through Aljoscha's streaming refactor.

Peter
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Matthias J. Sax
Hi Peter,

Thanks a lot for your feedback. It's exiting to see, that somebody uses
the layer already. :)

The current prototype is going to be merged soon. However, I am more
than happy to extend the functionality of the layer. Can you please
share your example with me, so I can see what the problem is and fix it?

I am pretty sure, that the fix will be merged later on, too. There are
many other limitation in the layer. Right now, it is still in beta state. ;)

-Matthias


On 05/27/2015 03:48 PM, Szabó Péter wrote:

> Hey everyone,
>
> I experimented with the Storm compatibility layer Matthias wrote, and ran
> some Storm examples on Flink. I found that Storm's SimpleJoin example does
> not work. I suppose it is because of the multiple input streams. I'm
> willing to add another example instead.
> Right now, I'm getting it through Aljoscha's streaming refactor.
>
> Peter
>


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Szabó Péter
Hi Matthias,

Of course, here is the package that contains the example's source classes.
https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
separated the spouts and the join bolt from the rest of the topology.
I would be happy, if you could fix it. Probably I'm overlooking something.

Peter

2015-05-27 17:13 GMT+02:00 Matthias J. Sax <[hidden email]>:

> Hi Peter,
>
> Thanks a lot for your feedback. It's exiting to see, that somebody uses
> the layer already. :)
>
> The current prototype is going to be merged soon. However, I am more
> than happy to extend the functionality of the layer. Can you please
> share your example with me, so I can see what the problem is and fix it?
>
> I am pretty sure, that the fix will be merged later on, too. There are
> many other limitation in the layer. Right now, it is still in beta state.
> ;)
>
> -Matthias
>
>
> On 05/27/2015 03:48 PM, Szabó Péter wrote:
> > Hey everyone,
> >
> > I experimented with the Storm compatibility layer Matthias wrote, and ran
> > some Storm examples on Flink. I found that Storm's SimpleJoin example
> does
> > not work. I suppose it is because of the multiple input streams. I'm
> > willing to add another example instead.
> > Right now, I'm getting it through Aljoscha's streaming refactor.
> >
> > Peter
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Matthias J. Sax
Hi Peter,

I started to look into the issue. However, I could not find the
following classes in the git repository:

org.apache.flink.stormcompatibility.util.AbstractStormSpout
org.apache.flink.stormcompatibility.util.OutputFormatter
org.apache.flink.stormcompatibility.util.StormBoltFileSink
org.apache.flink.stormcompatibility.util.StormBoltPrintSink
org.apache.flink.stormcompatibility.util.TupleOutputFormatter

Thus, I cannot compile and run the code. Can you please update the git.

However, I had a quick look into the code, and see a few issues right
away. For example the code uses "tuple.select(...)" what is not
supported so far. Right now, attributes can only be accessed via index.
Furthermore, the example uses a lot of meta information that cannot be
provided easily in Flink (ie, TopologyContext is only supported very
limited).

To add those things, I will need much more time. I don't think I should
be part of the first pull request, but be added later. I will integrate
the SingleJoinBolt example as a ITCase if the functionality is there. It
seems to be a good idea, to add more examples from storm-starter to
flink-storm-examples.


-Matthias


On 05/28/2015 09:37 AM, Szabó Péter wrote:

> Hi Matthias,
>
> Of course, here is the package that contains the example's source classes.
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
> separated the spouts and the join bolt from the rest of the topology.
> I would be happy, if you could fix it. Probably I'm overlooking something.
>
> Peter
>
> 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <[hidden email]>:
>
>> Hi Peter,
>>
>> Thanks a lot for your feedback. It's exiting to see, that somebody uses
>> the layer already. :)
>>
>> The current prototype is going to be merged soon. However, I am more
>> than happy to extend the functionality of the layer. Can you please
>> share your example with me, so I can see what the problem is and fix it?
>>
>> I am pretty sure, that the fix will be merged later on, too. There are
>> many other limitation in the layer. Right now, it is still in beta state.
>> ;)
>>
>> -Matthias
>>
>>
>> On 05/27/2015 03:48 PM, Szabó Péter wrote:
>>> Hey everyone,
>>>
>>> I experimented with the Storm compatibility layer Matthias wrote, and ran
>>> some Storm examples on Flink. I found that Storm's SimpleJoin example
>> does
>>> not work. I suppose it is because of the multiple input streams. I'm
>>> willing to add another example instead.
>>> Right now, I'm getting it through Aljoscha's streaming refactor.
>>>
>>> Peter
>>>
>>
>>
>


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Szabó Péter
Thank you very much, this explains a lot of things :)
I'm aware of that currently the support of TopologyContext is limited, so I
do not expect it to work smoothly. However, there was another issue with
the grouping by the "id" field, which seemed very strange. Anyway, I will
live the SimpleJoin example to you, then. I will write if I find out
something important.

I think the Strom compatibility layer is a really nice extension of the
Flink streaming API. It would be great if you could add more functionality
to it in the future.
By the way, I'm not completely sure if you know: I was asked by Marton to
prepare your pull request for merging into the master. I restructured your
commits, cleaned up the code and rebased the branch on the current master.
Actually, to follow the changes in the behaviour of the Flink streaming
sources, I'm doing a second refactor right now. If you are working on
something, and have not rebased yet, you can use my storm branch to follow
up.

Here are the links to my storm branches:
- storm-backup (exmaples and other experiments; currently a little bit
outdated, because it is supposed to be a backup branch):
https://github.com/mbalassi/flink/tree/storm-backup
- storm (last clean state of my work on the flink-storm-compatibility pull
request, including code cleanup & refactor and one or two simple examples):
https://github.com/mbalassi/flink/tree/storm

Peter


2015-05-29 10:36 GMT+02:00 Matthias J. Sax <[hidden email]>:

> Hi Peter,
>
> I started to look into the issue. However, I could not find the
> following classes in the git repository:
>
> org.apache.flink.stormcompatibility.util.AbstractStormSpout
> org.apache.flink.stormcompatibility.util.OutputFormatter
> org.apache.flink.stormcompatibility.util.StormBoltFileSink
> org.apache.flink.stormcompatibility.util.StormBoltPrintSink
> org.apache.flink.stormcompatibility.util.TupleOutputFormatter
>
> Thus, I cannot compile and run the code. Can you please update the git.
>
> However, I had a quick look into the code, and see a few issues right
> away. For example the code uses "tuple.select(...)" what is not
> supported so far. Right now, attributes can only be accessed via index.
> Furthermore, the example uses a lot of meta information that cannot be
> provided easily in Flink (ie, TopologyContext is only supported very
> limited).
>
> To add those things, I will need much more time. I don't think I should
> be part of the first pull request, but be added later. I will integrate
> the SingleJoinBolt example as a ITCase if the functionality is there. It
> seems to be a good idea, to add more examples from storm-starter to
> flink-storm-examples.
>
>
> -Matthias
>
>
> On 05/28/2015 09:37 AM, Szabó Péter wrote:
> > Hi Matthias,
> >
> > Of course, here is the package that contains the example's source
> classes.
> >
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> > It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
> > separated the spouts and the join bolt from the rest of the topology.
> > I would be happy, if you could fix it. Probably I'm overlooking
> something.
> >
> > Peter
> >
> > 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <
> [hidden email]>:
> >
> >> Hi Peter,
> >>
> >> Thanks a lot for your feedback. It's exiting to see, that somebody uses
> >> the layer already. :)
> >>
> >> The current prototype is going to be merged soon. However, I am more
> >> than happy to extend the functionality of the layer. Can you please
> >> share your example with me, so I can see what the problem is and fix it?
> >>
> >> I am pretty sure, that the fix will be merged later on, too. There are
> >> many other limitation in the layer. Right now, it is still in beta
> state.
> >> ;)
> >>
> >> -Matthias
> >>
> >>
> >> On 05/27/2015 03:48 PM, Szabó Péter wrote:
> >>> Hey everyone,
> >>>
> >>> I experimented with the Storm compatibility layer Matthias wrote, and
> ran
> >>> some Storm examples on Flink. I found that Storm's SimpleJoin example
> >> does
> >>> not work. I suppose it is because of the multiple input streams. I'm
> >>> willing to add another example instead.
> >>> Right now, I'm getting it through Aljoscha's streaming refactor.
> >>>
> >>> Peter
> >>>
> >>
> >>
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Robert Metzger
Great to see that you two are working together on the storm compatibility
layer.

Please let the other Flink committers know when Matthias PR is in a state
that we can review it again (= when you think its ready).
Given the feedback from Peter and the long list of missing features and the
current rework, I would suggest to merge the storm compatiblilty layer
after the 0.9 release.
The issues from the document Stephan send around two weeks ago are making
good progress, so the release will probably forked-off rather soon.

On Fri, May 29, 2015 at 2:10 PM, Szabó Péter <[hidden email]>
wrote:

> Thank you very much, this explains a lot of things :)
> I'm aware of that currently the support of TopologyContext is limited, so I
> do not expect it to work smoothly. However, there was another issue with
> the grouping by the "id" field, which seemed very strange. Anyway, I will
> live the SimpleJoin example to you, then. I will write if I find out
> something important.
>
> I think the Strom compatibility layer is a really nice extension of the
> Flink streaming API. It would be great if you could add more functionality
> to it in the future.
> By the way, I'm not completely sure if you know: I was asked by Marton to
> prepare your pull request for merging into the master. I restructured your
> commits, cleaned up the code and rebased the branch on the current master.
> Actually, to follow the changes in the behaviour of the Flink streaming
> sources, I'm doing a second refactor right now. If you are working on
> something, and have not rebased yet, you can use my storm branch to follow
> up.
>
> Here are the links to my storm branches:
> - storm-backup (exmaples and other experiments; currently a little bit
> outdated, because it is supposed to be a backup branch):
> https://github.com/mbalassi/flink/tree/storm-backup
> - storm (last clean state of my work on the flink-storm-compatibility pull
> request, including code cleanup & refactor and one or two simple examples):
> https://github.com/mbalassi/flink/tree/storm
>
> Peter
>
>
> 2015-05-29 10:36 GMT+02:00 Matthias J. Sax <[hidden email]
> >:
>
> > Hi Peter,
> >
> > I started to look into the issue. However, I could not find the
> > following classes in the git repository:
> >
> > org.apache.flink.stormcompatibility.util.AbstractStormSpout
> > org.apache.flink.stormcompatibility.util.OutputFormatter
> > org.apache.flink.stormcompatibility.util.StormBoltFileSink
> > org.apache.flink.stormcompatibility.util.StormBoltPrintSink
> > org.apache.flink.stormcompatibility.util.TupleOutputFormatter
> >
> > Thus, I cannot compile and run the code. Can you please update the git.
> >
> > However, I had a quick look into the code, and see a few issues right
> > away. For example the code uses "tuple.select(...)" what is not
> > supported so far. Right now, attributes can only be accessed via index.
> > Furthermore, the example uses a lot of meta information that cannot be
> > provided easily in Flink (ie, TopologyContext is only supported very
> > limited).
> >
> > To add those things, I will need much more time. I don't think I should
> > be part of the first pull request, but be added later. I will integrate
> > the SingleJoinBolt example as a ITCase if the functionality is there. It
> > seems to be a good idea, to add more examples from storm-starter to
> > flink-storm-examples.
> >
> >
> > -Matthias
> >
> >
> > On 05/28/2015 09:37 AM, Szabó Péter wrote:
> > > Hi Matthias,
> > >
> > > Of course, here is the package that contains the example's source
> > classes.
> > >
> >
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> > > It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
> > > separated the spouts and the join bolt from the rest of the topology.
> > > I would be happy, if you could fix it. Probably I'm overlooking
> > something.
> > >
> > > Peter
> > >
> > > 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <
> > [hidden email]>:
> > >
> > >> Hi Peter,
> > >>
> > >> Thanks a lot for your feedback. It's exiting to see, that somebody
> uses
> > >> the layer already. :)
> > >>
> > >> The current prototype is going to be merged soon. However, I am more
> > >> than happy to extend the functionality of the layer. Can you please
> > >> share your example with me, so I can see what the problem is and fix
> it?
> > >>
> > >> I am pretty sure, that the fix will be merged later on, too. There are
> > >> many other limitation in the layer. Right now, it is still in beta
> > state.
> > >> ;)
> > >>
> > >> -Matthias
> > >>
> > >>
> > >> On 05/27/2015 03:48 PM, Szabó Péter wrote:
> > >>> Hey everyone,
> > >>>
> > >>> I experimented with the Storm compatibility layer Matthias wrote, and
> > ran
> > >>> some Storm examples on Flink. I found that Storm's SimpleJoin example
> > >> does
> > >>> not work. I suppose it is because of the multiple input streams. I'm
> > >>> willing to add another example instead.
> > >>> Right now, I'm getting it through Aljoscha's streaming refactor.
> > >>>
> > >>> Peter
> > >>>
> > >>
> > >>
> > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Szabó Péter
@Robert
Thanks! I think the PR will be ready to merge soon :)

@Matthias
I fixed the finite-source issue on my branch, now every example and ITCase
runs and stopps without throwing an exception. Also, in case of finite
sources, the spout wrapper will not loop infinitely.
I will study your branch and make comments in the afternoon.

Peter

2015-06-01 21:46 GMT+02:00 Robert Metzger <[hidden email]>:

> Great to see that you two are working together on the storm compatibility
> layer.
>
> Please let the other Flink committers know when Matthias PR is in a state
> that we can review it again (= when you think its ready).
> Given the feedback from Peter and the long list of missing features and the
> current rework, I would suggest to merge the storm compatiblilty layer
> after the 0.9 release.
> The issues from the document Stephan send around two weeks ago are making
> good progress, so the release will probably forked-off rather soon.
>
> On Fri, May 29, 2015 at 2:10 PM, Szabó Péter <[hidden email]>
> wrote:
>
> > Thank you very much, this explains a lot of things :)
> > I'm aware of that currently the support of TopologyContext is limited,
> so I
> > do not expect it to work smoothly. However, there was another issue with
> > the grouping by the "id" field, which seemed very strange. Anyway, I will
> > live the SimpleJoin example to you, then. I will write if I find out
> > something important.
> >
> > I think the Strom compatibility layer is a really nice extension of the
> > Flink streaming API. It would be great if you could add more
> functionality
> > to it in the future.
> > By the way, I'm not completely sure if you know: I was asked by Marton to
> > prepare your pull request for merging into the master. I restructured
> your
> > commits, cleaned up the code and rebased the branch on the current
> master.
> > Actually, to follow the changes in the behaviour of the Flink streaming
> > sources, I'm doing a second refactor right now. If you are working on
> > something, and have not rebased yet, you can use my storm branch to
> follow
> > up.
> >
> > Here are the links to my storm branches:
> > - storm-backup (exmaples and other experiments; currently a little bit
> > outdated, because it is supposed to be a backup branch):
> > https://github.com/mbalassi/flink/tree/storm-backup
> > - storm (last clean state of my work on the flink-storm-compatibility
> pull
> > request, including code cleanup & refactor and one or two simple
> examples):
> > https://github.com/mbalassi/flink/tree/storm
> >
> > Peter
> >
> >
> > 2015-05-29 10:36 GMT+02:00 Matthias J. Sax <
> [hidden email]
> > >:
> >
> > > Hi Peter,
> > >
> > > I started to look into the issue. However, I could not find the
> > > following classes in the git repository:
> > >
> > > org.apache.flink.stormcompatibility.util.AbstractStormSpout
> > > org.apache.flink.stormcompatibility.util.OutputFormatter
> > > org.apache.flink.stormcompatibility.util.StormBoltFileSink
> > > org.apache.flink.stormcompatibility.util.StormBoltPrintSink
> > > org.apache.flink.stormcompatibility.util.TupleOutputFormatter
> > >
> > > Thus, I cannot compile and run the code. Can you please update the git.
> > >
> > > However, I had a quick look into the code, and see a few issues right
> > > away. For example the code uses "tuple.select(...)" what is not
> > > supported so far. Right now, attributes can only be accessed via index.
> > > Furthermore, the example uses a lot of meta information that cannot be
> > > provided easily in Flink (ie, TopologyContext is only supported very
> > > limited).
> > >
> > > To add those things, I will need much more time. I don't think I should
> > > be part of the first pull request, but be added later. I will integrate
> > > the SingleJoinBolt example as a ITCase if the functionality is there.
> It
> > > seems to be a good idea, to add more examples from storm-starter to
> > > flink-storm-examples.
> > >
> > >
> > > -Matthias
> > >
> > >
> > > On 05/28/2015 09:37 AM, Szabó Péter wrote:
> > > > Hi Matthias,
> > > >
> > > > Of course, here is the package that contains the example's source
> > > classes.
> > > >
> > >
> >
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> > > > It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
> > > > separated the spouts and the join bolt from the rest of the topology.
> > > > I would be happy, if you could fix it. Probably I'm overlooking
> > > something.
> > > >
> > > > Peter
> > > >
> > > > 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <
> > > [hidden email]>:
> > > >
> > > >> Hi Peter,
> > > >>
> > > >> Thanks a lot for your feedback. It's exiting to see, that somebody
> > uses
> > > >> the layer already. :)
> > > >>
> > > >> The current prototype is going to be merged soon. However, I am more
> > > >> than happy to extend the functionality of the layer. Can you please
> > > >> share your example with me, so I can see what the problem is and fix
> > it?
> > > >>
> > > >> I am pretty sure, that the fix will be merged later on, too. There
> are
> > > >> many other limitation in the layer. Right now, it is still in beta
> > > state.
> > > >> ;)
> > > >>
> > > >> -Matthias
> > > >>
> > > >>
> > > >> On 05/27/2015 03:48 PM, Szabó Péter wrote:
> > > >>> Hey everyone,
> > > >>>
> > > >>> I experimented with the Storm compatibility layer Matthias wrote,
> and
> > > ran
> > > >>> some Storm examples on Flink. I found that Storm's SimpleJoin
> example
> > > >> does
> > > >>> not work. I suppose it is because of the multiple input streams.
> I'm
> > > >>> willing to add another example instead.
> > > >>> Right now, I'm getting it through Aljoscha's streaming refactor.
> > > >>>
> > > >>> Peter
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

Robert Metzger
It looks like there is a now a PR request available for the storm
compatibility: https://github.com/apache/flink/pull/764

It seems were are not the only new stream processing system with
compatibility to Storm: http://dl.acm.org/citation.cfm?id=2742788

On Tue, Jun 2, 2015 at 11:09 AM, Szabó Péter <[hidden email]>
wrote:

> @Robert
> Thanks! I think the PR will be ready to merge soon :)
>
> @Matthias
> I fixed the finite-source issue on my branch, now every example and ITCase
> runs and stopps without throwing an exception. Also, in case of finite
> sources, the spout wrapper will not loop infinitely.
> I will study your branch and make comments in the afternoon.
>
> Peter
>
> 2015-06-01 21:46 GMT+02:00 Robert Metzger <[hidden email]>:
>
> > Great to see that you two are working together on the storm compatibility
> > layer.
> >
> > Please let the other Flink committers know when Matthias PR is in a state
> > that we can review it again (= when you think its ready).
> > Given the feedback from Peter and the long list of missing features and
> the
> > current rework, I would suggest to merge the storm compatiblilty layer
> > after the 0.9 release.
> > The issues from the document Stephan send around two weeks ago are making
> > good progress, so the release will probably forked-off rather soon.
> >
> > On Fri, May 29, 2015 at 2:10 PM, Szabó Péter <[hidden email]>
> > wrote:
> >
> > > Thank you very much, this explains a lot of things :)
> > > I'm aware of that currently the support of TopologyContext is limited,
> > so I
> > > do not expect it to work smoothly. However, there was another issue
> with
> > > the grouping by the "id" field, which seemed very strange. Anyway, I
> will
> > > live the SimpleJoin example to you, then. I will write if I find out
> > > something important.
> > >
> > > I think the Strom compatibility layer is a really nice extension of the
> > > Flink streaming API. It would be great if you could add more
> > functionality
> > > to it in the future.
> > > By the way, I'm not completely sure if you know: I was asked by Marton
> to
> > > prepare your pull request for merging into the master. I restructured
> > your
> > > commits, cleaned up the code and rebased the branch on the current
> > master.
> > > Actually, to follow the changes in the behaviour of the Flink streaming
> > > sources, I'm doing a second refactor right now. If you are working on
> > > something, and have not rebased yet, you can use my storm branch to
> > follow
> > > up.
> > >
> > > Here are the links to my storm branches:
> > > - storm-backup (exmaples and other experiments; currently a little bit
> > > outdated, because it is supposed to be a backup branch):
> > > https://github.com/mbalassi/flink/tree/storm-backup
> > > - storm (last clean state of my work on the flink-storm-compatibility
> > pull
> > > request, including code cleanup & refactor and one or two simple
> > examples):
> > > https://github.com/mbalassi/flink/tree/storm
> > >
> > > Peter
> > >
> > >
> > > 2015-05-29 10:36 GMT+02:00 Matthias J. Sax <
> > [hidden email]
> > > >:
> > >
> > > > Hi Peter,
> > > >
> > > > I started to look into the issue. However, I could not find the
> > > > following classes in the git repository:
> > > >
> > > > org.apache.flink.stormcompatibility.util.AbstractStormSpout
> > > > org.apache.flink.stormcompatibility.util.OutputFormatter
> > > > org.apache.flink.stormcompatibility.util.StormBoltFileSink
> > > > org.apache.flink.stormcompatibility.util.StormBoltPrintSink
> > > > org.apache.flink.stormcompatibility.util.TupleOutputFormatter
> > > >
> > > > Thus, I cannot compile and run the code. Can you please update the
> git.
> > > >
> > > > However, I had a quick look into the code, and see a few issues right
> > > > away. For example the code uses "tuple.select(...)" what is not
> > > > supported so far. Right now, attributes can only be accessed via
> index.
> > > > Furthermore, the example uses a lot of meta information that cannot
> be
> > > > provided easily in Flink (ie, TopologyContext is only supported very
> > > > limited).
> > > >
> > > > To add those things, I will need much more time. I don't think I
> should
> > > > be part of the first pull request, but be added later. I will
> integrate
> > > > the SingleJoinBolt example as a ITCase if the functionality is there.
> > It
> > > > seems to be a good idea, to add more examples from storm-starter to
> > > > flink-storm-examples.
> > > >
> > > >
> > > > -Matthias
> > > >
> > > >
> > > > On 05/28/2015 09:37 AM, Szabó Péter wrote:
> > > > > Hi Matthias,
> > > > >
> > > > > Of course, here is the package that contains the example's source
> > > > classes.
> > > > >
> > > >
> > >
> >
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> > > > > It is mostly a copy-paste of SimpleJoin from storm-starter,
> though, I
> > > > > separated the spouts and the join bolt from the rest of the
> topology.
> > > > > I would be happy, if you could fix it. Probably I'm overlooking
> > > > something.
> > > > >
> > > > > Peter
> > > > >
> > > > > 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <
> > > > [hidden email]>:
> > > > >
> > > > >> Hi Peter,
> > > > >>
> > > > >> Thanks a lot for your feedback. It's exiting to see, that somebody
> > > uses
> > > > >> the layer already. :)
> > > > >>
> > > > >> The current prototype is going to be merged soon. However, I am
> more
> > > > >> than happy to extend the functionality of the layer. Can you
> please
> > > > >> share your example with me, so I can see what the problem is and
> fix
> > > it?
> > > > >>
> > > > >> I am pretty sure, that the fix will be merged later on, too. There
> > are
> > > > >> many other limitation in the layer. Right now, it is still in beta
> > > > state.
> > > > >> ;)
> > > > >>
> > > > >> -Matthias
> > > > >>
> > > > >>
> > > > >> On 05/27/2015 03:48 PM, Szabó Péter wrote:
> > > > >>> Hey everyone,
> > > > >>>
> > > > >>> I experimented with the Storm compatibility layer Matthias wrote,
> > and
> > > > ran
> > > > >>> some Storm examples on Flink. I found that Storm's SimpleJoin
> > example
> > > > >> does
> > > > >>> not work. I suppose it is because of the multiple input streams.
> > I'm
> > > > >>> willing to add another example instead.
> > > > >>> Right now, I'm getting it through Aljoscha's streaming refactor.
> > > > >>>
> > > > >>> Peter
> > > > >>>
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
>