Hi all,
We've been trying to use events for two-way communication, but it seems like some of the backward events (at a BufferReader) are not sent at all if the connection is not initialized yet (i.e. the given input channel is UNKNOWN). Is there a way to make sure the events get to the destination? Cheers, Gabor |
On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote: > Hi all, > > We've been trying to use events for two-way communication, but it seems > like some of the backward events (at a BufferReader) are not sent at all if > the connection is not initialized yet (i.e. the given input channel is > UNKNOWN). > > Is there a way to make sure the events get to the destination? That's a very good point and the current behaviour is definitely a bug. Events going to an input channel, which has not been initialized yet are lost. Let me think about this for a sec and then push a fix. – Ufuk |
In reply to this post by Hermann Gábor
Hey Gabor,
can you try this branch: https://github.com/uce/incubator-flink/tree/flink-1496-lost Is this working for you? This is a quick fix. It keeps track of events, which could not be delivered to uninitialized channels and delivers them after they are initialized. (The BufferReader, which does this, needs some refactoring. I think I blew it up quite a bit when I "merged" it with the InputGate. I think I will revert that soon.) On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote: > Hi all, > > We've been trying to use events for two-way communication, but it seems > like some of the backward events (at a BufferReader) are not sent at all if > the connection is not initialized yet (i.e. the given input channel is > UNKNOWN). > > Is there a way to make sure the events get to the destination? > > Cheers, > Gabor |
Hey Ufuk,
Thanks for the fast reply and fix! It's working now :) Unfortunately the "other side" of the AbstractInvokable lifecycle is causing a bit of a trouble, as the AbstractInvokable does not wait for possible events before finishing, and it throws an IllegalStateException: Tried to send task event to producer before requesting a queue. exception when trying to send an event after the task finished. For now we have a workaround to this (the above fixed problem was the trickier one for us) in a higher level, but it would be nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the AbstractInvokable would be kept up until all Readers/Writers that subscribed to events are shut down. Cheers, Gabor On Mon Feb 09 2015 at 5:59:19 PM Ufuk Celebi <[hidden email]> wrote: > Hey Gabor, > > can you try this branch: https://github.com/uce/ > incubator-flink/tree/flink-1496-lost > > Is this working for you? > > This is a quick fix. It keeps track of events, which could not be > delivered to uninitialized channels and delivers them after they are > initialized. > > (The BufferReader, which does this, needs some refactoring. I think I blew > it up quite a bit when I "merged" it with the InputGate. I think I will > revert that soon.) > > On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote: > > > Hi all, > > > > We've been trying to use events for two-way communication, but it seems > > like some of the backward events (at a BufferReader) are not sent at all > if > > the connection is not initialized yet (i.e. the given input channel is > > UNKNOWN). > > > > Is there a way to make sure the events get to the destination? > > > > Cheers, > > Gabor > > |
On 10 Feb 2015, at 02:13, Hermann Gábor <[hidden email]> wrote: > Hey Ufuk, > > Thanks for the fast reply and fix! It's working now :) > > Unfortunately the "other side" of the AbstractInvokable lifecycle is > causing a bit of a trouble, > as the AbstractInvokable does not wait for possible events before > finishing, and it throws an > IllegalStateException: Tried to send task event to producer before > requesting a queue. > exception when trying to send an event after the task finished. > > For now we have a workaround to this (the above fixed problem was the > trickier one for us) in a higher level, but it would be > nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the > AbstractInvokable would be kept up > until all Readers/Writers that subscribed to events are shut down. OK, good to hear that it is working. The producer and consumer tasks used to wait for each other before they finished. With iterations and superstep events, the iteration task logic ensures that tasks don't finish prematurely. How are you doing it? Waiting a certain number of events? |
In our case we are actually waiting for some kind of acknowledgement events,
so it is easily verifiable if there are any further events. We check whether everything has been acknowledged. On Tue Feb 10 2015 at 9:42:14 AM Ufuk Celebi <[hidden email]> wrote: > > On 10 Feb 2015, at 02:13, Hermann Gábor <[hidden email]> wrote: > > > Hey Ufuk, > > > > Thanks for the fast reply and fix! It's working now :) > > > > Unfortunately the "other side" of the AbstractInvokable lifecycle is > > causing a bit of a trouble, > > as the AbstractInvokable does not wait for possible events before > > finishing, and it throws an > > IllegalStateException: Tried to send task event to producer before > > requesting a queue. > > exception when trying to send an event after the task finished. > > > > For now we have a workaround to this (the above fixed problem was the > > trickier one for us) in a higher level, but it would be > > nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the > > AbstractInvokable would be kept up > > until all Readers/Writers that subscribed to events are shut down. > > OK, good to hear that it is working. The producer and consumer tasks used > to wait for each other before they finished. With iterations and superstep > events, the iteration task logic ensures that tasks don't finish > prematurely. > > How are you doing it? Waiting a certain number of events? |
Free forum by Nabble | Edit this page |