Events not received

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Events not received

Hermann Gábor
Hi all,

We've been trying to use events for two-way communication, but it seems
like some of the backward events (at a BufferReader) are not sent at all if
the connection is not initialized yet (i.e. the given input channel is
UNKNOWN).

Is there a way to make sure the events get to the destination?

Cheers,
Gabor
Reply | Threaded
Open this post in threaded view
|

Re: Events not received

Ufuk Celebi-2

On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote:

> Hi all,
>
> We've been trying to use events for two-way communication, but it seems
> like some of the backward events (at a BufferReader) are not sent at all if
> the connection is not initialized yet (i.e. the given input channel is
> UNKNOWN).
>
> Is there a way to make sure the events get to the destination?

That's a very good point and the current behaviour is definitely a bug. Events going to an input channel, which has not been initialized yet are lost. Let me think about this for a sec and then push a fix.

– Ufuk
Reply | Threaded
Open this post in threaded view
|

Re: Events not received

Ufuk Celebi-2
In reply to this post by Hermann Gábor
Hey Gabor,

can you try this branch: https://github.com/uce/incubator-flink/tree/flink-1496-lost

Is this working for you?

This is a quick fix. It keeps track of events, which could not be delivered to uninitialized channels and delivers them after they are initialized.

(The BufferReader, which does this, needs some refactoring. I think I blew it up quite a bit when I "merged" it with the InputGate. I think I will revert that soon.)

On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote:

> Hi all,
>
> We've been trying to use events for two-way communication, but it seems
> like some of the backward events (at a BufferReader) are not sent at all if
> the connection is not initialized yet (i.e. the given input channel is
> UNKNOWN).
>
> Is there a way to make sure the events get to the destination?
>
> Cheers,
> Gabor

Reply | Threaded
Open this post in threaded view
|

Re: Events not received

Hermann Gábor
Hey Ufuk,

Thanks for the fast reply and fix! It's working now :)

Unfortunately the "other side" of the AbstractInvokable lifecycle is
causing a bit of a trouble,
as the AbstractInvokable does not wait for possible events before
finishing, and it throws an
IllegalStateException: Tried to send task event to producer before
requesting a queue.
exception when trying to send an event after the task finished.

For now we have a workaround to this (the above fixed problem was the
trickier one for us) in a higher level, but it would be
nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the
AbstractInvokable would be kept up
until all Readers/Writers that subscribed to events are shut down.

Cheers,
Gabor

On Mon Feb 09 2015 at 5:59:19 PM Ufuk Celebi <[hidden email]> wrote:

> Hey Gabor,
>
> can you try this branch: https://github.com/uce/
> incubator-flink/tree/flink-1496-lost
>
> Is this working for you?
>
> This is a quick fix. It keeps track of events, which could not be
> delivered to uninitialized channels and delivers them after they are
> initialized.
>
> (The BufferReader, which does this, needs some refactoring. I think I blew
> it up quite a bit when I "merged" it with the InputGate. I think I will
> revert that soon.)
>
> On 09 Feb 2015, at 15:06, Hermann Gábor <[hidden email]> wrote:
>
> > Hi all,
> >
> > We've been trying to use events for two-way communication, but it seems
> > like some of the backward events (at a BufferReader) are not sent at all
> if
> > the connection is not initialized yet (i.e. the given input channel is
> > UNKNOWN).
> >
> > Is there a way to make sure the events get to the destination?
> >
> > Cheers,
> > Gabor
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Events not received

Ufuk Celebi-2

On 10 Feb 2015, at 02:13, Hermann Gábor <[hidden email]> wrote:

> Hey Ufuk,
>
> Thanks for the fast reply and fix! It's working now :)
>
> Unfortunately the "other side" of the AbstractInvokable lifecycle is
> causing a bit of a trouble,
> as the AbstractInvokable does not wait for possible events before
> finishing, and it throws an
> IllegalStateException: Tried to send task event to producer before
> requesting a queue.
> exception when trying to send an event after the task finished.
>
> For now we have a workaround to this (the above fixed problem was the
> trickier one for us) in a higher level, but it would be
> nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the
> AbstractInvokable would be kept up
> until all Readers/Writers that subscribed to events are shut down.

OK, good to hear that it is working. The producer and consumer tasks used to wait for each other before they finished. With iterations and superstep events, the iteration task logic ensures that tasks don't finish prematurely.

How are you doing it? Waiting a certain number of events?
Reply | Threaded
Open this post in threaded view
|

Re: Events not received

Hermann Gábor
In our case we are actually waiting for some kind of acknowledgement events,
so it is easily verifiable if there are any further events. We check
whether everything
has been acknowledged.

On Tue Feb 10 2015 at 9:42:14 AM Ufuk Celebi <[hidden email]> wrote:

>
> On 10 Feb 2015, at 02:13, Hermann Gábor <[hidden email]> wrote:
>
> > Hey Ufuk,
> >
> > Thanks for the fast reply and fix! It's working now :)
> >
> > Unfortunately the "other side" of the AbstractInvokable lifecycle is
> > causing a bit of a trouble,
> > as the AbstractInvokable does not wait for possible events before
> > finishing, and it throws an
> > IllegalStateException: Tried to send task event to producer before
> > requesting a queue.
> > exception when trying to send an event after the task finished.
> >
> > For now we have a workaround to this (the above fixed problem was the
> > trickier one for us) in a higher level, but it would be
> > nice if the events adjusted to the AbstractInvokable lifecycle, i.e. the
> > AbstractInvokable would be kept up
> > until all Readers/Writers that subscribed to events are shut down.
>
> OK, good to hear that it is working. The producer and consumer tasks used
> to wait for each other before they finished. With iterations and superstep
> events, the iteration task logic ensures that tasks don't finish
> prematurely.
>
> How are you doing it? Waiting a certain number of events?