[QUESTION] In the DataExchangeMode.BATCH how does the consumer being notified to start consumer the data?

[Henry Saputra]

HI Flinksters,

Have one quick question, hopefully someone could help me get some answers:

When is the DataExchangeMode.BATCH is actually being used, and how
does the consumer know when to start consuming the buffer

In the pipelined mode, the producer tell JobManager via JobManager !
ScheduleOrUpdateConsumers to create network connection. But I could
not find trigger for blocking/ batch mode of execution.

- Henry

[Ufuk Celebi-2]

Hey Henry,

1) There is no extra message, but this is piggy backed with the finished
state transition (see Execution#markAsFinished). There it is essentially
the same mechanism.

2) It's part of my plan for this week to add  documentation for exactly
this flow of RPC messages related to the runtime results.

– Ufuk

On Tuesday, March 31, 2015, Henry Saputra <[hidden email]> wrote:

> HI Flinksters,
>
> Have one quick question, hopefully someone could help me get some answers:
>
> When is the DataExchangeMode.BATCH is actually being used, and how
> does the consumer know when to start consuming the buffer
>
> In the pipelined mode, the producer tell JobManager via JobManager !
> ScheduleOrUpdateConsumers to create network connection. But I could
> not find trigger for blocking/ batch mode of execution.
>
> - Henry
>

[Henry Saputra]

Ah cool. Thanks for the explanation, Ufuk.
Really helpful.

- Henry

On Tuesday, March 31, 2015, Ufuk Celebi <[hidden email]> wrote:

> Hey Henry,
>
> 1) There is no extra message, but this is piggy backed with the finished
> state transition (see Execution#markAsFinished). There it is essentially
> the same mechanism.
>
> 2) It's part of my plan for this week to add  documentation for exactly
> this flow of RPC messages related to the runtime results.
>
> – Ufuk
>
> On Tuesday, March 31, 2015, Henry Saputra <[hidden email]
> <javascript:;>> wrote:
>
> > HI Flinksters,
> >
> > Have one quick question, hopefully someone could help me get some
> answers:
> >
> > When is the DataExchangeMode.BATCH is actually being used, and how
> > does the consumer know when to start consuming the buffer
> >
> > In the pipelined mode, the producer tell JobManager via JobManager !
> > ScheduleOrUpdateConsumers to create network connection. But I could
> > not find trigger for blocking/ batch mode of execution.
> >
> > - Henry
> >
>