Dataset and eager scheduling

classic Classic list List threaded Threaded
3 messages Options
CPC
Reply | Threaded
Open this post in threaded view
|

Dataset and eager scheduling

CPC
Hi all,

Currently our team trying implement a runtime operator also playing with
scheduler. We are trying to understand batch optimizer but it will take
some time. What we want to know is whether changing batch scheduling mode
from LAZY_FROM_SOURCES to EAGER could affect optimizer? I mean whether
optimizer have some strong assumptions that batch jobs scheduling mode is
always lazy_from_sources?

Thanks...
Reply | Threaded
Open this post in threaded view
|

Re: Dataset and eager scheduling

Till Rohrmann
Hi CPC,

I think that the optimizer does not take the scheduling mode into account
when optimizing a Flink job.

Cheers,
Till

On Thu, Mar 2, 2017 at 11:52 AM, CPC <[hidden email]> wrote:

> Hi all,
>
> Currently our team trying implement a runtime operator also playing with
> scheduler. We are trying to understand batch optimizer but it will take
> some time. What we want to know is whether changing batch scheduling mode
> from LAZY_FROM_SOURCES to EAGER could affect optimizer? I mean whether
> optimizer have some strong assumptions that batch jobs scheduling mode is
> always lazy_from_sources?
>
> Thanks...
>
CPC
Reply | Threaded
Open this post in threaded view
|

Re: Dataset and eager scheduling

CPC
Hi Till,

Thank you.



On 2 March 2017 at 18:13, Till Rohrmann <[hidden email]> wrote:

> Hi CPC,
>
> I think that the optimizer does not take the scheduling mode into account
> when optimizing a Flink job.
>
> Cheers,
> Till
>
> On Thu, Mar 2, 2017 at 11:52 AM, CPC <[hidden email]> wrote:
>
> > Hi all,
> >
> > Currently our team trying implement a runtime operator also playing with
> > scheduler. We are trying to understand batch optimizer but it will take
> > some time. What we want to know is whether changing batch scheduling mode
> > from LAZY_FROM_SOURCES to EAGER could affect optimizer? I mean whether
> > optimizer have some strong assumptions that batch jobs scheduling mode is
> > always lazy_from_sources?
> >
> > Thanks...
> >
>