Static deployment scheme for failure scenario

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Static deployment scheme for failure scenario

Niklas Semmler
Hello Flink Community,

is it possible to force a job into a particular task-node placement?

I want to run a failure scenario, which depends on the machines'
configuration.

To my knowledge the current scheduler deploys tasks in a
first-come-first-serve fashion. In that case the only way to fix the
deployment scheme would be to change the scheduler, right?

Did anyone attempt this so far?

Best,
Niklas

--
Niklas Semmler
PhD Student / Research Assistant
TU Berlin, INET, Room MAR 4.027
Marchstr 23, 10587 Berlin
Tel.: +49 (0)30 314 75739
http://inet.tu-berlin.de/~nsemmler/
Reply | Threaded
Open this post in threaded view
|

Re: Static deployment scheme for failure scenario

Till Rohrmann
Hi Niklas,

the current Scheduler implementation is designed with location transparency
in mind. Tasks can express a location preference which is currently based
on location constraints wrt other tasks. However, the community is thinking
about enhancing the scheduling logic such that it can respect node
preferences more explicitly in order to support mounted persistent disks
for state storage.

To make a long story short, for your use case you will most likely have to
change the Scheduler at the moment.

Cheers,
Till

On Mon, May 15, 2017 at 7:47 PM, Niklas Semmler <[hidden email]>
wrote:

> Hello Flink Community,
>
> is it possible to force a job into a particular task-node placement?
>
> I want to run a failure scenario, which depends on the machines'
> configuration.
>
> To my knowledge the current scheduler deploys tasks in a
> first-come-first-serve fashion. In that case the only way to fix the
> deployment scheme would be to change the scheduler, right?
>
> Did anyone attempt this so far?
>
> Best,
> Niklas
>
> --
> Niklas Semmler
> PhD Student / Research Assistant
> TU Berlin, INET, Room MAR 4.027
> Marchstr 23, 10587 Berlin
> Tel.: +49 (0)30 314 75739
> http://inet.tu-berlin.de/~nsemmler/
>