Is there any way to change operator level parallelism dynamically?

[DEEP NARAYAN Singh]

Hi Team,

I just want quick help here. How to achieve the dynamic nature of
operator level parallelism for the flink job running in AWS EMR cluster
during runtime to avoid downtime and backpressure based on the incoming
load. As I am very new to flink and currently using flink 1.9 version.Is
there any way to solve this problem during runtime?.

Any lead will be highly appreciated.

Thanks & Regards,
-Deep

[Till Rohrmann]

Hi Deep,

at the moment the only way to change the parallelism is to take a
savepoint, stop the job and then resume the job with a changed parallelism
from the savepoint. The community works on a more automated way for this
problem. However, in the first versions it will use the same approach with
taking a savepoint which means that there will be a slight down time.

Cheers,
Till

On Wed, Oct 28, 2020 at 7:05 PM DEEP NARAYAN Singh <[hidden email]>
wrote:

> Hi Team,
>
> I just want quick help here. How to achieve the dynamic nature of
> operator level parallelism for the flink job running in AWS EMR cluster
> during runtime to avoid downtime and backpressure based on the incoming
> load. As I am very new to flink and currently using flink 1.9 version.Is
> there any way to solve this problem during runtime?.
>
> Any lead will be highly appreciated.
>
> Thanks & Regards,
> -Deep
>

[DEEP NARAYAN Singh]

Hi Till,

Got it.thanks for the prompt reply.

-Deep

On Thu, 29 Oct, 2020, 1:22 PM Till Rohrmann, <[hidden email]> wrote:

> Hi Deep,
>
> at the moment the only way to change the parallelism is to take a
> savepoint, stop the job and then resume the job with a changed parallelism
> from the savepoint. The community works on a more automated way for this
> problem. However, in the first versions it will use the same approach with
> taking a savepoint which means that there will be a slight down time.
>
> Cheers,
> Till
>
> On Wed, Oct 28, 2020 at 7:05 PM DEEP NARAYAN Singh <[hidden email]>
> wrote:
>
> > Hi Team,
> >
> > I just want quick help here. How to achieve the dynamic nature of
> > operator level parallelism for the flink job running in AWS EMR cluster
> > during runtime to avoid downtime and backpressure based on the incoming
> > load. As I am very new to flink and currently using flink 1.9 version.Is
> > there any way to solve this problem during runtime?.
> >
> > Any lead will be highly appreciated.
> >
> > Thanks & Regards,
> > -Deep
> >
>