Clarity on Flink 1.5 Rescale mechanism

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Clarity on Flink 1.5 Rescale mechanism

Sampath Bhat
Hello
In flink 1.5 release notes -
https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes

Various Other Features and Improvements:
Applications can be rescaled without manually triggering a savepoint. Under
the hood, Flink will still take a savepoint, stop the application, and
rescale it to the new parallelism.

What exactly does this statement mean?
How to rescale the application ?
If i'm running my flink job application with parallelism of 5 and then I
execute the following command
flink run <jar_path> -p 10 then will the application rescale to 10
parallelism?

Adding on what will be the behavior in flink 1.5 if I increase the number
of task manager in a flink cluster?

So if i have a flink job running on flink cluster having 1 task manager and
i increase the number of task manager to 2 will the flink rescale the flink
job also or will the flink job be unaffected?
Reply | Threaded
Open this post in threaded view
|

Re: Clarity on Flink 1.5 Rescale mechanism

Stefan Richter
Hi,

it means that you can now modify the parallelism of a running job with a new „modify“ command in the CLI, see [1]. Adding task manager will add their offered slots to the pool of available slots, it will not automatically change the parallelism.

Best,
Stefan

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html <https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html>

> Am 12.06.2018 um 11:50 schrieb Sampath Bhat <[hidden email]>:
>
> Hello
> In flink 1.5 release notes -
> https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes
>
> Various Other Features and Improvements:
> Applications can be rescaled without manually triggering a savepoint. Under
> the hood, Flink will still take a savepoint, stop the application, and
> rescale it to the new parallelism.
>
> What exactly does this statement mean?
> How to rescale the application ?
> If i'm running my flink job application with parallelism of 5 and then I
> execute the following command
> flink run <jar_path> -p 10 then will the application rescale to 10
> parallelism?
>
> Adding on what will be the behavior in flink 1.5 if I increase the number
> of task manager in a flink cluster?
>
> So if i have a flink job running on flink cluster having 1 task manager and
> i increase the number of task manager to 2 will the flink rescale the flink
> job also or will the flink job be unaffected?