Dynamic Scaling: State Sharding

[Till Rohrmann]

Hi Flink community,

I recently started working on dynamic scaling. As a first step we want to
introduce state sharding which is a requirement for partitioned state to be
re-distributable. The following design doc [1] explains the ideas behind
it. Any feedback is highly appreciated :-)

[1]
https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing

Cheers,
Till

[Aljoscha Krettek-2]

Comments are not enabled.

On Mon, 4 Apr 2016 at 13:58 Till Rohrmann <[hidden email]> wrote:

> Hi Flink community,
>
> I recently started working on dynamic scaling. As a first step we want to
> introduce state sharding which is a requirement for partitioned state to be
> re-distributable. The following design doc [1] explains the ideas behind
> it. Any feedback is highly appreciated :-)
>
> [1]
>
> https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing
>
> Cheers,
> Till
>

[Till Rohrmann]

Sorry, my bad. Comments should work now.

On Mon, Apr 4, 2016 at 3:51 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Comments are not enabled.
>
> On Mon, 4 Apr 2016 at 13:58 Till Rohrmann <[hidden email]> wrote:
>
> > Hi Flink community,
> >
> > I recently started working on dynamic scaling. As a first step we want to
> > introduce state sharding which is a requirement for partitioned state to
> be
> > re-distributable. The following design doc [1] explains the ideas behind
> > it. Any feedback is highly appreciated :-)
> >
> > [1]
> >
> >
> https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing
> >
> > Cheers,
> > Till
> >
>

[Mazen Ezzeddine]

So what is the state of dynamic scaling (non static, set by the user in the
code) for SLA gurantees now, is it already included in FLink, and is it
still necessaru to stop/restart the job for dynamic scaling.

Thanks.



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[Till Rohrmann]

Hi Mazen,

Flink does not support automatic scaling yet. Hence, the recommended way is
to monitor a Flink job and to trigger rescaling by stopping the job with a
savepoint and then to resume it with the adjusted parallelism. The
community is working on auto scaling but there is no concrete date yet when
it will be available.

Cheers,
Till

On Mon, Oct 5, 2020 at 8:47 AM Mazen Ezzeddine <[hidden email]>
wrote:

> So what is the state of dynamic scaling (non static, set by the user in the
> code) for SLA gurantees now, is it already included in FLink, and is it
> still necessaru to stop/restart the job for dynamic scaling.
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>