(DEPRECATED) Apache Flink Mailing List archive.

[DISCUSS] Releasing Stateful Functions 2.1.0 soon?

Classic

List

Threaded

11 messages Options

Tzu-Li (Gordon) Tai

[DISCUSS] Releasing Stateful Functions 2.1.0 soon?

Hi devs,

Since Stateful Functions 2.0 was released early April,
we've been getting some good feedback from various channels,
including the Flink mailing lists, JIRA issues, as well as Stack Overflow
questions.

Some of the discussions have actually translated into new features
currently being implemented into the project, such as:

- State TTL for the state primitives in Stateful Functions (for both
embedded/remote functions)
- Transport for remote functions via UNIX domain sockets, which would be
useful when remote functions are co-located with Flink StateFun workers
(i.e. the "sidecar" deployment mode)

Besides that, some critical shortcomings have already been addressed since
the last release:

- After upgrading to Flink 1.10.1, failure recovery in Stateful
Functions now works properly with the new scheduler.
- Support for concurrent checkpoints

With these ongoing threads, while it's only been just short of 2 months
since the last release,
we (Igal Shilman and I) have been thinking about aiming to already start
the next feature release (2.1.0) soon.
This is relatively shorter than the release cycle of what the community is
used to in Flink (usually 3 months at least),
but we think with the StateFun project in its early phases, having smaller
and more frequent feature releases could potentially help drive user
adoption.

So, what do you think about setting feature freeze for StateFun 2.1.0 by
next Wednesday (May 27th)?
Of course, whether or not to actually have another feature release already
is still an open discussion - if you prefer a richer feature release with
more features included besides the ones listed above, please do comment!

Cheers,
Gordon

Robert Metzger

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

I like the idea of releasing Statefun more frequently to have faster
feedback cycles!

No objections for releasing 2.1.0 from my side.

On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <[hidden email]>
wrote:

> Hi devs,
>
> Since Stateful Functions 2.0 was released early April,
> we've been getting some good feedback from various channels,
> including the Flink mailing lists, JIRA issues, as well as Stack Overflow
> questions.
>
> Some of the discussions have actually translated into new features
> currently being implemented into the project, such as:
>
> - State TTL for the state primitives in Stateful Functions (for both
> embedded/remote functions)
> - Transport for remote functions via UNIX domain sockets, which would be
> useful when remote functions are co-located with Flink StateFun workers
> (i.e. the "sidecar" deployment mode)
>
>
> Besides that, some critical shortcomings have already been addressed since
> the last release:
>
> - After upgrading to Flink 1.10.1, failure recovery in Stateful
> Functions now works properly with the new scheduler.
> - Support for concurrent checkpoints
>
>
> With these ongoing threads, while it's only been just short of 2 months
> since the last release,
> we (Igal Shilman and I) have been thinking about aiming to already start
> the next feature release (2.1.0) soon.
> This is relatively shorter than the release cycle of what the community is
> used to in Flink (usually 3 months at least),
> but we think with the StateFun project in its early phases, having smaller
> and more frequent feature releases could potentially help drive user
> adoption.
>
> So, what do you think about setting feature freeze for StateFun 2.1.0 by
> next Wednesday (May 27th)?
> Of course, whether or not to actually have another feature release already
> is still an open discussion - if you prefer a richer feature release with
> more features included besides the ones listed above, please do comment!
>
> Cheers,
> Gordon
>

Seth Wiesman-4

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

+1 for a fast release cycle

Seth

On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]> wrote:

> I like the idea of releasing Statefun more frequently to have faster
> feedback cycles!
>
> No objections for releasing 2.1.0 from my side.
>
> On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <[hidden email]>
> wrote:
>
> > Hi devs,
> >
> > Since Stateful Functions 2.0 was released early April,
> > we've been getting some good feedback from various channels,
> > including the Flink mailing lists, JIRA issues, as well as Stack Overflow
> > questions.
> >
> > Some of the discussions have actually translated into new features
> > currently being implemented into the project, such as:
> >
> > - State TTL for the state primitives in Stateful Functions (for both
> > embedded/remote functions)
> > - Transport for remote functions via UNIX domain sockets, which would
> be
> > useful when remote functions are co-located with Flink StateFun
> workers
> > (i.e. the "sidecar" deployment mode)
> >
> >
> > Besides that, some critical shortcomings have already been addressed
> since
> > the last release:
> >
> > - After upgrading to Flink 1.10.1, failure recovery in Stateful
> > Functions now works properly with the new scheduler.
> > - Support for concurrent checkpoints
> >
> >
> > With these ongoing threads, while it's only been just short of 2 months
> > since the last release,
> > we (Igal Shilman and I) have been thinking about aiming to already start
> > the next feature release (2.1.0) soon.
> > This is relatively shorter than the release cycle of what the community
> is
> > used to in Flink (usually 3 months at least),
> > but we think with the StateFun project in its early phases, having
> smaller
> > and more frequent feature releases could potentially help drive user
> > adoption.
> >
> > So, what do you think about setting feature freeze for StateFun 2.1.0 by
> > next Wednesday (May 27th)?
> > Of course, whether or not to actually have another feature release
> already
> > is still an open discussion - if you prefer a richer feature release with
> > more features included besides the ones listed above, please do comment!
> >
> > Cheers,
> > Gordon
> >
>

Zhijiang(wangzhijiang999)

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

I also like this idea, considering stateful functions flexible enough to have a faster release cycle. +1 from my side.

Best,
Zhijiang

------------------------------------------------------------------
From:Seth Wiesman <[hidden email]>
Send Time:2020年5月20日(星期三) 21:45
To:dev <[hidden email]>
Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

+1 for a fast release cycle

Seth

On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]> wrote:

Yu Li

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

+1, it makes a lot of sense for stateful functions to evolve faster.

Best Regards,
Yu

On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]>
wrote:

> I also like this idea, considering stateful functions flexible enough to
> have a faster release cycle. +1 from my side.
>
> Best,
> Zhijiang
>
>
> ------------------------------------------------------------------
> From:Seth Wiesman <[hidden email]>
> Send Time:2020年5月20日(星期三) 21:45
> To:dev <[hidden email]>
> Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
>
> +1 for a fast release cycle
>
> Seth
>
> On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]>
> wrote:
>
> > I like the idea of releasing Statefun more frequently to have faster
> > feedback cycles!
> >
> > No objections for releasing 2.1.0 from my side.
> >
> > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <[hidden email]
> >
> > wrote:
> >
> > > Hi devs,
> > >
> > > Since Stateful Functions 2.0 was released early April,
> > > we've been getting some good feedback from various channels,
> > > including the Flink mailing lists, JIRA issues, as well as Stack
> Overflow
> > > questions.
> > >
> > > Some of the discussions have actually translated into new features
> > > currently being implemented into the project, such as:
> > >
> > > - State TTL for the state primitives in Stateful Functions (for both
> > > embedded/remote functions)
> > > - Transport for remote functions via UNIX domain sockets, which
> would
> > be
> > > useful when remote functions are co-located with Flink StateFun
> > workers
> > > (i.e. the "sidecar" deployment mode)
> > >
> > >
> > > Besides that, some critical shortcomings have already been addressed
> > since
> > > the last release:
> > >
> > > - After upgrading to Flink 1.10.1, failure recovery in Stateful
> > > Functions now works properly with the new scheduler.
> > > - Support for concurrent checkpoints
> > >
> > >
> > > With these ongoing threads, while it's only been just short of 2 months
> > > since the last release,
> > > we (Igal Shilman and I) have been thinking about aiming to already
> start
> > > the next feature release (2.1.0) soon.
> > > This is relatively shorter than the release cycle of what the community
> > is
> > > used to in Flink (usually 3 months at least),
> > > but we think with the StateFun project in its early phases, having
> > smaller
> > > and more frequent feature releases could potentially help drive user
> > > adoption.
> > >
> > > So, what do you think about setting feature freeze for StateFun 2.1.0
> by
> > > next Wednesday (May 27th)?
> > > Of course, whether or not to actually have another feature release
> > already
> > > is still an open discussion - if you prefer a richer feature release
> with
> > > more features included besides the ones listed above, please do
> comment!
> > >
> > > Cheers,
> > > Gordon
> > >
> >
>
>

Marta Paes Moreira

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

+1 for more frequent releases with a shorter (but feedback-informed)
feature set.

Thanks, Gordon (and Igal)!

Marta

On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:

> +1, it makes a lot of sense for stateful functions to evolve faster.
>
> Best Regards,
> Yu
>
>
> On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]
> .invalid>
> wrote:
>
> > I also like this idea, considering stateful functions flexible enough to
> > have a faster release cycle. +1 from my side.
> >
> > Best,
> > Zhijiang
> >
> >
> > ------------------------------------------------------------------
> > From:Seth Wiesman <[hidden email]>
> > Send Time:2020年5月20日(星期三) 21:45
> > To:dev <[hidden email]>
> > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> >
> > +1 for a fast release cycle
> >
> > Seth
> >
> > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]>
> > wrote:
> >
> > > I like the idea of releasing Statefun more frequently to have faster
> > > feedback cycles!
> > >
> > > No objections for releasing 2.1.0 from my side.
> > >
> > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> [hidden email]
> > >
> > > wrote:
> > >
> > > > Hi devs,
> > > >
> > > > Since Stateful Functions 2.0 was released early April,
> > > > we've been getting some good feedback from various channels,
> > > > including the Flink mailing lists, JIRA issues, as well as Stack
> > Overflow
> > > > questions.
> > > >
> > > > Some of the discussions have actually translated into new features
> > > > currently being implemented into the project, such as:
> > > >
> > > > - State TTL for the state primitives in Stateful Functions (for
> both
> > > > embedded/remote functions)
> > > > - Transport for remote functions via UNIX domain sockets, which
> > would
> > > be
> > > > useful when remote functions are co-located with Flink StateFun
> > > workers
> > > > (i.e. the "sidecar" deployment mode)
> > > >
> > > >
> > > > Besides that, some critical shortcomings have already been addressed
> > > since
> > > > the last release:
> > > >
> > > > - After upgrading to Flink 1.10.1, failure recovery in Stateful
> > > > Functions now works properly with the new scheduler.
> > > > - Support for concurrent checkpoints
> > > >
> > > >
> > > > With these ongoing threads, while it's only been just short of 2
> months
> > > > since the last release,
> > > > we (Igal Shilman and I) have been thinking about aiming to already
> > start
> > > > the next feature release (2.1.0) soon.
> > > > This is relatively shorter than the release cycle of what the
> community
> > > is
> > > > used to in Flink (usually 3 months at least),
> > > > but we think with the StateFun project in its early phases, having
> > > smaller
> > > > and more frequent feature releases could potentially help drive user
> > > > adoption.
> > > >
> > > > So, what do you think about setting feature freeze for StateFun 2.1.0
> > by
> > > > next Wednesday (May 27th)?
> > > > Of course, whether or not to actually have another feature release
> > > already
> > > > is still an open discussion - if you prefer a richer feature release
> > with
> > > > more features included besides the ones listed above, please do
> > comment!
> > > >
> > > > Cheers,
> > > > Gordon
> > > >
> > >
> >
> >
>

Congxian Qiu

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

+1 from my side to have smaller and more frequent feature releases for the
project in its early phases.

Best,
Congxian

Marta Paes Moreira <[hidden email]> 于2020年5月21日周四下午12:49写道：

> +1 for more frequent releases with a shorter (but feedback-informed)
> feature set.
>
> Thanks, Gordon (and Igal)!
>
> Marta
>
> On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:
>
> > +1, it makes a lot of sense for stateful functions to evolve faster.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]
> > .invalid>
> > wrote:
> >
> > > I also like this idea, considering stateful functions flexible enough
> to
> > > have a faster release cycle. +1 from my side.
> > >
> > > Best,
> > > Zhijiang
> > >
> > >
> > > ------------------------------------------------------------------
> > > From:Seth Wiesman <[hidden email]>
> > > Send Time:2020年5月20日(星期三) 21:45
> > > To:dev <[hidden email]>
> > > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> > >
> > > +1 for a fast release cycle
> > >
> > > Seth
> > >
> > > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]>
> > > wrote:
> > >
> > > > I like the idea of releasing Statefun more frequently to have faster
> > > > feedback cycles!
> > > >
> > > > No objections for releasing 2.1.0 from my side.
> > > >
> > > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> > [hidden email]
> > > >
> > > > wrote:
> > > >
> > > > > Hi devs,
> > > > >
> > > > > Since Stateful Functions 2.0 was released early April,
> > > > > we've been getting some good feedback from various channels,
> > > > > including the Flink mailing lists, JIRA issues, as well as Stack
> > > Overflow
> > > > > questions.
> > > > >
> > > > > Some of the discussions have actually translated into new features
> > > > > currently being implemented into the project, such as:
> > > > >
> > > > > - State TTL for the state primitives in Stateful Functions (for
> > both
> > > > > embedded/remote functions)
> > > > > - Transport for remote functions via UNIX domain sockets, which
> > > would
> > > > be
> > > > > useful when remote functions are co-located with Flink StateFun
> > > > workers
> > > > > (i.e. the "sidecar" deployment mode)
> > > > >
> > > > >
> > > > > Besides that, some critical shortcomings have already been
> addressed
> > > > since
> > > > > the last release:
> > > > >
> > > > > - After upgrading to Flink 1.10.1, failure recovery in Stateful
> > > > > Functions now works properly with the new scheduler.
> > > > > - Support for concurrent checkpoints
> > > > >
> > > > >
> > > > > With these ongoing threads, while it's only been just short of 2
> > months
> > > > > since the last release,
> > > > > we (Igal Shilman and I) have been thinking about aiming to already
> > > start
> > > > > the next feature release (2.1.0) soon.
> > > > > This is relatively shorter than the release cycle of what the
> > community
> > > > is
> > > > > used to in Flink (usually 3 months at least),
> > > > > but we think with the StateFun project in its early phases, having
> > > > smaller
> > > > > and more frequent feature releases could potentially help drive
> user
> > > > > adoption.
> > > > >
> > > > > So, what do you think about setting feature freeze for StateFun
> 2.1.0
> > > by
> > > > > next Wednesday (May 27th)?
> > > > > Of course, whether or not to actually have another feature release
> > > > already
> > > > > is still an open discussion - if you prefer a richer feature
> release
> > > with
> > > > > more features included besides the ones listed above, please do
> > > comment!
> > > > >
> > > > > Cheers,
> > > > > Gordon
> > > > >
> > > >
> > >
> > >
> >
>

curcur

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

faster iteration definitely helps early-stage projects.

+1

Best,
Yuan

On Thu, May 21, 2020 at 4:14 PM Congxian Qiu <[hidden email]> wrote:

> +1 from my side to have smaller and more frequent feature releases for the
> project in its early phases.
>
> Best,
> Congxian
>
>
> Marta Paes Moreira <[hidden email]> 于2020年5月21日周四下午12:49写道：
>
> > +1 for more frequent releases with a shorter (but feedback-informed)
> > feature set.
> >
> > Thanks, Gordon (and Igal)!
> >
> > Marta
> >
> > On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:
> >
> > > +1, it makes a lot of sense for stateful functions to evolve faster.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]
> > > .invalid>
> > > wrote:
> > >
> > > > I also like this idea, considering stateful functions flexible enough
> > to
> > > > have a faster release cycle. +1 from my side.
> > > >
> > > > Best,
> > > > Zhijiang
> > > >
> > > >
> > > > ------------------------------------------------------------------
> > > > From:Seth Wiesman <[hidden email]>
> > > > Send Time:2020年5月20日(星期三) 21:45
> > > > To:dev <[hidden email]>
> > > > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> > > >
> > > > +1 for a fast release cycle
> > > >
> > > > Seth
> > > >
> > > > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <[hidden email]>
> > > > wrote:
> > > >
> > > > > I like the idea of releasing Statefun more frequently to have
> faster
> > > > > feedback cycles!
> > > > >
> > > > > No objections for releasing 2.1.0 from my side.
> > > > >
> > > > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi devs,
> > > > > >
> > > > > > Since Stateful Functions 2.0 was released early April,
> > > > > > we've been getting some good feedback from various channels,
> > > > > > including the Flink mailing lists, JIRA issues, as well as Stack
> > > > Overflow
> > > > > > questions.
> > > > > >
> > > > > > Some of the discussions have actually translated into new
> features
> > > > > > currently being implemented into the project, such as:
> > > > > >
> > > > > > - State TTL for the state primitives in Stateful Functions
> (for
> > > both
> > > > > > embedded/remote functions)
> > > > > > - Transport for remote functions via UNIX domain sockets,
> which
> > > > would
> > > > > be
> > > > > > useful when remote functions are co-located with Flink
> StateFun
> > > > > workers
> > > > > > (i.e. the "sidecar" deployment mode)
> > > > > >
> > > > > >
> > > > > > Besides that, some critical shortcomings have already been
> > addressed
> > > > > since
> > > > > > the last release:
> > > > > >
> > > > > > - After upgrading to Flink 1.10.1, failure recovery in
> Stateful
> > > > > > Functions now works properly with the new scheduler.
> > > > > > - Support for concurrent checkpoints
> > > > > >
> > > > > >
> > > > > > With these ongoing threads, while it's only been just short of 2
> > > months
> > > > > > since the last release,
> > > > > > we (Igal Shilman and I) have been thinking about aiming to
> already
> > > > start
> > > > > > the next feature release (2.1.0) soon.
> > > > > > This is relatively shorter than the release cycle of what the
> > > community
> > > > > is
> > > > > > used to in Flink (usually 3 months at least),
> > > > > > but we think with the StateFun project in its early phases,
> having
> > > > > smaller
> > > > > > and more frequent feature releases could potentially help drive
> > user
> > > > > > adoption.
> > > > > >
> > > > > > So, what do you think about setting feature freeze for StateFun
> > 2.1.0
> > > > by
> > > > > > next Wednesday (May 27th)?
> > > > > > Of course, whether or not to actually have another feature
> release
> > > > > already
> > > > > > is still an open discussion - if you prefer a richer feature
> > release
> > > > with
> > > > > > more features included besides the ones listed above, please do
> > > > comment!
> > > > > >
> > > > > > Cheers,
> > > > > > Gordon
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

Tzu-Li (Gordon) Tai

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

Thanks for the positive feedback so far.

Lets then set the feature freeze date for Stateful Functions 2.1.0 to be
next Wednesday (May 27th).

We've made good progress over the past days, all mentioned features merged
besides the following:
- https://issues.apache.org/jira/browse/FLINK-17875 State TTL support for
remote functions

Will keep track of that and hopefully cut the feature branch as scheduled.

Cheers,
Gordon

On Thu, May 21, 2020 at 7:22 PM Yuan Mei <[hidden email]> wrote:

> faster iteration definitely helps early-stage projects.
>
> +1
>
> Best,
> Yuan
>
>
> On Thu, May 21, 2020 at 4:14 PM Congxian Qiu <[hidden email]>
> wrote:
>
> > +1 from my side to have smaller and more frequent feature releases for
> the
> > project in its early phases.
> >
> > Best,
> > Congxian
> >
> >
> > Marta Paes Moreira <[hidden email]> 于2020年5月21日周四下午12:49写道：
> >
> > > +1 for more frequent releases with a shorter (but feedback-informed)
> > > feature set.
> > >
> > > Thanks, Gordon (and Igal)!
> > >
> > > Marta
> > >
> > > On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:
> > >
> > > > +1, it makes a lot of sense for stateful functions to evolve faster.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]
> > > > .invalid>
> > > > wrote:
> > > >
> > > > > I also like this idea, considering stateful functions flexible
> enough
> > > to
> > > > > have a faster release cycle. +1 from my side.
> > > > >
> > > > > Best,
> > > > > Zhijiang
> > > > >
> > > > >
> > > > > ------------------------------------------------------------------
> > > > > From:Seth Wiesman <[hidden email]>
> > > > > Send Time:2020年5月20日(星期三) 21:45
> > > > > To:dev <[hidden email]>
> > > > > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> > > > >
> > > > > +1 for a fast release cycle
> > > > >
> > > > > Seth
> > > > >
> > > > > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <
> [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > I like the idea of releasing Statefun more frequently to have
> > faster
> > > > > > feedback cycles!
> > > > > >
> > > > > > No objections for releasing 2.1.0 from my side.
> > > > > >
> > > > > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> > > > [hidden email]
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi devs,
> > > > > > >
> > > > > > > Since Stateful Functions 2.0 was released early April,
> > > > > > > we've been getting some good feedback from various channels,
> > > > > > > including the Flink mailing lists, JIRA issues, as well as
> Stack
> > > > > Overflow
> > > > > > > questions.
> > > > > > >
> > > > > > > Some of the discussions have actually translated into new
> > features
> > > > > > > currently being implemented into the project, such as:
> > > > > > >
> > > > > > > - State TTL for the state primitives in Stateful Functions
> > (for
> > > > both
> > > > > > > embedded/remote functions)
> > > > > > > - Transport for remote functions via UNIX domain sockets,
> > which
> > > > > would
> > > > > > be
> > > > > > > useful when remote functions are co-located with Flink
> > StateFun
> > > > > > workers
> > > > > > > (i.e. the "sidecar" deployment mode)
> > > > > > >
> > > > > > >
> > > > > > > Besides that, some critical shortcomings have already been
> > > addressed
> > > > > > since
> > > > > > > the last release:
> > > > > > >
> > > > > > > - After upgrading to Flink 1.10.1, failure recovery in
> > Stateful
> > > > > > > Functions now works properly with the new scheduler.
> > > > > > > - Support for concurrent checkpoints
> > > > > > >
> > > > > > >
> > > > > > > With these ongoing threads, while it's only been just short of
> 2
> > > > months
> > > > > > > since the last release,
> > > > > > > we (Igal Shilman and I) have been thinking about aiming to
> > already
> > > > > start
> > > > > > > the next feature release (2.1.0) soon.
> > > > > > > This is relatively shorter than the release cycle of what the
> > > > community
> > > > > > is
> > > > > > > used to in Flink (usually 3 months at least),
> > > > > > > but we think with the StateFun project in its early phases,
> > having
> > > > > > smaller
> > > > > > > and more frequent feature releases could potentially help drive
> > > user
> > > > > > > adoption.
> > > > > > >
> > > > > > > So, what do you think about setting feature freeze for StateFun
> > > 2.1.0
> > > > > by
> > > > > > > next Wednesday (May 27th)?
> > > > > > > Of course, whether or not to actually have another feature
> > release
> > > > > > already
> > > > > > > is still an open discussion - if you prefer a richer feature
> > > release
> > > > > with
> > > > > > > more features included besides the ones listed above, please do
> > > > > comment!
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Gordon
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Stephan Ewen

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

Nice work, thanks for pushing this, Gordon!

+1 also from my side for a quick release.

I think it already warrants a release to have the 1.10.1 upgrade and the
fix to not fail on savepoints that are triggered concurrently to a
checkpoint.
Even nicer that there are two cool new features included.

On Fri, May 22, 2020 at 7:13 AM Tzu-Li (Gordon) Tai <[hidden email]>
wrote:

> Thanks for the positive feedback so far.
>
> Lets then set the feature freeze date for Stateful Functions 2.1.0 to be
> next Wednesday (May 27th).
>
> We've made good progress over the past days, all mentioned features merged
> besides the following:
> - https://issues.apache.org/jira/browse/FLINK-17875 State TTL support for
> remote functions
>
> Will keep track of that and hopefully cut the feature branch as scheduled.
>
> Cheers,
> Gordon
>
> On Thu, May 21, 2020 at 7:22 PM Yuan Mei <[hidden email]> wrote:
>
> > faster iteration definitely helps early-stage projects.
> >
> > +1
> >
> > Best,
> > Yuan
> >
> >
> > On Thu, May 21, 2020 at 4:14 PM Congxian Qiu <[hidden email]>
> > wrote:
> >
> > > +1 from my side to have smaller and more frequent feature releases for
> > the
> > > project in its early phases.
> > >
> > > Best,
> > > Congxian
> > >
> > >
> > > Marta Paes Moreira <[hidden email]> 于2020年5月21日周四下午12:49写道：
> > >
> > > > +1 for more frequent releases with a shorter (but feedback-informed)
> > > > feature set.
> > > >
> > > > Thanks, Gordon (and Igal)!
> > > >
> > > > Marta
> > > >
> > > > On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:
> > > >
> > > > > +1, it makes a lot of sense for stateful functions to evolve
> faster.
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > > >
> > > > > On Wed, 20 May 2020 at 23:36, Zhijiang <[hidden email]
> > > > > .invalid>
> > > > > wrote:
> > > > >
> > > > > > I also like this idea, considering stateful functions flexible
> > enough
> > > > to
> > > > > > have a faster release cycle. +1 from my side.
> > > > > >
> > > > > > Best,
> > > > > > Zhijiang
> > > > > >
> > > > > >
> > > > > >
> ------------------------------------------------------------------
> > > > > > From:Seth Wiesman <[hidden email]>
> > > > > > Send Time:2020年5月20日(星期三) 21:45
> > > > > > To:dev <[hidden email]>
> > > > > > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> > > > > >
> > > > > > +1 for a fast release cycle
> > > > > >
> > > > > > Seth
> > > > > >
> > > > > > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <
> > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > I like the idea of releasing Statefun more frequently to have
> > > faster
> > > > > > > feedback cycles!
> > > > > > >
> > > > > > > No objections for releasing 2.1.0 from my side.
> > > > > > >
> > > > > > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> > > > > [hidden email]
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi devs,
> > > > > > > >
> > > > > > > > Since Stateful Functions 2.0 was released early April,
> > > > > > > > we've been getting some good feedback from various channels,
> > > > > > > > including the Flink mailing lists, JIRA issues, as well as
> > Stack
> > > > > > Overflow
> > > > > > > > questions.
> > > > > > > >
> > > > > > > > Some of the discussions have actually translated into new
> > > features
> > > > > > > > currently being implemented into the project, such as:
> > > > > > > >
> > > > > > > > - State TTL for the state primitives in Stateful Functions
> > > (for
> > > > > both
> > > > > > > > embedded/remote functions)
> > > > > > > > - Transport for remote functions via UNIX domain sockets,
> > > which
> > > > > > would
> > > > > > > be
> > > > > > > > useful when remote functions are co-located with Flink
> > > StateFun
> > > > > > > workers
> > > > > > > > (i.e. the "sidecar" deployment mode)
> > > > > > > >
> > > > > > > >
> > > > > > > > Besides that, some critical shortcomings have already been
> > > > addressed
> > > > > > > since
> > > > > > > > the last release:
> > > > > > > >
> > > > > > > > - After upgrading to Flink 1.10.1, failure recovery in
> > > Stateful
> > > > > > > > Functions now works properly with the new scheduler.
> > > > > > > > - Support for concurrent checkpoints
> > > > > > > >
> > > > > > > >
> > > > > > > > With these ongoing threads, while it's only been just short
> of
> > 2
> > > > > months
> > > > > > > > since the last release,
> > > > > > > > we (Igal Shilman and I) have been thinking about aiming to
> > > already
> > > > > > start
> > > > > > > > the next feature release (2.1.0) soon.
> > > > > > > > This is relatively shorter than the release cycle of what the
> > > > > community
> > > > > > > is
> > > > > > > > used to in Flink (usually 3 months at least),
> > > > > > > > but we think with the StateFun project in its early phases,
> > > having
> > > > > > > smaller
> > > > > > > > and more frequent feature releases could potentially help
> drive
> > > > user
> > > > > > > > adoption.
> > > > > > > >
> > > > > > > > So, what do you think about setting feature freeze for
> StateFun
> > > > 2.1.0
> > > > > > by
> > > > > > > > next Wednesday (May 27th)?
> > > > > > > > Of course, whether or not to actually have another feature
> > > release
> > > > > > > already
> > > > > > > > is still an open discussion - if you prefer a richer feature
> > > > release
> > > > > > with
> > > > > > > > more features included besides the ones listed above, please
> do
> > > > > > comment!
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Gordon
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Tzu-Li (Gordon) Tai

Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?

Hi all,

Quick update on progress:
There is still ongoing work on adding state TTL for remote functions
(FLINK-17875), but the PR is expected to be mergeable over the next day or
two.

Therefore the feature branch cut will be slightly delayed. I'll announce
the cut in a separate email once it happens.

Gordon

On Tue, May 26, 2020, 1:32 AM Stephan Ewen <[hidden email]> wrote:

> Nice work, thanks for pushing this, Gordon!
>
> +1 also from my side for a quick release.
>
> I think it already warrants a release to have the 1.10.1 upgrade and the
> fix to not fail on savepoints that are triggered concurrently to a
> checkpoint.
> Even nicer that there are two cool new features included.
>
> On Fri, May 22, 2020 at 7:13 AM Tzu-Li (Gordon) Tai <[hidden email]>
> wrote:
>
> > Thanks for the positive feedback so far.
> >
> > Lets then set the feature freeze date for Stateful Functions 2.1.0 to be
> > next Wednesday (May 27th).
> >
> > We've made good progress over the past days, all mentioned features
> merged
> > besides the following:
> > - https://issues.apache.org/jira/browse/FLINK-17875 State TTL support
> for
> > remote functions
> >
> > Will keep track of that and hopefully cut the feature branch as
> scheduled.
> >
> > Cheers,
> > Gordon
> >
> > On Thu, May 21, 2020 at 7:22 PM Yuan Mei <[hidden email]> wrote:
> >
> > > faster iteration definitely helps early-stage projects.
> > >
> > > +1
> > >
> > > Best,
> > > Yuan
> > >
> > >
> > > On Thu, May 21, 2020 at 4:14 PM Congxian Qiu <[hidden email]>
> > > wrote:
> > >
> > > > +1 from my side to have smaller and more frequent feature releases
> for
> > > the
> > > > project in its early phases.
> > > >
> > > > Best,
> > > > Congxian
> > > >
> > > >
> > > > Marta Paes Moreira <[hidden email]> 于2020年5月21日周四下午12:49写道：
> > > >
> > > > > +1 for more frequent releases with a shorter (but
> feedback-informed)
> > > > > feature set.
> > > > >
> > > > > Thanks, Gordon (and Igal)!
> > > > >
> > > > > Marta
> > > > >
> > > > > On Thu, 21 May 2020 at 03:44, Yu Li <[hidden email]> wrote:
> > > > >
> > > > > > +1, it makes a lot of sense for stateful functions to evolve
> > faster.
> > > > > >
> > > > > > Best Regards,
> > > > > > Yu
> > > > > >
> > > > > >
> > > > > > On Wed, 20 May 2020 at 23:36, Zhijiang <
> [hidden email]
> > > > > > .invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > I also like this idea, considering stateful functions flexible
> > > enough
> > > > > to
> > > > > > > have a faster release cycle. +1 from my side.
> > > > > > >
> > > > > > > Best,
> > > > > > > Zhijiang
> > > > > > >
> > > > > > >
> > > > > > >
> > ------------------------------------------------------------------
> > > > > > > From:Seth Wiesman <[hidden email]>
> > > > > > > Send Time:2020年5月20日(星期三) 21:45
> > > > > > > To:dev <[hidden email]>
> > > > > > > Subject:Re: [DISCUSS] Releasing Stateful Functions 2.1.0 soon?
> > > > > > >
> > > > > > > +1 for a fast release cycle
> > > > > > >
> > > > > > > Seth
> > > > > > >
> > > > > > > On Wed, May 20, 2020 at 8:43 AM Robert Metzger <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I like the idea of releasing Statefun more frequently to have
> > > > faster
> > > > > > > > feedback cycles!
> > > > > > > >
> > > > > > > > No objections for releasing 2.1.0 from my side.
> > > > > > > >
> > > > > > > > On Wed, May 20, 2020 at 2:22 PM Tzu-Li (Gordon) Tai <
> > > > > > [hidden email]
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi devs,
> > > > > > > > >
> > > > > > > > > Since Stateful Functions 2.0 was released early April,
> > > > > > > > > we've been getting some good feedback from various
> channels,
> > > > > > > > > including the Flink mailing lists, JIRA issues, as well as
> > > Stack
> > > > > > > Overflow
> > > > > > > > > questions.
> > > > > > > > >
> > > > > > > > > Some of the discussions have actually translated into new
> > > > features
> > > > > > > > > currently being implemented into the project, such as:
> > > > > > > > >
> > > > > > > > > - State TTL for the state primitives in Stateful
> Functions
> > > > (for
> > > > > > both
> > > > > > > > > embedded/remote functions)
> > > > > > > > > - Transport for remote functions via UNIX domain
> sockets,
> > > > which
> > > > > > > would
> > > > > > > > be
> > > > > > > > > useful when remote functions are co-located with Flink
> > > > StateFun
> > > > > > > > workers
> > > > > > > > > (i.e. the "sidecar" deployment mode)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Besides that, some critical shortcomings have already been
> > > > > addressed
> > > > > > > > since
> > > > > > > > > the last release:
> > > > > > > > >
> > > > > > > > > - After upgrading to Flink 1.10.1, failure recovery in
> > > > Stateful
> > > > > > > > > Functions now works properly with the new scheduler.
> > > > > > > > > - Support for concurrent checkpoints
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > With these ongoing threads, while it's only been just short
> > of
> > > 2
> > > > > > months
> > > > > > > > > since the last release,
> > > > > > > > > we (Igal Shilman and I) have been thinking about aiming to
> > > > already
> > > > > > > start
> > > > > > > > > the next feature release (2.1.0) soon.
> > > > > > > > > This is relatively shorter than the release cycle of what
> the
> > > > > > community
> > > > > > > > is
> > > > > > > > > used to in Flink (usually 3 months at least),
> > > > > > > > > but we think with the StateFun project in its early phases,
> > > > having
> > > > > > > > smaller
> > > > > > > > > and more frequent feature releases could potentially help
> > drive
> > > > > user
> > > > > > > > > adoption.
> > > > > > > > >
> > > > > > > > > So, what do you think about setting feature freeze for
> > StateFun
> > > > > 2.1.0
> > > > > > > by
> > > > > > > > > next Wednesday (May 27th)?
> > > > > > > > > Of course, whether or not to actually have another feature
> > > > release
> > > > > > > > already
> > > > > > > > > is still an open discussion - if you prefer a richer
> feature
> > > > > release
> > > > > > > with
> > > > > > > > > more features included besides the ones listed above,
> please
> > do
> > > > > > > comment!
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Gordon
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>