[DISCUSS] Releasing Flink 1.4

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Releasing Flink 1.4

Till Rohrmann
Hi all,

I want to revive the discussion about releasing Flink 1.4 [1] and the set
of features to include.

The gist of the previous discussion was that we postponed the feature
freeze for 1.4 in order to include some more features which were being
developed. By now, we have completed a good set of features such as exactly
once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
many bug fixes. I believe that these features will make good release and
users are already waiting for them.

Some of the other features which we wanted to include, mainly Flip-6, to
some extent the network stack enhancements and the state decoupling still
need some more time. Since these features are major changes to Flink's
runtime, it would be in my opinion a good idea to cut a stable release with
the above-mentioned feature set now and give the engine features a bit more
time to ripen and be properly tested.

Therefore, I would actually be in favour of aiming for a quick release
meaning that we now concentrate mainly on fixing bugs and critical issues.
Moreover, I'm optimistic that the delayed features will be completed soon
such that we can deliver them with the next release. What do you think?

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html

Cheers,
Till
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Renjie Liu
Cool!!!

On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]> wrote:

> Hi all,
>
> I want to revive the discussion about releasing Flink 1.4 [1] and the set
> of features to include.
>
> The gist of the previous discussion was that we postponed the feature
> freeze for 1.4 in order to include some more features which were being
> developed. By now, we have completed a good set of features such as exactly
> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> many bug fixes. I believe that these features will make good release and
> users are already waiting for them.
>
> Some of the other features which we wanted to include, mainly Flip-6, to
> some extent the network stack enhancements and the state decoupling still
> need some more time. Since these features are major changes to Flink's
> runtime, it would be in my opinion a good idea to cut a stable release with
> the above-mentioned feature set now and give the engine features a bit more
> time to ripen and be properly tested.
>
> Therefore, I would actually be in favour of aiming for a quick release
> meaning that we now concentrate mainly on fixing bugs and critical issues.
> Moreover, I'm optimistic that the delayed features will be completed soon
> such that we can deliver them with the next release. What do you think?
>
> [1]
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>
> Cheers,
> Till
>
--
Liu, Renjie
Software Engineer, MVAD
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Aljoscha Krettek-2
+1 Excellent

I'd like to volunteer as release manager. I already set up a Kanban board to monitor the open blocking (and non-blocking) issues for 1.4, though this is independent of me volunteering as release manager. We should all go over these issues and see which ones should actually be blocking and which ones are not yet on that list.

> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
>
> Cool!!!
>
> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]> wrote:
>
>> Hi all,
>>
>> I want to revive the discussion about releasing Flink 1.4 [1] and the set
>> of features to include.
>>
>> The gist of the previous discussion was that we postponed the feature
>> freeze for 1.4 in order to include some more features which were being
>> developed. By now, we have completed a good set of features such as exactly
>> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
>> many bug fixes. I believe that these features will make good release and
>> users are already waiting for them.
>>
>> Some of the other features which we wanted to include, mainly Flip-6, to
>> some extent the network stack enhancements and the state decoupling still
>> need some more time. Since these features are major changes to Flink's
>> runtime, it would be in my opinion a good idea to cut a stable release with
>> the above-mentioned feature set now and give the engine features a bit more
>> time to ripen and be properly tested.
>>
>> Therefore, I would actually be in favour of aiming for a quick release
>> meaning that we now concentrate mainly on fixing bugs and critical issues.
>> Moreover, I'm optimistic that the delayed features will be completed soon
>> such that we can deliver them with the next release. What do you think?
>>
>> [1]
>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>
>> Cheers,
>> Till
>>
> --
> Liu, Renjie
> Software Engineer, MVAD

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Stephan Ewen
I am in favor of doing this, if we can set it up in the following way.

  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
stable cut before the fundamental changes go in.

  - We merge the very big changes (FLIP-6, Network stack, localized state
restore, etc). directly (or very soon) after.
  - We try to stabilize these changes and release 1.5 asap after that.
Ideally Around end of year or so.

The reason I am bringing this up is that I know various users waiting very
much for FLIP-6 and Network Stack enhancements. Given that these issues
were flagged for release 1.4, the users were planning to have them rather
soon.

Stephan


On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
wrote:

> +1 Excellent
>
> I'd like to volunteer as release manager. I already set up a Kanban board
> to monitor the open blocking (and non-blocking) issues for 1.4, though this
> is independent of me volunteering as release manager. We should all go over
> these issues and see which ones should actually be blocking and which ones
> are not yet on that list.
>
> > On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
> >
> > Cool!!!
> >
> > On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
> wrote:
> >
> >> Hi all,
> >>
> >> I want to revive the discussion about releasing Flink 1.4 [1] and the
> set
> >> of features to include.
> >>
> >> The gist of the previous discussion was that we postponed the feature
> >> freeze for 1.4 in order to include some more features which were being
> >> developed. By now, we have completed a good set of features such as
> exactly
> >> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> >> many bug fixes. I believe that these features will make good release and
> >> users are already waiting for them.
> >>
> >> Some of the other features which we wanted to include, mainly Flip-6, to
> >> some extent the network stack enhancements and the state decoupling
> still
> >> need some more time. Since these features are major changes to Flink's
> >> runtime, it would be in my opinion a good idea to cut a stable release
> with
> >> the above-mentioned feature set now and give the engine features a bit
> more
> >> time to ripen and be properly tested.
> >>
> >> Therefore, I would actually be in favour of aiming for a quick release
> >> meaning that we now concentrate mainly on fixing bugs and critical
> issues.
> >> Moreover, I'm optimistic that the delayed features will be completed
> soon
> >> such that we can deliver them with the next release. What do you think?
> >>
> >> [1]
> >>
> >> http://apache-flink-mailing-list-archive.1008284.n3.
> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>
> >> Cheers,
> >> Till
> >>
> > --
> > Liu, Renjie
> > Software Engineer, MVAD
>
>
Reply | Threaded
Open this post in threaded view
|

回复:[DISCUSS] Releasing Flink 1.4

wangzhijiang
In reply to this post by Aljoscha Krettek-2
totally agree with the way.------------------------------------------------------------------发件人:Stephan Ewen <[hidden email]>发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
I am in favor of doing this, if we can set it up in the following way.

  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
stable cut before the fundamental changes go in.

  - We merge the very big changes (FLIP-6, Network stack, localized state
restore, etc). directly (or very soon) after.
  - We try to stabilize these changes and release 1.5 asap after that.
Ideally Around end of year or so.

The reason I am bringing this up is that I know various users waiting very
much for FLIP-6 and Network Stack enhancements. Given that these issues
were flagged for release 1.4, the users were planning to have them rather
soon.

Stephan


On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
wrote:

> +1 Excellent
>
> I'd like to volunteer as release manager. I already set up a Kanban board
> to monitor the open blocking (and non-blocking) issues for 1.4, though this
> is independent of me volunteering as release manager. We should all go over
> these issues and see which ones should actually be blocking and which ones
> are not yet on that list.
>
> > On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
> >
> > Cool!!!
> >
> > On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
> wrote:
> >
> >> Hi all,
> >>
> >> I want to revive the discussion about releasing Flink 1.4 [1] and the
> set
> >> of features to include.
> >>
> >> The gist of the previous discussion was that we postponed the feature
> >> freeze for 1.4 in order to include some more features which were being
> >> developed. By now, we have completed a good set of features such as
> exactly
> >> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> >> many bug fixes. I believe that these features will make good release and
> >> users are already waiting for them.
> >>
> >> Some of the other features which we wanted to include, mainly Flip-6, to
> >> some extent the network stack enhancements and the state decoupling
> still
> >> need some more time. Since these features are major changes to Flink's
> >> runtime, it would be in my opinion a good idea to cut a stable release
> with
> >> the above-mentioned feature set now and give the engine features a bit
> more
> >> time to ripen and be properly tested.
> >>
> >> Therefore, I would actually be in favour of aiming for a quick release
> >> meaning that we now concentrate mainly on fixing bugs and critical
> issues.
> >> Moreover, I'm optimistic that the delayed features will be completed
> soon
> >> such that we can deliver them with the next release. What do you think?
> >>
> >> [1]
> >>
> >> http://apache-flink-mailing-list-archive.1008284.n3.
> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>
> >> Cheers,
> >> Till
> >>
> > --
> > Liu, Renjie
> > Software Engineer, MVAD
>
>

Reply | Threaded
Open this post in threaded view
|

Re: 回复:[DISCUSS] Releasing Flink 1.4

Bowen Li
+1.

To give users a heads up, might be good to start to change the ‘target release’ of some Jira tickets from 1.4 to 1.5.

Sent from my iPhone

> On Oct 13, 2017, at 11:02 PM, Zhijiang(wangzhijiang999) <[hidden email]> wrote:
>
> totally agree with the way.------------------------------------------------------------------发件人:Stephan Ewen <[hidden email]>发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
> I am in favor of doing this, if we can set it up in the following way.
>
>  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> stable cut before the fundamental changes go in.
>
>  - We merge the very big changes (FLIP-6, Network stack, localized state
> restore, etc). directly (or very soon) after.
>  - We try to stabilize these changes and release 1.5 asap after that.
> Ideally Around end of year or so.
>
> The reason I am bringing this up is that I know various users waiting very
> much for FLIP-6 and Network Stack enhancements. Given that these issues
> were flagged for release 1.4, the users were planning to have them rather
> soon.
>
> Stephan
>
>
> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> +1 Excellent
>>
>> I'd like to volunteer as release manager. I already set up a Kanban board
>> to monitor the open blocking (and non-blocking) issues for 1.4, though this
>> is independent of me volunteering as release manager. We should all go over
>> these issues and see which ones should actually be blocking and which ones
>> are not yet on that list.
>>
>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
>>>
>>> Cool!!!
>>>
>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I want to revive the discussion about releasing Flink 1.4 [1] and the
>> set
>>>> of features to include.
>>>>
>>>> The gist of the previous discussion was that we postponed the feature
>>>> freeze for 1.4 in order to include some more features which were being
>>>> developed. By now, we have completed a good set of features such as
>> exactly
>>>> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
>>>> many bug fixes. I believe that these features will make good release and
>>>> users are already waiting for them.
>>>>
>>>> Some of the other features which we wanted to include, mainly Flip-6, to
>>>> some extent the network stack enhancements and the state decoupling
>> still
>>>> need some more time. Since these features are major changes to Flink's
>>>> runtime, it would be in my opinion a good idea to cut a stable release
>> with
>>>> the above-mentioned feature set now and give the engine features a bit
>> more
>>>> time to ripen and be properly tested.
>>>>
>>>> Therefore, I would actually be in favour of aiming for a quick release
>>>> meaning that we now concentrate mainly on fixing bugs and critical
>> issues.
>>>> Moreover, I'm optimistic that the delayed features will be completed
>> soon
>>>> such that we can deliver them with the next release. What do you think?
>>>>
>>>> [1]
>>>>
>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>>>
>>>> Cheers,
>>>> Till
>>> --
>>> Liu, Renjie
>>> Software Engineer, MVAD
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Alexx
In reply to this post by wangzhijiang
great

On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
[hidden email]> wrote:

> totally agree with the way.--------------------------
> ----------------------------------------发件人:Stephan Ewen <[hidden email]
> >发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <[hidden email]>主
> 题:Re: [DISCUSS] Releasing Flink 1.4
> I am in favor of doing this, if we can set it up in the following way.
>
>   - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> stable cut before the fundamental changes go in.
>
>   - We merge the very big changes (FLIP-6, Network stack, localized state
> restore, etc). directly (or very soon) after.
>   - We try to stabilize these changes and release 1.5 asap after that.
> Ideally Around end of year or so.
>
> The reason I am bringing this up is that I know various users waiting very
> much for FLIP-6 and Network Stack enhancements. Given that these issues
> were flagged for release 1.4, the users were planning to have them rather
> soon.
>
> Stephan
>
>
> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > +1 Excellent
> >
> > I'd like to volunteer as release manager. I already set
> up a Kanban board
> > to monitor the open blocking (and non-blocking) issues
> for 1.4, though this
> > is independent of me volunteering as release manager. We
> should all go over
> > these issues and see which ones should actually be blockin
> g and which ones
> > are not yet on that list.
> >
> > > On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
> > >
> > > Cool!!!
> > >
> > > On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
> > wrote:
> > >
> > >> Hi all,
> > >>
> > >> I want to revive the discussion about releasing Flink 1.4 [1] and the
> > set
> > >> of features to include.
> > >>
> > >> The gist of the previous discussion was that we postponed the feature
> > >> freeze for 1.4 in order to include some more features w
> hich were being
> > >> developed. By now, we have completed a good set of features such as
> > exactly
> > >> once Kafka producer, reduced dependency footprint, Hado
> op-free Flink and
> > >> many bug fixes. I believe that these features will
> make good release and
> > >> users are already waiting for them.
> > >>
> > >> Some of the other features which we wanted to include,
> mainly Flip-6, to
> > >> some extent the network stack enhancements and the state decoupling
> > still
> > >> need some more time. Since these features are major
> changes to Flink's
> > >> runtime, it would be in my opinion a good idea to cut a
>  stable release
> > with
> > >> the above-mentioned feature set now and give the engine
>  features a bit
> > more
> > >> time to ripen and be properly tested.
> > >>
> > >> Therefore, I would actually be in favour of aiming for
> a quick release
> > >> meaning that we now concentrate mainly on fixing bugs and critical
> > issues.
> > >> Moreover, I'm optimistic that the delayed features will be completed
> > soon
> > >> such that we can deliver them with the next release. Wh
> at do you think?
> > >>
> > >> [1]
> > >>
> > >> http://apache-flink-mailing-list-archive.1008284.n3.
> > nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > > --
> > > Liu, Renjie
> > > Software Engineer, MVAD
> >
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Aljoscha Krettek-2
@Bowen I started marking essential stuff as blocking (with fixVersion 1.4.0). You're right, that we should start moving things to 1.5.0 that are not blocking and that we don't think will make it into 1.4.0. I think we can only release 1.4.0 if there are 0 (zero) unresolved issues with fixVersion 1.4.0.

> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]> wrote:
>
> great
>
> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> [hidden email]> wrote:
>
>> totally agree with the way.--------------------------
>> ----------------------------------------发件人:Stephan Ewen <[hidden email]
>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <[hidden email]>主
>> 题:Re: [DISCUSS] Releasing Flink 1.4
>> I am in favor of doing this, if we can set it up in the following way.
>>
>>  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
>> stable cut before the fundamental changes go in.
>>
>>  - We merge the very big changes (FLIP-6, Network stack, localized state
>> restore, etc). directly (or very soon) after.
>>  - We try to stabilize these changes and release 1.5 asap after that.
>> Ideally Around end of year or so.
>>
>> The reason I am bringing this up is that I know various users waiting very
>> much for FLIP-6 and Network Stack enhancements. Given that these issues
>> were flagged for release 1.4, the users were planning to have them rather
>> soon.
>>
>> Stephan
>>
>>
>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>>> +1 Excellent
>>>
>>> I'd like to volunteer as release manager. I already set
>> up a Kanban board
>>> to monitor the open blocking (and non-blocking) issues
>> for 1.4, though this
>>> is independent of me volunteering as release manager. We
>> should all go over
>>> these issues and see which ones should actually be blockin
>> g and which ones
>>> are not yet on that list.
>>>
>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]> wrote:
>>>>
>>>> Cool!!!
>>>>
>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I want to revive the discussion about releasing Flink 1.4 [1] and the
>>> set
>>>>> of features to include.
>>>>>
>>>>> The gist of the previous discussion was that we postponed the feature
>>>>> freeze for 1.4 in order to include some more features w
>> hich were being
>>>>> developed. By now, we have completed a good set of features such as
>>> exactly
>>>>> once Kafka producer, reduced dependency footprint, Hado
>> op-free Flink and
>>>>> many bug fixes. I believe that these features will
>> make good release and
>>>>> users are already waiting for them.
>>>>>
>>>>> Some of the other features which we wanted to include,
>> mainly Flip-6, to
>>>>> some extent the network stack enhancements and the state decoupling
>>> still
>>>>> need some more time. Since these features are major
>> changes to Flink's
>>>>> runtime, it would be in my opinion a good idea to cut a
>> stable release
>>> with
>>>>> the above-mentioned feature set now and give the engine
>> features a bit
>>> more
>>>>> time to ripen and be properly tested.
>>>>>
>>>>> Therefore, I would actually be in favour of aiming for
>> a quick release
>>>>> meaning that we now concentrate mainly on fixing bugs and critical
>>> issues.
>>>>> Moreover, I'm optimistic that the delayed features will be completed
>>> soon
>>>>> such that we can deliver them with the next release. Wh
>> at do you think?
>>>>>
>>>>> [1]
>>>>>
>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>>>>
>>>>> Cheers,
>>>>> Till
>>>>>
>>>> --
>>>> Liu, Renjie
>>>> Software Engineer, MVAD
>>>
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Fabian Hueske-2
OK, sounds good to me.

We have a couple of bugs to fix for the Table API / SQL but have PRs for
most of them.

There's only one major issue that I'd like to include in 1.4.0 which is a
refactoring of the TableSource interface.
This effort has already started and is currently waiting for reviews /
comments.
I'm quite confident that we can get it in within the next two weeks.

Cheers, Fabian

2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:

> @Bowen I started marking essential stuff as blocking (with fixVersion
> 1.4.0). You're right, that we should start moving things to 1.5.0 that are
> not blocking and that we don't think will make it into 1.4.0. I think we
> can only release 1.4.0 if there are 0 (zero) unresolved issues with
> fixVersion 1.4.0.
>
> > On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
> wrote:
> >
> > great
> >
> > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> > [hidden email]> wrote:
> >
> >> totally agree with the way.--------------------------
> >> ----------------------------------------发件人:Stephan Ewen <
> [hidden email]
> >>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
> [hidden email]>主
> >> 题:Re: [DISCUSS] Releasing Flink 1.4
> >> I am in favor of doing this, if we can set it up in the following way.
> >>
> >>  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> >> stable cut before the fundamental changes go in.
> >>
> >>  - We merge the very big changes (FLIP-6, Network stack, localized state
> >> restore, etc). directly (or very soon) after.
> >>  - We try to stabilize these changes and release 1.5 asap after that.
> >> Ideally Around end of year or so.
> >>
> >> The reason I am bringing this up is that I know various users waiting
> very
> >> much for FLIP-6 and Network Stack enhancements. Given that these issues
> >> were flagged for release 1.4, the users were planning to have them
> rather
> >> soon.
> >>
> >> Stephan
> >>
> >>
> >> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <[hidden email]>
> >> wrote:
> >>
> >>> +1 Excellent
> >>>
> >>> I'd like to volunteer as release manager. I already set
> >> up a Kanban board
> >>> to monitor the open blocking (and non-blocking) issues
> >> for 1.4, though this
> >>> is independent of me volunteering as release manager. We
> >> should all go over
> >>> these issues and see which ones should actually be blockin
> >> g and which ones
> >>> are not yet on that list.
> >>>
> >>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
> wrote:
> >>>>
> >>>> Cool!!!
> >>>>
> >>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]>
> >>> wrote:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> I want to revive the discussion about releasing Flink 1.4 [1] and the
> >>> set
> >>>>> of features to include.
> >>>>>
> >>>>> The gist of the previous discussion was that we postponed the feature
> >>>>> freeze for 1.4 in order to include some more features w
> >> hich were being
> >>>>> developed. By now, we have completed a good set of features such as
> >>> exactly
> >>>>> once Kafka producer, reduced dependency footprint, Hado
> >> op-free Flink and
> >>>>> many bug fixes. I believe that these features will
> >> make good release and
> >>>>> users are already waiting for them.
> >>>>>
> >>>>> Some of the other features which we wanted to include,
> >> mainly Flip-6, to
> >>>>> some extent the network stack enhancements and the state decoupling
> >>> still
> >>>>> need some more time. Since these features are major
> >> changes to Flink's
> >>>>> runtime, it would be in my opinion a good idea to cut a
> >> stable release
> >>> with
> >>>>> the above-mentioned feature set now and give the engine
> >> features a bit
> >>> more
> >>>>> time to ripen and be properly tested.
> >>>>>
> >>>>> Therefore, I would actually be in favour of aiming for
> >> a quick release
> >>>>> meaning that we now concentrate mainly on fixing bugs and critical
> >>> issues.
> >>>>> Moreover, I'm optimistic that the delayed features will be completed
> >>> soon
> >>>>> such that we can deliver them with the next release. Wh
> >> at do you think?
> >>>>>
> >>>>> [1]
> >>>>>
> >>>>> http://apache-flink-mailing-list-archive.1008284.n3.
> >>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>>>>
> >>>>> Cheers,
> >>>>> Till
> >>>>>
> >>>> --
> >>>> Liu, Renjie
> >>>> Software Engineer, MVAD
> >>>
> >>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Eron Wright-2
+1 from our side on this plan.

On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]> wrote:

> OK, sounds good to me.
>
> We have a couple of bugs to fix for the Table API / SQL but have PRs for
> most of them.
>
> There's only one major issue that I'd like to include in 1.4.0 which is a
> refactoring of the TableSource interface.
> This effort has already started and is currently waiting for reviews /
> comments.
> I'm quite confident that we can get it in within the next two weeks.
>
> Cheers, Fabian
>
> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>
> > @Bowen I started marking essential stuff as blocking (with fixVersion
> > 1.4.0). You're right, that we should start moving things to 1.5.0 that
> are
> > not blocking and that we don't think will make it into 1.4.0. I think we
> > can only release 1.4.0 if there are 0 (zero) unresolved issues with
> > fixVersion 1.4.0.
> >
> > > On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
> > wrote:
> > >
> > > great
> > >
> > > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> > > [hidden email]> wrote:
> > >
> > >> totally agree with the way.--------------------------
> > >> ----------------------------------------发件人:Stephan Ewen <
> > [hidden email]
> > >>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
> > [hidden email]>主
> > >> 题:Re: [DISCUSS] Releasing Flink 1.4
> > >> I am in favor of doing this, if we can set it up in the following way.
> > >>
> > >>  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> > >> stable cut before the fundamental changes go in.
> > >>
> > >>  - We merge the very big changes (FLIP-6, Network stack, localized
> state
> > >> restore, etc). directly (or very soon) after.
> > >>  - We try to stabilize these changes and release 1.5 asap after that.
> > >> Ideally Around end of year or so.
> > >>
> > >> The reason I am bringing this up is that I know various users waiting
> > very
> > >> much for FLIP-6 and Network Stack enhancements. Given that these
> issues
> > >> were flagged for release 1.4, the users were planning to have them
> > rather
> > >> soon.
> > >>
> > >> Stephan
> > >>
> > >>
> > >> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
> [hidden email]>
> > >> wrote:
> > >>
> > >>> +1 Excellent
> > >>>
> > >>> I'd like to volunteer as release manager. I already set
> > >> up a Kanban board
> > >>> to monitor the open blocking (and non-blocking) issues
> > >> for 1.4, though this
> > >>> is independent of me volunteering as release manager. We
> > >> should all go over
> > >>> these issues and see which ones should actually be blockin
> > >> g and which ones
> > >>> are not yet on that list.
> > >>>
> > >>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
> > wrote:
> > >>>>
> > >>>> Cool!!!
> > >>>>
> > >>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]
> >
> > >>> wrote:
> > >>>>
> > >>>>> Hi all,
> > >>>>>
> > >>>>> I want to revive the discussion about releasing Flink 1.4 [1] and
> the
> > >>> set
> > >>>>> of features to include.
> > >>>>>
> > >>>>> The gist of the previous discussion was that we postponed the
> feature
> > >>>>> freeze for 1.4 in order to include some more features w
> > >> hich were being
> > >>>>> developed. By now, we have completed a good set of features such as
> > >>> exactly
> > >>>>> once Kafka producer, reduced dependency footprint, Hado
> > >> op-free Flink and
> > >>>>> many bug fixes. I believe that these features will
> > >> make good release and
> > >>>>> users are already waiting for them.
> > >>>>>
> > >>>>> Some of the other features which we wanted to include,
> > >> mainly Flip-6, to
> > >>>>> some extent the network stack enhancements and the state decoupling
> > >>> still
> > >>>>> need some more time. Since these features are major
> > >> changes to Flink's
> > >>>>> runtime, it would be in my opinion a good idea to cut a
> > >> stable release
> > >>> with
> > >>>>> the above-mentioned feature set now and give the engine
> > >> features a bit
> > >>> more
> > >>>>> time to ripen and be properly tested.
> > >>>>>
> > >>>>> Therefore, I would actually be in favour of aiming for
> > >> a quick release
> > >>>>> meaning that we now concentrate mainly on fixing bugs and critical
> > >>> issues.
> > >>>>> Moreover, I'm optimistic that the delayed features will be
> completed
> > >>> soon
> > >>>>> such that we can deliver them with the next release. Wh
> > >> at do you think?
> > >>>>>
> > >>>>> [1]
> > >>>>>
> > >>>>> http://apache-flink-mailing-list-archive.1008284.n3.
> > >>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Till
> > >>>>>
> > >>>> --
> > >>>> Liu, Renjie
> > >>>> Software Engineer, MVAD
> > >>>
> > >>>
> > >>
> > >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Piotr Nowojski
+1

I would only try to merge as many of the smaller network stack improvements as possible for 1.4, since they give quite big performance improvement.

Piotrek

> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
>
> +1 from our side on this plan.
>
> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]> wrote:
>
>> OK, sounds good to me.
>>
>> We have a couple of bugs to fix for the Table API / SQL but have PRs for
>> most of them.
>>
>> There's only one major issue that I'd like to include in 1.4.0 which is a
>> refactoring of the TableSource interface.
>> This effort has already started and is currently waiting for reviews /
>> comments.
>> I'm quite confident that we can get it in within the next two weeks.
>>
>> Cheers, Fabian
>>
>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>>
>>> @Bowen I started marking essential stuff as blocking (with fixVersion
>>> 1.4.0). You're right, that we should start moving things to 1.5.0 that
>> are
>>> not blocking and that we don't think will make it into 1.4.0. I think we
>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
>>> fixVersion 1.4.0.
>>>
>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
>>> wrote:
>>>>
>>>> great
>>>>
>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
>>>> [hidden email]> wrote:
>>>>
>>>>> totally agree with the way.--------------------------
>>>>> ----------------------------------------发件人:Stephan Ewen <
>>> [hidden email]
>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
>>> [hidden email]>主
>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
>>>>> I am in favor of doing this, if we can set it up in the following way.
>>>>>
>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
>>>>> stable cut before the fundamental changes go in.
>>>>>
>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
>> state
>>>>> restore, etc). directly (or very soon) after.
>>>>> - We try to stabilize these changes and release 1.5 asap after that.
>>>>> Ideally Around end of year or so.
>>>>>
>>>>> The reason I am bringing this up is that I know various users waiting
>>> very
>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
>> issues
>>>>> were flagged for release 1.4, the users were planning to have them
>>> rather
>>>>> soon.
>>>>>
>>>>> Stephan
>>>>>
>>>>>
>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> +1 Excellent
>>>>>>
>>>>>> I'd like to volunteer as release manager. I already set
>>>>> up a Kanban board
>>>>>> to monitor the open blocking (and non-blocking) issues
>>>>> for 1.4, though this
>>>>>> is independent of me volunteering as release manager. We
>>>>> should all go over
>>>>>> these issues and see which ones should actually be blockin
>>>>> g and which ones
>>>>>> are not yet on that list.
>>>>>>
>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
>>> wrote:
>>>>>>>
>>>>>>> Cool!!!
>>>>>>>
>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]
>>>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I want to revive the discussion about releasing Flink 1.4 [1] and
>> the
>>>>>> set
>>>>>>>> of features to include.
>>>>>>>>
>>>>>>>> The gist of the previous discussion was that we postponed the
>> feature
>>>>>>>> freeze for 1.4 in order to include some more features w
>>>>> hich were being
>>>>>>>> developed. By now, we have completed a good set of features such as
>>>>>> exactly
>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
>>>>> op-free Flink and
>>>>>>>> many bug fixes. I believe that these features will
>>>>> make good release and
>>>>>>>> users are already waiting for them.
>>>>>>>>
>>>>>>>> Some of the other features which we wanted to include,
>>>>> mainly Flip-6, to
>>>>>>>> some extent the network stack enhancements and the state decoupling
>>>>>> still
>>>>>>>> need some more time. Since these features are major
>>>>> changes to Flink's
>>>>>>>> runtime, it would be in my opinion a good idea to cut a
>>>>> stable release
>>>>>> with
>>>>>>>> the above-mentioned feature set now and give the engine
>>>>> features a bit
>>>>>> more
>>>>>>>> time to ripen and be properly tested.
>>>>>>>>
>>>>>>>> Therefore, I would actually be in favour of aiming for
>>>>> a quick release
>>>>>>>> meaning that we now concentrate mainly on fixing bugs and critical
>>>>>> issues.
>>>>>>>> Moreover, I'm optimistic that the delayed features will be
>> completed
>>>>>> soon
>>>>>>>> such that we can deliver them with the next release. Wh
>>>>> at do you think?
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Till
>>>>>>>>
>>>>>>> --
>>>>>>> Liu, Renjie
>>>>>>> Software Engineer, MVAD
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Aljoscha Krettek-2
It seems we all agree on the general plan. Now we have to talk actual dates. I propose an aggressive feature-freeze date of October 31st (that would be in 2 weeks) so that we can release 1.4 early and can go full-speed with the features moved to 1.5.

We can still merge in new features during the next two weeks but I propose that we then cut a release branch, start testing, and only merge bug fixes until the release. What do you think?

> On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]> wrote:
>
> +1
>
> I would only try to merge as many of the smaller network stack improvements as possible for 1.4, since they give quite big performance improvement.
>
> Piotrek
>
>> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
>>
>> +1 from our side on this plan.
>>
>> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]> wrote:
>>
>>> OK, sounds good to me.
>>>
>>> We have a couple of bugs to fix for the Table API / SQL but have PRs for
>>> most of them.
>>>
>>> There's only one major issue that I'd like to include in 1.4.0 which is a
>>> refactoring of the TableSource interface.
>>> This effort has already started and is currently waiting for reviews /
>>> comments.
>>> I'm quite confident that we can get it in within the next two weeks.
>>>
>>> Cheers, Fabian
>>>
>>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>>>
>>>> @Bowen I started marking essential stuff as blocking (with fixVersion
>>>> 1.4.0). You're right, that we should start moving things to 1.5.0 that
>>> are
>>>> not blocking and that we don't think will make it into 1.4.0. I think we
>>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
>>>> fixVersion 1.4.0.
>>>>
>>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
>>>> wrote:
>>>>>
>>>>> great
>>>>>
>>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> totally agree with the way.--------------------------
>>>>>> ----------------------------------------发件人:Stephan Ewen <
>>>> [hidden email]
>>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
>>>> [hidden email]>主
>>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
>>>>>> I am in favor of doing this, if we can set it up in the following way.
>>>>>>
>>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
>>>>>> stable cut before the fundamental changes go in.
>>>>>>
>>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
>>> state
>>>>>> restore, etc). directly (or very soon) after.
>>>>>> - We try to stabilize these changes and release 1.5 asap after that.
>>>>>> Ideally Around end of year or so.
>>>>>>
>>>>>> The reason I am bringing this up is that I know various users waiting
>>>> very
>>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
>>> issues
>>>>>> were flagged for release 1.4, the users were planning to have them
>>>> rather
>>>>>> soon.
>>>>>>
>>>>>> Stephan
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 Excellent
>>>>>>>
>>>>>>> I'd like to volunteer as release manager. I already set
>>>>>> up a Kanban board
>>>>>>> to monitor the open blocking (and non-blocking) issues
>>>>>> for 1.4, though this
>>>>>>> is independent of me volunteering as release manager. We
>>>>>> should all go over
>>>>>>> these issues and see which ones should actually be blockin
>>>>>> g and which ones
>>>>>>> are not yet on that list.
>>>>>>>
>>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
>>>> wrote:
>>>>>>>>
>>>>>>>> Cool!!!
>>>>>>>>
>>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]
>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I want to revive the discussion about releasing Flink 1.4 [1] and
>>> the
>>>>>>> set
>>>>>>>>> of features to include.
>>>>>>>>>
>>>>>>>>> The gist of the previous discussion was that we postponed the
>>> feature
>>>>>>>>> freeze for 1.4 in order to include some more features w
>>>>>> hich were being
>>>>>>>>> developed. By now, we have completed a good set of features such as
>>>>>>> exactly
>>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
>>>>>> op-free Flink and
>>>>>>>>> many bug fixes. I believe that these features will
>>>>>> make good release and
>>>>>>>>> users are already waiting for them.
>>>>>>>>>
>>>>>>>>> Some of the other features which we wanted to include,
>>>>>> mainly Flip-6, to
>>>>>>>>> some extent the network stack enhancements and the state decoupling
>>>>>>> still
>>>>>>>>> need some more time. Since these features are major
>>>>>> changes to Flink's
>>>>>>>>> runtime, it would be in my opinion a good idea to cut a
>>>>>> stable release
>>>>>>> with
>>>>>>>>> the above-mentioned feature set now and give the engine
>>>>>> features a bit
>>>>>>> more
>>>>>>>>> time to ripen and be properly tested.
>>>>>>>>>
>>>>>>>>> Therefore, I would actually be in favour of aiming for
>>>>>> a quick release
>>>>>>>>> meaning that we now concentrate mainly on fixing bugs and critical
>>>>>>> issues.
>>>>>>>>> Moreover, I'm optimistic that the delayed features will be
>>> completed
>>>>>>> soon
>>>>>>>>> such that we can deliver them with the next release. Wh
>>>>>> at do you think?
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Till
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Liu, Renjie
>>>>>>>> Software Engineer, MVAD
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

yew1eb
Cool!!!
Very much looking forward flink 1.4.

Cheers,
Hai Zhou

> 在 2017年10月18日,03:04,Aljoscha Krettek <[hidden email]> 写道:
>
> Cool!!!

Reply | Threaded
Open this post in threaded view
|

回复:[DISCUSS] Releasing Flink 1.4

wangzhijiang
In reply to this post by Piotr Nowojski
@Stephan Ewen , do you think the barrier alignment improvement should be covered in release-1.4 or release-1.5?It can avoid spilling data on receiver side for exactly-once and speed the barrier alignment.
We have implemented this feature based on new credit-based flow control and verified the good performance.If needed, I will try to submit the related PRs after network stack merged into 1.4 ASAP.
------------------------------------------------------------------发件人:Aljoscha Krettek <[hidden email]>发送时间:2017年10月18日(星期三) 03:04收件人:dev <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
It seems we all agree on the general plan. Now we have to talk actual dates. I propose an aggressive feature-freeze date of October 31st (that would be in 2 weeks) so that we can release 1.4 early and can go full-speed with the features moved to 1.5.

We can still merge in new features during the next two weeks but I propose that we then cut a release branch, start testing, and only merge bug fixes until the release. What do you think?

> On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]> wrote:

> +1

> I would only try to merge as many of the smaller network stack improvements as possible for 1.4, since they give quite big performance improvement.

> Piotrek

>> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
>> 
>> +1 from our side on this plan.
>> 
>> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]> wrote:
>> 
>>> OK, sounds good to me.
>>> 
>>> We have a couple of bugs to fix for the Table API / SQL but have PRs for
>>> most of them.
>>> 
>>> There's only one major issue that I'd like to include in 1.4.0 which is a
>>> refactoring of the TableSource interface.
>>> This effort has already started and is currently waiting for reviews /
>>> comments.
>>> I'm quite confident that we can get it in within the next two weeks.
>>> 
>>> Cheers, Fabian
>>> 
>>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>>> 
>>>> @Bowen I started marking essential stuff as blocking (with fixVersion
>>>> 1.4.0). You're right, that we should start moving things to 1.5.0 that
>>> are
>>>> not blocking and that we don't think will make it into 1.4.0. I think we
>>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
>>>> fixVersion 1.4.0.
>>>> 
>>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
>>>> wrote:
>>>>> 
>>>>> great
>>>>> 
>>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
>>>>> [hidden email]> wrote:
>>>>> 
>>>>>> totally agree with the way.--------------------------
>>>>>> ----------------------------------------发件人:Stephan Ewen <
>>>> [hidden email]
>>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
>>>> [hidden email]>主
>>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
>>>>>> I am in favor of doing this, if we can set it up in the following way.
>>>>>> 
>>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
>>>>>> stable cut before the fundamental changes go in.
>>>>>> 
>>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
>>> state
>>>>>> restore, etc). directly (or very soon) after.
>>>>>> - We try to stabilize these changes and release 1.5 asap after that.
>>>>>> Ideally Around end of year or so.
>>>>>> 
>>>>>> The reason I am bringing this up is that I know various users waiting
>>>> very
>>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
>>> issues
>>>>>> were flagged for release 1.4, the users were planning to have them
>>>> rather
>>>>>> soon.
>>>>>> 
>>>>>> Stephan
>>>>>> 
>>>>>> 
>>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
>>> [hidden email]>
>>>>>> wrote:
>>>>>> 
>>>>>>> +1 Excellent
>>>>>>> 
>>>>>>> I'd like to volunteer as release manager. I already set
>>>>>> up a Kanban board
>>>>>>> to monitor the open blocking (and non-blocking) issues
>>>>>> for 1.4, though this
>>>>>>> is independent of me volunteering as release manager. We
>>>>>> should all go over
>>>>>>> these issues and see which ones should actually be blockin
>>>>>> g and which ones
>>>>>>> are not yet on that list.
>>>>>>> 
>>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
>>>> wrote:
>>>>>>>> 
>>>>>>>> Cool!!!
>>>>>>>> 
>>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]
>>>> 
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> I want to revive the discussion about releasing Flink 1.4 [1] and
>>> the
>>>>>>> set
>>>>>>>>> of features to include.
>>>>>>>>> 
>>>>>>>>> The gist of the previous discussion was that we postponed the
>>> feature
>>>>>>>>> freeze for 1.4 in order to include some more features w
>>>>>> hich were being
>>>>>>>>> developed. By now, we have completed a good set of features such as
>>>>>>> exactly
>>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
>>>>>> op-free Flink and
>>>>>>>>> many bug fixes. I believe that these features will
>>>>>> make good release and
>>>>>>>>> users are already waiting for them.
>>>>>>>>> 
>>>>>>>>> Some of the other features which we wanted to include,
>>>>>> mainly Flip-6, to
>>>>>>>>> some extent the network stack enhancements and the state decoupling
>>>>>>> still
>>>>>>>>> need some more time. Since these features are major
>>>>>> changes to Flink's
>>>>>>>>> runtime, it would be in my opinion a good idea to cut a
>>>>>> stable release
>>>>>>> with
>>>>>>>>> the above-mentioned feature set now and give the engine
>>>>>> features a bit
>>>>>>> more
>>>>>>>>> time to ripen and be properly tested.
>>>>>>>>> 
>>>>>>>>> Therefore, I would actually be in favour of aiming for
>>>>>> a quick release
>>>>>>>>> meaning that we now concentrate mainly on fixing bugs and critical
>>>>>>> issues.
>>>>>>>>> Moreover, I'm optimistic that the delayed features will be
>>> completed
>>>>>>> soon
>>>>>>>>> such that we can deliver them with the next release. Wh
>>>>>> at do you think?
>>>>>>>>> 
>>>>>>>>> [1]
>>>>>>>>> 
>>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Till
>>>>>>>>> 
>>>>>>>> --
>>>>>>>> Liu, Renjie
>>>>>>>> Software Engineer, MVAD
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Till Rohrmann
Thanks for driving this effort Aljoscha. I would be in favour of quick
feature freeze (31st of October). However, looking at the list of open and
in progress blockers we still have 53 blocker issues. I think it is
unrealistic to complete all of them within the next two weeks. I would
therefore propose to go again over the list of blockers to decide what is
really a blocker for 1.4 and what not.

@Zhijiang, my gut feeling is that we will have this feature in Flink 1.5.
There are simply too many other open issues which we have to address first
in order to finish the 1.4 release. Moreover, merging it early in the
release cycle of 1.5 will give the barrier alignment improvements a good
amount of exposure to spot potential problems before releasing it
officially.

Cheers,
Till

On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) <
[hidden email]> wrote:

> @Stephan Ewen , do you think the barrier alignment improvement should be
> covered in release-1.4 or release-1.5?It can avoid spilling data on
> receiver side for exactly-once and speed the barrier alignment.
> We have implemented this feature based on new credit-based flow control
> and verified the good performance.If needed, I will try to submit the
> related PRs after network stack merged into 1.4 ASAP.
> ------------------------------------------------------------------发件人:Aljoscha
> Krettek <[hidden email]>发送时间:2017年10月18日(星期三) 03:04收件人:dev <
> [hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
> It seems we all agree on the general plan. Now we have to
> talk actual dates. I propose an aggressive feature-freeze
> date of October 31st (that would be in 2 weeks) so that
> we can release 1.4 early and can go full-speed with the
> features moved to 1.5.
>
> We can still merge in new features during the next two
> weeks but I propose that we then cut a release branch,
> start testing, and only merge bug fixes until the release.
> What do you think?
>
> > On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]
> > wrote:
> >
> > +1
> >
> > I would only try to merge as many of the smaller
> network stack improvements as possible for 1.4, since they
> give quite big performance improvement.
> >
> > Piotrek
> >
> >> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
> >>
> >> +1 from our side on this plan.
> >>
> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]
> > wrote:
> >>
> >>> OK, sounds good to me.
> >>>
> >>> We have a couple of bugs to fix for the Table API /
> SQL but have PRs for
> >>> most of them.
> >>>
> >>> There's only one major issue that I'd like to
> include in 1.4.0 which is a
> >>> refactoring of the TableSource interface.
> >>> This effort has already started and is currently waiting for reviews /
> >>> comments.
> >>> I'm quite confident that we can get it in within the next two weeks.
> >>>
> >>> Cheers, Fabian
> >>>
> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> >>>
> >>>> @Bowen I started marking essential stuff as blocking (with fixVersion
> >>>> 1.4.0). You're right, that we should start moving
> things to 1.5.0 that
> >>> are
> >>>> not blocking and that we don't think will make it into
> 1.4.0. I think we
> >>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
> >>>> fixVersion 1.4.0.
> >>>>
> >>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
> >>>> wrote:
> >>>>>
> >>>>> great
> >>>>>
> >>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> >>>>> [hidden email]> wrote:
> >>>>>
> >>>>>> totally agree with the way.--------------------------
> >>>>>> ----------------------------------------发件人:Stephan Ewen <
> >>>> [hidden email]
> >>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
> >>>> [hidden email]>主
> >>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
> >>>>>> I am in favor of doing this, if we can set it up in
> the following way.
> >>>>>>
> >>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> >>>>>> stable cut before the fundamental changes go in.
> >>>>>>
> >>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
> >>> state
> >>>>>> restore, etc). directly (or very soon) after.
> >>>>>> - We try to stabilize these changes and release 1.5
> asap after that.
> >>>>>> Ideally Around end of year or so.
> >>>>>>
> >>>>>> The reason I am bringing this up is that I
> know various users waiting
> >>>> very
> >>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
> >>> issues
> >>>>>> were flagged for release 1.4, the users were planning to have them
> >>>> rather
> >>>>>> soon.
> >>>>>>
> >>>>>> Stephan
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
> >>> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 Excellent
> >>>>>>>
> >>>>>>> I'd like to volunteer as release manager. I already set
> >>>>>> up a Kanban board
> >>>>>>> to monitor the open blocking (and non-blocking) issues
> >>>>>> for 1.4, though this
> >>>>>>> is independent of me volunteering as release manager. We
> >>>>>> should all go over
> >>>>>>> these issues and see which ones should actually be blockin
> >>>>>> g and which ones
> >>>>>>> are not yet on that list.
> >>>>>>>
> >>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> Cool!!!
> >>>>>>>>
> >>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <troh
> [hidden email]
> >>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> I want to revive the discussion about
> releasing Flink 1.4 [1] and
> >>> the
> >>>>>>> set
> >>>>>>>>> of features to include.
> >>>>>>>>>
> >>>>>>>>> The gist of the previous discussion was that we postponed the
> >>> feature
> >>>>>>>>> freeze for 1.4 in order to include some more features w
> >>>>>> hich were being
> >>>>>>>>> developed. By now, we have completed a good set
> of features such as
> >>>>>>> exactly
> >>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
> >>>>>> op-free Flink and
> >>>>>>>>> many bug fixes. I believe that these features will
> >>>>>> make good release and
> >>>>>>>>> users are already waiting for them.
> >>>>>>>>>
> >>>>>>>>> Some of the other features which we wanted to include,
> >>>>>> mainly Flip-6, to
> >>>>>>>>> some extent the network stack enhancements
> and the state decoupling
> >>>>>>> still
> >>>>>>>>> need some more time. Since these features are major
> >>>>>> changes to Flink's
> >>>>>>>>> runtime, it would be in my opinion a good idea to cut a
> >>>>>> stable release
> >>>>>>> with
> >>>>>>>>> the above-mentioned feature set now and give the engine
> >>>>>> features a bit
> >>>>>>> more
> >>>>>>>>> time to ripen and be properly tested.
> >>>>>>>>>
> >>>>>>>>> Therefore, I would actually be in favour of aiming for
> >>>>>> a quick release
> >>>>>>>>> meaning that we now concentrate mainly on fixing
> bugs and critical
> >>>>>>> issues.
> >>>>>>>>> Moreover, I'm optimistic that the delayed features will be
> >>> completed
> >>>>>>> soon
> >>>>>>>>> such that we can deliver them with the next release. Wh
> >>>>>> at do you think?
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
> >>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Till
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Liu, Renjie
> >>>>>>>> Software Engineer, MVAD
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>
Reply | Threaded
Open this post in threaded view
|

回复:[DISCUSS] Releasing Flink 1.4

wangzhijiang
In reply to this post by wangzhijiang
        Thanks for replying @Till Rohrmann , and I agree with your suggestion.
        Cheers,        Zhijiang
------------------------------------------------------------------发件人:Till Rohrmann <[hidden email]>发送时间:2017年10月18日(星期三) 16:27收件人:dev <[hidden email]>; Zhijiang(wangzhijiang999) <[hidden email]>抄 送:Stephan Ewen <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
Thanks for driving this effort Aljoscha. I would be in favour of quick
feature freeze (31st of October). However, looking at the list of open and
in progress blockers we still have 53 blocker issues. I think it is
unrealistic to complete all of them within the next two weeks. I would
therefore propose to go again over the list of blockers to decide what is
really a blocker for 1.4 and what not.

@Zhijiang, my gut feeling is that we will have this feature in Flink 1.5.
There are simply too many other open issues which we have to address first
in order to finish the 1.4 release. Moreover, merging it early in the
release cycle of 1.5 will give the barrier alignment improvements a good
amount of exposure to spot potential problems before releasing it
officially.

Cheers,
Till

On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) <
[hidden email]> wrote:

> @Stephan Ewen , do you think the barrier alignment improvement should be
> covered in release-1.4 or release-1.5?It can avoid spilling data on
> receiver side for exactly-once and speed the barrier alignment.
> We have implemented this feature based on new credit-based flow control
> and verified the good performance.If needed, I will try to submit the
> related PRs after network stack merged into 1.4 ASAP.
> ------------------------------------------------------------------发件人:Aljoscha
> Krettek <[hidden email]>发送时间:2017年10月18日(星期三) 03:04收件人:dev <
[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
> It seems we all agree on the general plan. Now we have to
> talk actual dates. I propose an aggressive feature-freeze
> date of October 31st (that would be in 2 weeks) so that
> we can release 1.4 early and can go full-speed with the
> features moved to 1.5.
>
> We can still merge in new features during the next two
> weeks but I propose that we then cut a release branch,
> start testing, and only merge bug fixes until the release.
> What do you think?
>
> > On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]
> > wrote:
> >
> > +1
> >
> > I would only try to merge as many of the smaller
> network stack improvements as possible for 1.4, since they
> give quite big performance improvement.
> >
> > Piotrek
> >
> >> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
> >>
> >> +1 from our side on this plan.
> >>
> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]
> > wrote:
> >>
> >>> OK, sounds good to me.
> >>>
> >>> We have a couple of bugs to fix for the Table API /
> SQL but have PRs for
> >>> most of them.
> >>>
> >>> There's only one major issue that I'd like to
> include in 1.4.0 which is a
> >>> refactoring of the TableSource interface.
> >>> This effort has already started and is currently waiting for reviews /
> >>> comments.
> >>> I'm quite confident that we can get it in within the next two weeks.
> >>>
> >>> Cheers, Fabian
> >>>
> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> >>>
> >>>> @Bowen I started marking essential stuff as blocking (with fixVersion
> >>>> 1.4.0). You're right, that we should start moving
> things to 1.5.0 that
> >>> are
> >>>> not blocking and that we don't think will make it into
> 1.4.0. I think we
> >>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
> >>>> fixVersion 1.4.0.
> >>>>
> >>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
> >>>> wrote:
> >>>>>
> >>>>> great
> >>>>>
> >>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> >>>>> [hidden email]> wrote:
> >>>>>
> >>>>>> totally agree with the way.--------------------------
> >>>>>> ----------------------------------------发件人:Stephan Ewen <
> >>>> [hidden email]
> >>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
> >>>> [hidden email]>主
> >>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
> >>>>>> I am in favor of doing this, if we can set it up in
> the following way.
> >>>>>>
> >>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> >>>>>> stable cut before the fundamental changes go in.
> >>>>>>
> >>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
> >>> state
> >>>>>> restore, etc). directly (or very soon) after.
> >>>>>> - We try to stabilize these changes and release 1.5
> asap after that.
> >>>>>> Ideally Around end of year or so.
> >>>>>>
> >>>>>> The reason I am bringing this up is that I
> know various users waiting
> >>>> very
> >>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
> >>> issues
> >>>>>> were flagged for release 1.4, the users were planning to have them
> >>>> rather
> >>>>>> soon.
> >>>>>>
> >>>>>> Stephan
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
> >>> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 Excellent
> >>>>>>>
> >>>>>>> I'd like to volunteer as release manager. I already set
> >>>>>> up a Kanban board
> >>>>>>> to monitor the open blocking (and non-blocking) issues
> >>>>>> for 1.4, though this
> >>>>>>> is independent of me volunteering as release manager. We
> >>>>>> should all go over
> >>>>>>> these issues and see which ones should actually be blockin
> >>>>>> g and which ones
> >>>>>>> are not yet on that list.
> >>>>>>>
> >>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> Cool!!!
> >>>>>>>>
> >>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <troh
[hidden email]
> >>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> I want to revive the discussion about
> releasing Flink 1.4 [1] and
> >>> the
> >>>>>>> set
> >>>>>>>>> of features to include.
> >>>>>>>>>
> >>>>>>>>> The gist of the previous discussion was that we postponed the
> >>> feature
> >>>>>>>>> freeze for 1.4 in order to include some more features w
> >>>>>> hich were being
> >>>>>>>>> developed. By now, we have completed a good set
> of features such as
> >>>>>>> exactly
> >>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
> >>>>>> op-free Flink and
> >>>>>>>>> many bug fixes. I believe that these features will
> >>>>>> make good release and
> >>>>>>>>> users are already waiting for them.
> >>>>>>>>>
> >>>>>>>>> Some of the other features which we wanted to include,
> >>>>>> mainly Flip-6, to
> >>>>>>>>> some extent the network stack enhancements
> and the state decoupling
> >>>>>>> still
> >>>>>>>>> need some more time. Since these features are major
> >>>>>> changes to Flink's
> >>>>>>>>> runtime, it would be in my opinion a good idea to cut a
> >>>>>> stable release
> >>>>>>> with
> >>>>>>>>> the above-mentioned feature set now and give the engine
> >>>>>> features a bit
> >>>>>>> more
> >>>>>>>>> time to ripen and be properly tested.
> >>>>>>>>>
> >>>>>>>>> Therefore, I would actually be in favour of aiming for
> >>>>>> a quick release
> >>>>>>>>> meaning that we now concentrate mainly on fixing
> bugs and critical
> >>>>>>> issues.
> >>>>>>>>> Moreover, I'm optimistic that the delayed features will be
> >>> completed
> >>>>>>> soon
> >>>>>>>>> such that we can deliver them with the next release. Wh
> >>>>>> at do you think?
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
> >>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Till
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Liu, Renjie
> >>>>>>>> Software Engineer, MVAD
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Stefan Richter
In reply to this post by Till Rohrmann
I think this is a good plan.

+1 from me.

> Am 13.10.2017 um 17:49 schrieb Till Rohrmann <[hidden email]>:
>
> Hi all,
>
> I want to revive the discussion about releasing Flink 1.4 [1] and the set
> of features to include.
>
> The gist of the previous discussion was that we postponed the feature
> freeze for 1.4 in order to include some more features which were being
> developed. By now, we have completed a good set of features such as exactly
> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> many bug fixes. I believe that these features will make good release and
> users are already waiting for them.
>
> Some of the other features which we wanted to include, mainly Flip-6, to
> some extent the network stack enhancements and the state decoupling still
> need some more time. Since these features are major changes to Flink's
> runtime, it would be in my opinion a good idea to cut a stable release with
> the above-mentioned feature set now and give the engine features a bit more
> time to ripen and be properly tested.
>
> Therefore, I would actually be in favour of aiming for a quick release
> meaning that we now concentrate mainly on fixing bugs and critical issues.
> Moreover, I'm optimistic that the delayed features will be completed soon
> such that we can deliver them with the next release. What do you think?
>
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>
> Cheers,
> Till

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Till Rohrmann
I went through the list of blocking issues and thinned it down to 25 open
blockers and 12 in progress blockers. Please object if you don't agree with
a specific issue. If not, then I suggest that we should concentrate on
fixing these issues.

Cheers,
Till

On Wed, Oct 18, 2017 at 10:54 AM, Stefan Richter <
[hidden email]> wrote:

> I think this is a good plan.
>
> +1 from me.
>
> > Am 13.10.2017 um 17:49 schrieb Till Rohrmann <[hidden email]>:
> >
> > Hi all,
> >
> > I want to revive the discussion about releasing Flink 1.4 [1] and the set
> > of features to include.
> >
> > The gist of the previous discussion was that we postponed the feature
> > freeze for 1.4 in order to include some more features which were being
> > developed. By now, we have completed a good set of features such as
> exactly
> > once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> > many bug fixes. I believe that these features will make good release and
> > users are already waiting for them.
> >
> > Some of the other features which we wanted to include, mainly Flip-6, to
> > some extent the network stack enhancements and the state decoupling still
> > need some more time. Since these features are major changes to Flink's
> > runtime, it would be in my opinion a good idea to cut a stable release
> with
> > the above-mentioned feature set now and give the engine features a bit
> more
> > time to ripen and be properly tested.
> >
> > Therefore, I would actually be in favour of aiming for a quick release
> > meaning that we now concentrate mainly on fixing bugs and critical
> issues.
> > Moreover, I'm optimistic that the delayed features will be completed soon
> > such that we can deliver them with the next release. What do you think?
> >
> > [1]
> > http://apache-flink-mailing-list-archive.1008284.n3.
> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >
> > Cheers,
> > Till
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Releasing Flink 1.4

Stephan Ewen-3
In reply to this post by Till Rohrmann
Hi Zhijiang!

The proposal is to release 1.4 asap and have the master switch so 1.5 soon
(early November or so). Merge the network code directly after that - the
1.5 release would come after flip-6, network stack, and local state caching
is merged and tested.

That gives us a better QA window.

The reason is that we got feedback from Netflix and some other users that
1.3 had too many bugs, because we merged big changes directly before the
release. We are worried that users lose trust in Flink of we release these
big new features too fast and without intensively testing them.

What do you think?

Stephan

On Oct 18, 2017 09:27, "Till Rohrmann" <[hidden email]> wrote:

> Thanks for driving this effort Aljoscha. I would be in favour of quick
> feature freeze (31st of October). However, looking at the list of open and
> in progress blockers we still have 53 blocker issues. I think it is
> unrealistic to complete all of them within the next two weeks. I would
> therefore propose to go again over the list of blockers to decide what is
> really a blocker for 1.4 and what not.
>
> @Zhijiang, my gut feeling is that we will have this feature in Flink 1.5.
> There are simply too many other open issues which we have to address first
> in order to finish the 1.4 release. Moreover, merging it early in the
> release cycle of 1.5 will give the barrier alignment improvements a good
> amount of exposure to spot potential problems before releasing it
> officially.
>
> Cheers,
> Till
>
> On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) <
> [hidden email]> wrote:
>
>> @Stephan Ewen , do you think the barrier alignment improvement should be
>> covered in release-1.4 or release-1.5?It can avoid spilling data on
>> receiver side for exactly-once and speed the barrier alignment.
>> We have implemented this feature based on new credit-based flow control
>> and verified the good performance.If needed, I will try to submit the
>> related PRs after network stack merged into 1.4 ASAP.
>> ------------------------------------------------------------------发件人:Aljoscha
>> Krettek <[hidden email]>发送时间:2017年10月18日(星期三) 03:04收件人:dev <
>> [hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
>> It seems we all agree on the general plan. Now we have to ta
>> lk actual dates. I propose an aggressive feature-freeze date
>>  of October 31st (that would be in 2 weeks) so that we can
>> release 1.4 early and can go full-speed with the features moved to 1.5.
>>
>> We can still merge in new features during the next two weeks
>>  but I propose that we then cut a release branch, start
>> testing, and only merge bug fixes until the release. What do you think?
>>
>> > On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]
>> > wrote:
>> >
>> > +1
>> >
>> > I would only try to merge as many of the smaller network
>> stack improvements as possible for 1.4, since they give
>> quite big performance improvement.
>> >
>> > Piotrek
>> >
>> >> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:
>> >>
>> >> +1 from our side on this plan.
>> >>
>> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]
>> > wrote:
>> >>
>> >>> OK, sounds good to me.
>> >>>
>> >>> We have a couple of bugs to fix for the Table API / SQL
>> but have PRs for
>> >>> most of them.
>> >>>
>> >>> There's only one major issue that I'd like to include
>> in 1.4.0 which is a
>> >>> refactoring of the TableSource interface.
>> >>> This effort has already started and is currently waiting
>>  for reviews /
>> >>> comments.
>> >>> I'm quite confident that we can get it in within the next two weeks.
>> >>>
>> >>> Cheers, Fabian
>> >>>
>> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>> >>>
>> >>>> @Bowen I started marking essential stuff as blocking (w
>> ith fixVersion
>> >>>> 1.4.0). You're right, that we should start moving thing
>> s to 1.5.0 that
>> >>> are
>> >>>> not blocking and that we don't think will make it into
>> 1.4.0. I think we
>> >>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with
>> >>>> fixVersion 1.4.0.
>> >>>>
>> >>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>
>> >>>> wrote:
>> >>>>>
>> >>>>> great
>> >>>>>
>> >>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
>> >>>>> [hidden email]> wrote:
>> >>>>>
>> >>>>>> totally agree with the way.--------------------------
>> >>>>>> ----------------------------------------发件人:Stephan Ewen <
>> >>>> [hidden email]
>> >>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <
>> >>>> [hidden email]>主
>> >>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4
>> >>>>>> I am in favor of doing this, if we can set it up in t
>> he following way.
>> >>>>>>
>> >>>>>> - We put out the 1.4 release now, as Till and Aljosch
>> a suggested. A
>> >>>>>> stable cut before the fundamental changes go in.
>> >>>>>>
>> >>>>>> - We merge the very big changes (FLIP-6, Network stack, localized
>> >>> state
>> >>>>>> restore, etc). directly (or very soon) after.
>> >>>>>> - We try to stabilize these changes and release 1.5 a
>> sap after that.
>> >>>>>> Ideally Around end of year or so.
>> >>>>>>
>> >>>>>> The reason I am bringing this up is that I know
>> various users waiting
>> >>>> very
>> >>>>>> much for FLIP-6 and Network Stack enhancements. Given that these
>> >>> issues
>> >>>>>> were flagged for release 1.4, the users were planning to have them
>> >>>> rather
>> >>>>>> soon.
>> >>>>>>
>> >>>>>> Stephan
>> >>>>>>
>> >>>>>>
>> >>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
>> >>> [hidden email]>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> +1 Excellent
>> >>>>>>>
>> >>>>>>> I'd like to volunteer as release manager. I already set
>> >>>>>> up a Kanban board
>> >>>>>>> to monitor the open blocking (and non-blocking) issues
>> >>>>>> for 1.4, though this
>> >>>>>>> is independent of me volunteering as release manager. We
>> >>>>>> should all go over
>> >>>>>>> these issues and see which ones should actually be blockin
>> >>>>>> g and which ones
>> >>>>>>> are not yet on that list.
>> >>>>>>>
>> >>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>
>> >>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Cool!!!
>> >>>>>>>>
>> >>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <troh
>> [hidden email]
>> >>>>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi all,
>> >>>>>>>>>
>> >>>>>>>>> I want to revive the discussion about releasing
>> Flink 1.4 [1] and
>> >>> the
>> >>>>>>> set
>> >>>>>>>>> of features to include.
>> >>>>>>>>>
>> >>>>>>>>> The gist of the previous discussion was that we postponed the
>> >>> feature
>> >>>>>>>>> freeze for 1.4 in order to include some more features w
>> >>>>>> hich were being
>> >>>>>>>>> developed. By now, we have completed a good set of
>>  features such as
>> >>>>>>> exactly
>> >>>>>>>>> once Kafka producer, reduced dependency footprint, Hado
>> >>>>>> op-free Flink and
>> >>>>>>>>> many bug fixes. I believe that these features will
>> >>>>>> make good release and
>> >>>>>>>>> users are already waiting for them.
>> >>>>>>>>>
>> >>>>>>>>> Some of the other features which we wanted to include,
>> >>>>>> mainly Flip-6, to
>> >>>>>>>>> some extent the network stack enhancements and
>> the state decoupling
>> >>>>>>> still
>> >>>>>>>>> need some more time. Since these features are major
>> >>>>>> changes to Flink's
>> >>>>>>>>> runtime, it would be in my opinion a good idea to cut a
>> >>>>>> stable release
>> >>>>>>> with
>> >>>>>>>>> the above-mentioned feature set now and give the engine
>> >>>>>> features a bit
>> >>>>>>> more
>> >>>>>>>>> time to ripen and be properly tested.
>> >>>>>>>>>
>> >>>>>>>>> Therefore, I would actually be in favour of aiming for
>> >>>>>> a quick release
>> >>>>>>>>> meaning that we now concentrate mainly on fixing b
>> ugs and critical
>> >>>>>>> issues.
>> >>>>>>>>> Moreover, I'm optimistic that the delayed features will be
>> >>> completed
>> >>>>>>> soon
>> >>>>>>>>> such that we can deliver them with the next release. Wh
>> >>>>>> at do you think?
>> >>>>>>>>>
>> >>>>>>>>> [1]
>> >>>>>>>>>
>> >>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.
>> >>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
>> >>>>>>>>>
>> >>>>>>>>> Cheers,
>> >>>>>>>>> Till
>> >>>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Liu, Renjie
>> >>>>>>>> Software Engineer, MVAD
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>>>
>> >>>
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

回复:[DISCUSS] Releasing Flink 1.4

wangzhijiang
In reply to this post by Till Rohrmann
       Hi Stephan!
       Thanks for replying.
       I am clearly now, all the big new features would be covered in 1.5. And it is very necessary to fully test big changes before release.
       Zhijiang
------------------------------------------------------------------发件人:Stephan Ewen <[hidden email]>发送时间:2017年10月18日(星期三) 22:54收件人:Till Rohrmann <[hidden email]>抄 送:dev <[hidden email]>; Zhijiang(wangzhijiang999) <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
Hi Zhijiang!
The proposal is to release 1.4 asap and have the master switch so 1.5 soon (early November or so). Merge the network code directly after that - the 1.5 release would come after flip-6, network stack, and local state caching is merged and tested.
That gives us a better QA window.
The reason is that we got feedback from Netflix and some other users that 1.3 had too many bugs, because we merged big changes directly before the release. We are worried that users lose trust in Flink of we release these big new features too fast and without intensively testing them.
What do you think?
Stephan
On Oct 18, 2017 09:27, "Till Rohrmann" <[hidden email]> wrote:
Thanks for driving this effort Aljoscha. I would be in favour of quick feature freeze (31st of October). However, looking at the list of open and in progress blockers we still have 53 blocker issues. I think it is unrealistic to complete all of them within the next two weeks. I would therefore propose to go again over the list of blockers to decide what is really a blocker for 1.4 and what not.
@Zhijiang, my gut feeling is that we will have this feature in Flink 1.5. There are simply too many other open issues which we have to address first in order to finish the 1.4 release. Moreover, merging it early in the release cycle of 1.5 will give the barrier alignment improvements a good amount of exposure to spot potential problems before releasing it officially.
Cheers,Till
On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) <[hidden email]> wrote:
@Stephan Ewen , do you think the barrier alignment improvement should be covered in release-1.4 or release-1.5?It can avoid spilling data on receiver side for exactly-once and speed the barrier alignment.

We have implemented this feature based on new credit-based flow control and verified the good performance.If needed, I will try to submit the related PRs after network stack merged into 1.4 ASAP.

------------------------------------------------------------------发件人:Aljoscha Krettek <[hidden email]>发送时间:2017年10月18日(星期三) 03:04收件人:dev <[hidden email]>主 题:Re: [DISCUSS] Releasing Flink 1.4
It seems we all agree on the general plan. Now we have to talk actual dates. I propose an aggressive feature-freeze date of October 31st (that would be in 2 weeks) so that we can release 1.4 early and can go full-speed with the features moved to 1.5.

We can still merge in new features during the next two weeks but I propose that we then cut a release branch, start testing, and only merge bug fixes until the release. What do you think?


> On 17. Oct 2017, at 08:37, Piotr Nowojski <[hidden email]> wrote:


> +1



> I would only try to merge as many of the smaller network stack improvements as possible for 1.4, since they give quite big performance improvement.



> Piotrek



>> On 16 Oct 2017, at 17:42, Eron Wright <[hidden email]> wrote:

>> 

>> +1 from our side on this plan.

>> 

>> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske <[hidden email]> wrote:

>> 

>>> OK, sounds good to me.

>>> 

>>> We have a couple of bugs to fix for the Table API / SQL but have PRs for

>>> most of them.

>>> 

>>> There's only one major issue that I'd like to include in 1.4.0 which is a

>>> refactoring of the TableSource interface.

>>> This effort has already started and is currently waiting for reviews /

>>> comments.

>>> I'm quite confident that we can get it in within the next two weeks.

>>> 

>>> Cheers, Fabian

>>> 

>>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek <[hidden email]>:

>>> 

>>>> @Bowen I started marking essential stuff as blocking (with fixVersion

>>>> 1.4.0). You're right, that we should start moving things to 1.5.0 that

>>> are

>>>> not blocking and that we don't think will make it into 1.4.0. I think we

>>>> can only release 1.4.0 if there are 0 (zero) unresolved issues with

>>>> fixVersion 1.4.0.

>>>> 

>>>>> On 14. Oct 2017, at 07:34, Alexandru Gutan <[hidden email]>

>>>> wrote:

>>>>> 

>>>>> great

>>>>> 

>>>>> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <

>>>>> [hidden email]> wrote:

>>>>> 

>>>>>> totally agree with the way.--------------------------

>>>>>> ----------------------------------------发件人:Stephan Ewen <

>>>> [hidden email]

>>>>>>> 发送时间:2017年10月13日(星期五) 21:29收件人:[hidden email] <

>>>> [hidden email]>主

>>>>>> 题:Re: [DISCUSS] Releasing Flink 1.4

>>>>>> I am in favor of doing this, if we can set it up in the following way.

>>>>>> 

>>>>>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A

>>>>>> stable cut before the fundamental changes go in.

>>>>>> 

>>>>>> - We merge the very big changes (FLIP-6, Network stack, localized

>>> state

>>>>>> restore, etc). directly (or very soon) after.

>>>>>> - We try to stabilize these changes and release 1.5 asap after that.

>>>>>> Ideally Around end of year or so.

>>>>>> 

>>>>>> The reason I am bringing this up is that I know various users waiting

>>>> very

>>>>>> much for FLIP-6 and Network Stack enhancements. Given that these

>>> issues

>>>>>> were flagged for release 1.4, the users were planning to have them

>>>> rather

>>>>>> soon.

>>>>>> 

>>>>>> Stephan

>>>>>> 

>>>>>> 

>>>>>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <

>>> [hidden email]>

>>>>>> wrote:

>>>>>> 

>>>>>>> +1 Excellent

>>>>>>> 

>>>>>>> I'd like to volunteer as release manager. I already set

>>>>>> up a Kanban board

>>>>>>> to monitor the open blocking (and non-blocking) issues

>>>>>> for 1.4, though this

>>>>>>> is independent of me volunteering as release manager. We

>>>>>> should all go over

>>>>>>> these issues and see which ones should actually be blockin

>>>>>> g and which ones

>>>>>>> are not yet on that list.

>>>>>>> 

>>>>>>>> On 13. Oct 2017, at 12:24, Renjie Liu <[hidden email]>

>>>> wrote:

>>>>>>>> 

>>>>>>>> Cool!!!

>>>>>>>> 

>>>>>>>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann <[hidden email]

>>>> 

>>>>>>> wrote:

>>>>>>>> 

>>>>>>>>> Hi all,

>>>>>>>>> 

>>>>>>>>> I want to revive the discussion about releasing Flink 1.4 [1] and

>>> the

>>>>>>> set

>>>>>>>>> of features to include.

>>>>>>>>> 

>>>>>>>>> The gist of the previous discussion was that we postponed the

>>> feature

>>>>>>>>> freeze for 1.4 in order to include some more features w

>>>>>> hich were being

>>>>>>>>> developed. By now, we have completed a good set of features such as

>>>>>>> exactly

>>>>>>>>> once Kafka producer, reduced dependency footprint, Hado

>>>>>> op-free Flink and

>>>>>>>>> many bug fixes. I believe that these features will

>>>>>> make good release and

>>>>>>>>> users are already waiting for them.

>>>>>>>>> 

>>>>>>>>> Some of the other features which we wanted to include,

>>>>>> mainly Flip-6, to

>>>>>>>>> some extent the network stack enhancements and the state decoupling

>>>>>>> still

>>>>>>>>> need some more time. Since these features are major

>>>>>> changes to Flink's

>>>>>>>>> runtime, it would be in my opinion a good idea to cut a

>>>>>> stable release

>>>>>>> with

>>>>>>>>> the above-mentioned feature set now and give the engine

>>>>>> features a bit

>>>>>>> more

>>>>>>>>> time to ripen and be properly tested.

>>>>>>>>> 

>>>>>>>>> Therefore, I would actually be in favour of aiming for

>>>>>> a quick release

>>>>>>>>> meaning that we now concentrate mainly on fixing bugs and critical

>>>>>>> issues.

>>>>>>>>> Moreover, I'm optimistic that the delayed features will be

>>> completed

>>>>>>> soon

>>>>>>>>> such that we can deliver them with the next release. Wh

>>>>>> at do you think?

>>>>>>>>> 

>>>>>>>>> [1]

>>>>>>>>> 

>>>>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.

>>>>>>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html

>>>>>>>>> 

>>>>>>>>> Cheers,

>>>>>>>>> Till

>>>>>>>>> 

>>>>>>>> --

>>>>>>>> Liu, Renjie

>>>>>>>> Software Engineer, MVAD

>>>>>>> 

>>>>>>> 

>>>>>> 

>>>>>> 

>>>> 

>>>> 

>>>