[DISCUSS] Schedule and Scope for Flink 1.2
Locked
12
12
[DISCUSS] Schedule and Scope for Flink 1.2
 
|
Hi everybody,
Flink 1.1.0 was released in August and I think it is time to kick off a discussion about the schedule and scope of Flink 1.2.0. == Scope We started to collect features for Flink 1.2.0 in the Flink Release wiki page [1]. I copy the feature list for convenience: - Dynamic Scaling / Key Groups (FLINK-3755 <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> ) - Rescalable Non-Partitioned State (FLIP-8 <https://cwiki.apache.org/confluence/display/FLINK/FLIP-8%3A+Rescalable+Non-Partitioned+State> ) - Backwards compatible savepoints (FLINK-4797 <https://issues.apache.org/jira/browse/FLINK-4797>) - Integrate Flink with Apache Mesos (FLINK-1984 <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQxf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> ) - Secure Data Access (FLINK-3930 <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc <https://docs.google.com/document/d/1-GQB6uVOyoaXGwtqwqLV8BHDxWiMO2WnVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> ) - Queryable State (FLINK-3779 <https://issues.apache.org/jira/browse/FLINK-3779>) - Metrics in Webinterface (FLIP-7 <https://cwiki.apache.org/confluence/display/FLINK/FLIP-7%3A+Expose+metrics+to+WebInterface> ) - Kafka 0.10 support (FLINK-4035 <http://issues.apache.org/jira/browse/FLINK-4035>) - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) - Table API: Scalar Functions (FLINK-3097) Is a feature missing or something too ambitious? == Schedule In my opinion we should try to release 1.2.0 before Christmas. That means, we should have a RC1 by end November / early December and a feature freeze two weeks earlier in mid of November. Does that sound appropriate or should we aim for an earlier or later release? What do others think about the scope and schedule? Please discuss. Best, Fabian [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+and+Feature+Plan |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
I think this sounds very reasonable, +1 to the schedule.
I would definitely add - FLIP-10 (unify checkpoints and savepoints) - FLIP-7 (metrics in web UI) - FLIP-12 (async request operators) Those should be all safe bets, as they are basically done. Let's see what else is in shape until the feature freeze day. On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> wrote: > Hi everybody, > > Flink 1.1.0 was released in August and I think it is time to kick off a > discussion about the schedule and scope of Flink 1.2.0. > > == Scope > > We started to collect features for Flink 1.2.0 in the Flink Release wiki > page [1]. > I copy the feature list for convenience: > > - Dynamic Scaling / Key Groups (FLINK-3755 > <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > ) > - Rescalable Non-Partitioned State (FLIP-8 > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > 8%3A+Rescalable+Non-Partitioned+State> > ) > - Backwards compatible savepoints (FLINK-4797 > <https://issues.apache.org/jira/browse/FLINK-4797>) > - > > Integrate Flink with Apache Mesos (FLINK-1984 > <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > ) > - Secure Data Access (FLINK-3930 > <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > <https://docs.google.com/document/d/1-GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > ) > - Queryable State (FLINK-3779 > <https://issues.apache.org/jira/browse/FLINK-3779>) > - Metrics in Webinterface (FLIP-7 > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > 7%3A+Expose+metrics+to+WebInterface> > ) > - Kafka 0.10 support (FLINK-4035 > <http://issues.apache.org/jira/browse/FLINK-4035>) > - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > - Table API: Scalar Functions (FLINK-3097) > > > Is a feature missing or something too ambitious? > > == Schedule > > In my opinion we should try to release 1.2.0 before Christmas. That means, > we should have a RC1 by end November / early December and a feature freeze > two weeks earlier in mid of November. Does that sound appropriate or should > we aim for an earlier or later release? > > What do others think about the scope and schedule? > Please discuss. > > Best, > Fabian > > [1] > https://cwiki.apache.org/confluence/display/FLINK/ > Flink+Release+and+Feature+Plan > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
+1 the schedule proposed so far.
Do we also want to get in the "Trigger DSL" that we've had brewing for a while now? On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: > I think this sounds very reasonable, +1 to the schedule. > > I would definitely add > - FLIP-10 (unify checkpoints and savepoints) > - FLIP-7 (metrics in web UI) > - FLIP-12 (async request operators) > > Those should be all safe bets, as they are basically done. > > Let's see what else is in shape until the feature freeze day. > > > > On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> wrote: > > > Hi everybody, > > > > Flink 1.1.0 was released in August and I think it is time to kick off a > > discussion about the schedule and scope of Flink 1.2.0. > > > > == Scope > > > > We started to collect features for Flink 1.2.0 in the Flink Release wiki > > page [1]. > > I copy the feature list for convenience: > > > > - Dynamic Scaling / Key Groups (FLINK-3755 > > <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > > <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > > LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > > ) > > - Rescalable Non-Partitioned State (FLIP-8 > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > 8%3A+Rescalable+Non-Partitioned+State> > > ) > > - Backwards compatible savepoints (FLINK-4797 > > <https://issues.apache.org/jira/browse/FLINK-4797>) > > - > > > > Integrate Flink with Apache Mesos (FLINK-1984 > > <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > > <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > > xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > > ) > > - Secure Data Access (FLINK-3930 > > <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > > <https://docs.google.com/document/d/1-GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > > nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > > ) > > - Queryable State (FLINK-3779 > > <https://issues.apache.org/jira/browse/FLINK-3779>) > > - Metrics in Webinterface (FLIP-7 > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > 7%3A+Expose+metrics+to+WebInterface> > > ) > > - Kafka 0.10 support (FLINK-4035 > > <http://issues.apache.org/jira/browse/FLINK-4035>) > > - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > - Table API: Scalar Functions (FLINK-3097) > > > > > > Is a feature missing or something too ambitious? > > > > == Schedule > > > > In my opinion we should try to release 1.2.0 before Christmas. That > means, > > we should have a RC1 by end November / early December and a feature > freeze > > two weeks earlier in mid of November. Does that sound appropriate or > should > > we aim for an earlier or later release? > > > > What do others think about the scope and schedule? > > Please discuss. > > > > Best, > > Fabian > > > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/ > > Flink+Release+and+Feature+Plan > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
+1 for the features to include.
What is the state of the Trigger DSL? How much is left to be done before merging? Cheers, Till On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek <[hidden email]> wrote: > +1 the schedule proposed so far. > > Do we also want to get in the "Trigger DSL" that we've had brewing for a > while now? > > On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: > > > I think this sounds very reasonable, +1 to the schedule. > > > > I would definitely add > > - FLIP-10 (unify checkpoints and savepoints) > > - FLIP-7 (metrics in web UI) > > - FLIP-12 (async request operators) > > > > Those should be all safe bets, as they are basically done. > > > > Let's see what else is in shape until the feature freeze day. > > > > > > > > On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> > wrote: > > > > > Hi everybody, > > > > > > Flink 1.1.0 was released in August and I think it is time to kick off a > > > discussion about the schedule and scope of Flink 1.2.0. > > > > > > == Scope > > > > > > We started to collect features for Flink 1.2.0 in the Flink Release > wiki > > > page [1]. > > > I copy the feature list for convenience: > > > > > > - Dynamic Scaling / Key Groups (FLINK-3755 > > > <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > > > <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > > > LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > > > ) > > > - Rescalable Non-Partitioned State (FLIP-8 > > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > > 8%3A+Rescalable+Non-Partitioned+State> > > > ) > > > - Backwards compatible savepoints (FLINK-4797 > > > <https://issues.apache.org/jira/browse/FLINK-4797>) > > > - > > > > > > Integrate Flink with Apache Mesos (FLINK-1984 > > > <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > > > <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > > > xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > > > ) > > > - Secure Data Access (FLINK-3930 > > > <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > > > <https://docs.google.com/document/d/1- > GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > > > nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > > > ) > > > - Queryable State (FLINK-3779 > > > <https://issues.apache.org/jira/browse/FLINK-3779>) > > > - Metrics in Webinterface (FLIP-7 > > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > > 7%3A+Expose+metrics+to+WebInterface> > > > ) > > > - Kafka 0.10 support (FLINK-4035 > > > <http://issues.apache.org/jira/browse/FLINK-4035>) > > > - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > > - Table API: Scalar Functions (FLINK-3097) > > > > > > > > > Is a feature missing or something too ambitious? > > > > > > == Schedule > > > > > > In my opinion we should try to release 1.2.0 before Christmas. That > > means, > > > we should have a RC1 by end November / early December and a feature > > freeze > > > two weeks earlier in mid of November. Does that sound appropriate or > > should > > > we aim for an earlier or later release? > > > > > > What do others think about the scope and schedule? > > > Please discuss. > > > > > > Best, > > > Fabian > > > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/ > > > Flink+Release+and+Feature+Plan > > > > > > |
|
|
+1 to the schedule
I hope to add 1) Table API: User defined table function (FLINK-4469 <https://issues.apache.org/jira/browse/FLINK-4469>) 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 <https://issues.apache.org/jira/browse/FLINK-4704>) The 1) has been discussed for a long time and a pull request has been on github. The 2) would be a large change, but I hope it could be included in 1.2 release, otherwise it will be more difficult to fix it. - Jark Wu > 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > > +1 for the features to include. > > What is the state of the Trigger DSL? How much is left to be done before > merging? > > Cheers, > Till > > On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek <[hidden email]> > wrote: > >> +1 the schedule proposed so far. >> >> Do we also want to get in the "Trigger DSL" that we've had brewing for a >> while now? >> >> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: >> >>> I think this sounds very reasonable, +1 to the schedule. >>> >>> I would definitely add >>> - FLIP-10 (unify checkpoints and savepoints) >>> - FLIP-7 (metrics in web UI) >>> - FLIP-12 (async request operators) >>> >>> Those should be all safe bets, as they are basically done. >>> >>> Let's see what else is in shape until the feature freeze day. >>> >>> >>> >>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> >> wrote: >>> >>>> Hi everybody, >>>> >>>> Flink 1.1.0 was released in August and I think it is time to kick off a >>>> discussion about the schedule and scope of Flink 1.2.0. >>>> >>>> == Scope >>>> >>>> We started to collect features for Flink 1.2.0 in the Flink Release >> wiki >>>> page [1]. >>>> I copy the feature list for convenience: >>>> >>>> - Dynamic Scaling / Key Groups (FLINK-3755 >>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc >>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- >>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> >>>> ) >>>> - Rescalable Non-Partitioned State (FLIP-8 >>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >>>> 8%3A+Rescalable+Non-Partitioned+State> >>>> ) >>>> - Backwards compatible savepoints (FLINK-4797 >>>> <https://issues.apache.org/jira/browse/FLINK-4797>) >>>> - >>>> >>>> Integrate Flink with Apache Mesos (FLINK-1984 >>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc >>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ >>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> >>>> ) >>>> - Secure Data Access (FLINK-3930 >>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc >>>> <https://docs.google.com/document/d/1- >> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W >>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> >>>> ) >>>> - Queryable State (FLINK-3779 >>>> <https://issues.apache.org/jira/browse/FLINK-3779>) >>>> - Metrics in Webinterface (FLIP-7 >>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >>>> 7%3A+Expose+metrics+to+WebInterface> >>>> ) >>>> - Kafka 0.10 support (FLINK-4035 >>>> <http://issues.apache.org/jira/browse/FLINK-4035>) >>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) >>>> - Table API: Scalar Functions (FLINK-3097) >>>> >>>> >>>> Is a feature missing or something too ambitious? >>>> >>>> == Schedule >>>> >>>> In my opinion we should try to release 1.2.0 before Christmas. That >>> means, >>>> we should have a RC1 by end November / early December and a feature >>> freeze >>>> two weeks earlier in mid of November. Does that sound appropriate or >>> should >>>> we aim for an earlier or later release? >>>> >>>> What do others think about the scope and schedule? >>>> Please discuss. >>>> >>>> Best, >>>> Fabian >>>> >>>> [1] >>>> https://cwiki.apache.org/confluence/display/FLINK/ >>>> Flink+Release+and+Feature+Plan >>>> >>> >> |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
@Till: For the TriggerDSL we have to merge my open PR [1] for refactoring
the WindowOperator/Trigger tests and then refactor the tests of the TriggerDSL to use the new test facilities introduced there. Plus then maybe some fixups. On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> wrote: > +1 to the schedule > > I hope to add > > 1) Table API: User defined table function (FLINK-4469 < > https://issues.apache.org/jira/browse/FLINK-4469>) > 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 < > https://issues.apache.org/jira/browse/FLINK-4704>) > > The 1) has been discussed for a long time and a pull request has been on > github. The 2) would be a large change, but I hope it could be included in > 1.2 release, otherwise it will be more difficult to fix it. > > - Jark Wu > > > 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > > > > +1 for the features to include. > > > > What is the state of the Trigger DSL? How much is left to be done before > > merging? > > > > Cheers, > > Till > > > > On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek <[hidden email]> > > wrote: > > > >> +1 the schedule proposed so far. > >> > >> Do we also want to get in the "Trigger DSL" that we've had brewing for a > >> while now? > >> > >> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: > >> > >>> I think this sounds very reasonable, +1 to the schedule. > >>> > >>> I would definitely add > >>> - FLIP-10 (unify checkpoints and savepoints) > >>> - FLIP-7 (metrics in web UI) > >>> - FLIP-12 (async request operators) > >>> > >>> Those should be all safe bets, as they are basically done. > >>> > >>> Let's see what else is in shape until the feature freeze day. > >>> > >>> > >>> > >>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> > >> wrote: > >>> > >>>> Hi everybody, > >>>> > >>>> Flink 1.1.0 was released in August and I think it is time to kick off > a > >>>> discussion about the schedule and scope of Flink 1.2.0. > >>>> > >>>> == Scope > >>>> > >>>> We started to collect features for Flink 1.2.0 in the Flink Release > >> wiki > >>>> page [1]. > >>>> I copy the feature list for convenience: > >>>> > >>>> - Dynamic Scaling / Key Groups (FLINK-3755 > >>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > >>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > >>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > >>>> ) > >>>> - Rescalable Non-Partitioned State (FLIP-8 > >>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >>>> 8%3A+Rescalable+Non-Partitioned+State> > >>>> ) > >>>> - Backwards compatible savepoints (FLINK-4797 > >>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > >>>> - > >>>> > >>>> Integrate Flink with Apache Mesos (FLINK-1984 > >>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > >>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > >>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > >>>> ) > >>>> - Secure Data Access (FLINK-3930 > >>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > >>>> <https://docs.google.com/document/d/1- > >> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > >>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > >>>> ) > >>>> - Queryable State (FLINK-3779 > >>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > >>>> - Metrics in Webinterface (FLIP-7 > >>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >>>> 7%3A+Expose+metrics+to+WebInterface> > >>>> ) > >>>> - Kafka 0.10 support (FLINK-4035 > >>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > >>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > >>>> - Table API: Scalar Functions (FLINK-3097) > >>>> > >>>> > >>>> Is a feature missing or something too ambitious? > >>>> > >>>> == Schedule > >>>> > >>>> In my opinion we should try to release 1.2.0 before Christmas. That > >>> means, > >>>> we should have a RC1 by end November / early December and a feature > >>> freeze > >>>> two weeks earlier in mid of November. Does that sound appropriate or > >>> should > >>>> we aim for an earlier or later release? > >>>> > >>>> What do others think about the scope and schedule? > >>>> Please discuss. > >>>> > >>>> Best, > >>>> Fabian > >>>> > >>>> [1] > >>>> https://cwiki.apache.org/confluence/display/FLINK/ > >>>> Flink+Release+and+Feature+Plan > >>>> > >>> > >> > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hello,
As Aljoshca said, the TriggerDSL depends on another PR, before it can be merged. In addition, this will change also the whole structure of how we test window/trigger related functionality, so some refactoring will be required. Kostas > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek <[hidden email]> wrote: > > @Till: For the TriggerDSL we have to merge my open PR [1] for refactoring > the WindowOperator/Trigger tests and then refactor the tests of the > TriggerDSL to use the new test facilities introduced there. Plus then maybe > some fixups. > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> wrote: > >> +1 to the schedule >> >> I hope to add >> >> 1) Table API: User defined table function (FLINK-4469 < >> https://issues.apache.org/jira/browse/FLINK-4469>) >> 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 < >> https://issues.apache.org/jira/browse/FLINK-4704>) >> >> The 1) has been discussed for a long time and a pull request has been on >> github. The 2) would be a large change, but I hope it could be included in >> 1.2 release, otherwise it will be more difficult to fix it. >> >> - Jark Wu >> >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： >>> >>> +1 for the features to include. >>> >>> What is the state of the Trigger DSL? How much is left to be done before >>> merging? >>> >>> Cheers, >>> Till >>> >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek <[hidden email]> >>> wrote: >>> >>>> +1 the schedule proposed so far. >>>> >>>> Do we also want to get in the "Trigger DSL" that we've had brewing for a >>>> while now? >>>> >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: >>>> >>>>> I think this sounds very reasonable, +1 to the schedule. >>>>> >>>>> I would definitely add >>>>> - FLIP-10 (unify checkpoints and savepoints) >>>>> - FLIP-7 (metrics in web UI) >>>>> - FLIP-12 (async request operators) >>>>> >>>>> Those should be all safe bets, as they are basically done. >>>>> >>>>> Let's see what else is in shape until the feature freeze day. >>>>> >>>>> >>>>> >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> >>>> wrote: >>>>> >>>>>> Hi everybody, >>>>>> >>>>>> Flink 1.1.0 was released in August and I think it is time to kick off >> a >>>>>> discussion about the schedule and scope of Flink 1.2.0. >>>>>> >>>>>> == Scope >>>>>> >>>>>> We started to collect features for Flink 1.2.0 in the Flink Release >>>> wiki >>>>>> page [1]. >>>>>> I copy the feature list for convenience: >>>>>> >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> >>>>>> ) >>>>>> - Rescalable Non-Partitioned State (FLIP-8 >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >>>>>> 8%3A+Rescalable+Non-Partitioned+State> >>>>>> ) >>>>>> - Backwards compatible savepoints (FLINK-4797 >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) >>>>>> - >>>>>> >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc >>>>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> >>>>>> ) >>>>>> - Secure Data Access (FLINK-3930 >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc >>>>>> <https://docs.google.com/document/d/1- >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> >>>>>> ) >>>>>> - Queryable State (FLINK-3779 >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) >>>>>> - Metrics in Webinterface (FLIP-7 >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >>>>>> 7%3A+Expose+metrics+to+WebInterface> >>>>>> ) >>>>>> - Kafka 0.10 support (FLINK-4035 >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) >>>>>> - Table API: Scalar Functions (FLINK-3097) >>>>>> >>>>>> >>>>>> Is a feature missing or something too ambitious? >>>>>> >>>>>> == Schedule >>>>>> >>>>>> In my opinion we should try to release 1.2.0 before Christmas. That >>>>> means, >>>>>> we should have a RC1 by end November / early December and a feature >>>>> freeze >>>>>> two weeks earlier in mid of November. Does that sound appropriate or >>>>> should >>>>>> we aim for an earlier or later release? >>>>>> >>>>>> What do others think about the scope and schedule? >>>>>> Please discuss. >>>>>> >>>>>> Best, >>>>>> Fabian >>>>>> >>>>>> [1] >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ >>>>>> Flink+Release+and+Feature+Plan >>>>>> >>>>> >>>> >> >> |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi Jark,
FLINK-4469: I agree, UDTFs would be a great feature. I don't think we should make it a blocker but I will help to get it the master before the feature freeze. FLINK-4704: Regarding the renaming of the namespace, we have two positive votes in JIRA, but we should also start a discussion on the dev and user list to collect some more input. The actual change should be simple. @Everybody: I created a wiki page for the 1.2 release [1] and added all features discussed in this thread so far. Please have a look at the page and update it if you know the JIRAs and/or state of the features. Cheers, Fabian [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release 2016-10-28 15:28 GMT+02:00 Kostas Kloudas <[hidden email]>: > Hello, > > As Aljoshca said, the TriggerDSL depends on another PR, before it can be > merged. > In addition, this will change also the whole structure of how we test > window/trigger > related functionality, so some refactoring will be required. > > Kostas > > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek <[hidden email]> > wrote: > > > > @Till: For the TriggerDSL we have to merge my open PR [1] for refactoring > > the WindowOperator/Trigger tests and then refactor the tests of the > > TriggerDSL to use the new test facilities introduced there. Plus then > maybe > > some fixups. > > > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> wrote: > > > >> +1 to the schedule > >> > >> I hope to add > >> > >> 1) Table API: User defined table function (FLINK-4469 < > >> https://issues.apache.org/jira/browse/FLINK-4469>) > >> 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 < > >> https://issues.apache.org/jira/browse/FLINK-4704>) > >> > >> The 1) has been discussed for a long time and a pull request has been on > >> github. The 2) would be a large change, but I hope it could be included > in > >> 1.2 release, otherwise it will be more difficult to fix it. > >> > >> - Jark Wu > >> > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > >>> > >>> +1 for the features to include. > >>> > >>> What is the state of the Trigger DSL? How much is left to be done > before > >>> merging? > >>> > >>> Cheers, > >>> Till > >>> > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek <[hidden email] > > > >>> wrote: > >>> > >>>> +1 the schedule proposed so far. > >>>> > >>>> Do we also want to get in the "Trigger DSL" that we've had brewing > for a > >>>> while now? > >>>> > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: > >>>> > >>>>> I think this sounds very reasonable, +1 to the schedule. > >>>>> > >>>>> I would definitely add > >>>>> - FLIP-10 (unify checkpoints and savepoints) > >>>>> - FLIP-7 (metrics in web UI) > >>>>> - FLIP-12 (async request operators) > >>>>> > >>>>> Those should be all safe bets, as they are basically done. > >>>>> > >>>>> Let's see what else is in shape until the feature freeze day. > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> > >>>> wrote: > >>>>> > >>>>>> Hi everybody, > >>>>>> > >>>>>> Flink 1.1.0 was released in August and I think it is time to kick > off > >> a > >>>>>> discussion about the schedule and scope of Flink 1.2.0. > >>>>>> > >>>>>> == Scope > >>>>>> > >>>>>> We started to collect features for Flink 1.2.0 in the Flink Release > >>>> wiki > >>>>>> page [1]. > >>>>>> I copy the feature list for convenience: > >>>>>> > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > >>>>>> ) > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> > >>>>>> ) > >>>>>> - Backwards compatible savepoints (FLINK-4797 > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > >>>>>> - > >>>>>> > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > >>>>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > >>>>>> ) > >>>>>> - Secure Data Access (FLINK-3930 > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > >>>>>> <https://docs.google.com/document/d/1- > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > >>>>>> ) > >>>>>> - Queryable State (FLINK-3779 > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > >>>>>> - Metrics in Webinterface (FLIP-7 > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >>>>>> 7%3A+Expose+metrics+to+WebInterface> > >>>>>> ) > >>>>>> - Kafka 0.10 support (FLINK-4035 > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > >>>>>> - Table API: Scalar Functions (FLINK-3097) > >>>>>> > >>>>>> > >>>>>> Is a feature missing or something too ambitious? > >>>>>> > >>>>>> == Schedule > >>>>>> > >>>>>> In my opinion we should try to release 1.2.0 before Christmas. That > >>>>> means, > >>>>>> we should have a RC1 by end November / early December and a feature > >>>>> freeze > >>>>>> two weeks earlier in mid of November. Does that sound appropriate or > >>>>> should > >>>>>> we aim for an earlier or later release? > >>>>>> > >>>>>> What do others think about the scope and schedule? > >>>>>> Please discuss. > >>>>>> > >>>>>> Best, > >>>>>> Fabian > >>>>>> > >>>>>> [1] > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ > >>>>>> Flink+Release+and+Feature+Plan > >>>>>> > >>>>> > >>>> > >> > >> > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi to all,
for the 1.2 release do you think it will be possible to include also a PR to implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491>? The PR is waiting for a review at https://github.com/apache/flink/pull/2790. Best, Flavio On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske <[hidden email]> wrote: > Hi Jark, > > FLINK-4469: I agree, UDTFs would be a great feature. I don't think we > should make it a blocker but I will help to get it the master before the > feature freeze. > FLINK-4704: Regarding the renaming of the namespace, we have two positive > votes in JIRA, but we should also start a discussion on the dev and user > list to collect some more input. The actual change should be simple. > > @Everybody: I created a wiki page for the 1.2 release [1] and added all > features discussed in this thread so far. > Please have a look at the page and update it if you know the JIRAs and/or > state of the features. > > Cheers, Fabian > > [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > > 2016-10-28 15:28 GMT+02:00 Kostas Kloudas <[hidden email]>: > > > Hello, > > > > As Aljoshca said, the TriggerDSL depends on another PR, before it can be > > merged. > > In addition, this will change also the whole structure of how we test > > window/trigger > > related functionality, so some refactoring will be required. > > > > Kostas > > > > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek <[hidden email]> > > wrote: > > > > > > @Till: For the TriggerDSL we have to merge my open PR [1] for > refactoring > > > the WindowOperator/Trigger tests and then refactor the tests of the > > > TriggerDSL to use the new test facilities introduced there. Plus then > > maybe > > > some fixups. > > > > > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> > wrote: > > > > > >> +1 to the schedule > > >> > > >> I hope to add > > >> > > >> 1) Table API: User defined table function (FLINK-4469 < > > >> https://issues.apache.org/jira/browse/FLINK-4469>) > > >> 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 < > > >> https://issues.apache.org/jira/browse/FLINK-4704>) > > >> > > >> The 1) has been discussed for a long time and a pull request has been > on > > >> github. The 2) would be a large change, but I hope it could be > included > > in > > >> 1.2 release, otherwise it will be more difficult to fix it. > > >> > > >> - Jark Wu > > >> > > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > > >>> > > >>> +1 for the features to include. > > >>> > > >>> What is the state of the Trigger DSL? How much is left to be done > > before > > >>> merging? > > >>> > > >>> Cheers, > > >>> Till > > >>> > > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek < > [hidden email] > > > > > >>> wrote: > > >>> > > >>>> +1 the schedule proposed so far. > > >>>> > > >>>> Do we also want to get in the "Trigger DSL" that we've had brewing > > for a > > >>>> while now? > > >>>> > > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> wrote: > > >>>> > > >>>>> I think this sounds very reasonable, +1 to the schedule. > > >>>>> > > >>>>> I would definitely add > > >>>>> - FLIP-10 (unify checkpoints and savepoints) > > >>>>> - FLIP-7 (metrics in web UI) > > >>>>> - FLIP-12 (async request operators) > > >>>>> > > >>>>> Those should be all safe bets, as they are basically done. > > >>>>> > > >>>>> Let's see what else is in shape until the feature freeze day. > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email]> > > >>>> wrote: > > >>>>> > > >>>>>> Hi everybody, > > >>>>>> > > >>>>>> Flink 1.1.0 was released in August and I think it is time to kick > > off > > >> a > > >>>>>> discussion about the schedule and scope of Flink 1.2.0. > > >>>>>> > > >>>>>> == Scope > > >>>>>> > > >>>>>> We started to collect features for Flink 1.2.0 in the Flink > Release > > >>>> wiki > > >>>>>> page [1]. > > >>>>>> I copy the feature list for convenience: > > >>>>>> > > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 > > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc > > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > > >>>>>> ) > > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 > > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> > > >>>>>> ) > > >>>>>> - Backwards compatible savepoints (FLINK-4797 > > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > > >>>>>> - > > >>>>>> > > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 > > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc > > >>>>>> <https://docs.google.com/document/d/ > 1WItafBmGbjlaBbP8Of5PAFOH9GUJQ > > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > > >>>>>> ) > > >>>>>> - Secure Data Access (FLINK-3930 > > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc > > >>>>>> <https://docs.google.com/document/d/1- > > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > > >>>>>> ) > > >>>>>> - Queryable State (FLINK-3779 > > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > > >>>>>> - Metrics in Webinterface (FLIP-7 > > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > >>>>>> 7%3A+Expose+metrics+to+WebInterface> > > >>>>>> ) > > >>>>>> - Kafka 0.10 support (FLINK-4035 > > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > >>>>>> - Table API: Scalar Functions (FLINK-3097) > > >>>>>> > > >>>>>> > > >>>>>> Is a feature missing or something too ambitious? > > >>>>>> > > >>>>>> == Schedule > > >>>>>> > > >>>>>> In my opinion we should try to release 1.2.0 before Christmas. > That > > >>>>> means, > > >>>>>> we should have a RC1 by end November / early December and a > feature > > >>>>> freeze > > >>>>>> two weeks earlier in mid of November. Does that sound appropriate > or > > >>>>> should > > >>>>>> we aim for an earlier or later release? > > >>>>>> > > >>>>>> What do others think about the scope and schedule? > > >>>>>> Please discuss. > > >>>>>> > > >>>>>> Best, > > >>>>>> Fabian > > >>>>>> > > >>>>>> [1] > > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ > > >>>>>> Flink+Release+and+Feature+Plan > > >>>>>> > > >>>>> > > >>>> > > >> > > >> > > > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi to all,
we've completed the PR #2790 <https://github.com/apache/flink/pull/2790> to implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491>: we've added the comment that @zentol asked for a week ago and we also fixed a potential annoying problem we faced during indexing (if a malformed document generates an error during indexing, the close() method throws an exception forcing the entire job to fail, killing other threads that are still indexing). I hope this PR could hep other people working with Elasticsearch, right from the next release ;) On Mon, Nov 14, 2016 at 2:47 PM, Flavio Pompermaier <[hidden email]> wrote: > Hi to all, > for the 1.2 release do you think it will be possible to include also a PR > to implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491> > ? > The PR is waiting for a review at https://github.com/apache/ > flink/pull/2790. > > Best, > Flavio > > On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske <[hidden email]> wrote: > >> Hi Jark, >> >> FLINK-4469: I agree, UDTFs would be a great feature. I don't think we >> should make it a blocker but I will help to get it the master before the >> feature freeze. >> FLINK-4704: Regarding the renaming of the namespace, we have two positive >> votes in JIRA, but we should also start a discussion on the dev and user >> list to collect some more input. The actual change should be simple. >> >> @Everybody: I created a wiki page for the 1.2 release [1] and added all >> features discussed in this thread so far. >> Please have a look at the page and update it if you know the JIRAs and/or >> state of the features. >> >> Cheers, Fabian >> >> [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release >> >> 2016-10-28 15:28 GMT+02:00 Kostas Kloudas <[hidden email]>: >> >> > Hello, >> > >> > As Aljoshca said, the TriggerDSL depends on another PR, before it can be >> > merged. >> > In addition, this will change also the whole structure of how we test >> > window/trigger >> > related functionality, so some refactoring will be required. >> > >> > Kostas >> > >> > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek <[hidden email]> >> > wrote: >> > > >> > > @Till: For the TriggerDSL we have to merge my open PR [1] for >> refactoring >> > > the WindowOperator/Trigger tests and then refactor the tests of the >> > > TriggerDSL to use the new test facilities introduced there. Plus then >> > maybe >> > > some fixups. >> > > >> > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> >> wrote: >> > > >> > >> +1 to the schedule >> > >> >> > >> I hope to add >> > >> >> > >> 1) Table API: User defined table function (FLINK-4469 < >> > >> https://issues.apache.org/jira/browse/FLINK-4469>) >> > >> 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 < >> > >> https://issues.apache.org/jira/browse/FLINK-4704>) >> > >> >> > >> The 1) has been discussed for a long time and a pull request has >> been on >> > >> github. The 2) would be a large change, but I hope it could be >> included >> > in >> > >> 1.2 release, otherwise it will be more difficult to fix it. >> > >> >> > >> - Jark Wu >> > >> >> > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： >> > >>> >> > >>> +1 for the features to include. >> > >>> >> > >>> What is the state of the Trigger DSL? How much is left to be done >> > before >> > >>> merging? >> > >>> >> > >>> Cheers, >> > >>> Till >> > >>> >> > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek < >> [hidden email] >> > > >> > >>> wrote: >> > >>> >> > >>>> +1 the schedule proposed so far. >> > >>>> >> > >>>> Do we also want to get in the "Trigger DSL" that we've had brewing >> > for a >> > >>>> while now? >> > >>>> >> > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> >> wrote: >> > >>>> >> > >>>>> I think this sounds very reasonable, +1 to the schedule. >> > >>>>> >> > >>>>> I would definitely add >> > >>>>> - FLIP-10 (unify checkpoints and savepoints) >> > >>>>> - FLIP-7 (metrics in web UI) >> > >>>>> - FLIP-12 (async request operators) >> > >>>>> >> > >>>>> Those should be all safe bets, as they are basically done. >> > >>>>> >> > >>>>> Let's see what else is in shape until the feature freeze day. >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske <[hidden email] >> > >> > >>>> wrote: >> > >>>>> >> > >>>>>> Hi everybody, >> > >>>>>> >> > >>>>>> Flink 1.1.0 was released in August and I think it is time to kick >> > off >> > >> a >> > >>>>>> discussion about the schedule and scope of Flink 1.2.0. >> > >>>>>> >> > >>>>>> == Scope >> > >>>>>> >> > >>>>>> We started to collect features for Flink 1.2.0 in the Flink >> Release >> > >>>> wiki >> > >>>>>> page [1]. >> > >>>>>> I copy the feature list for convenience: >> > >>>>>> >> > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design Doc >> > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- >> > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> >> > >>>>>> ) >> > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >> > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> >> > >>>>>> ) >> > >>>>>> - Backwards compatible savepoints (FLINK-4797 >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) >> > >>>>>> - >> > >>>>>> >> > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design Doc >> > >>>>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFO >> H9GUJQ >> > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> >> > >>>>>> ) >> > >>>>>> - Secure Data Access (FLINK-3930 >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design Doc >> > >>>>>> <https://docs.google.com/document/d/1- >> > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W >> > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> >> > >>>>>> ) >> > >>>>>> - Queryable State (FLINK-3779 >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) >> > >>>>>> - Metrics in Webinterface (FLIP-7 >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- >> > >>>>>> 7%3A+Expose+metrics+to+WebInterface> >> > >>>>>> ) >> > >>>>>> - Kafka 0.10 support (FLINK-4035 >> > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) >> > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) >> > >>>>>> - Table API: Scalar Functions (FLINK-3097) >> > >>>>>> >> > >>>>>> >> > >>>>>> Is a feature missing or something too ambitious? >> > >>>>>> >> > >>>>>> == Schedule >> > >>>>>> >> > >>>>>> In my opinion we should try to release 1.2.0 before Christmas. >> That >> > >>>>> means, >> > >>>>>> we should have a RC1 by end November / early December and a >> feature >> > >>>>> freeze >> > >>>>>> two weeks earlier in mid of November. Does that sound >> appropriate or >> > >>>>> should >> > >>>>>> we aim for an earlier or later release? >> > >>>>>> >> > >>>>>> What do others think about the scope and schedule? >> > >>>>>> Please discuss. >> > >>>>>> >> > >>>>>> Best, >> > >>>>>> Fabian >> > >>>>>> >> > >>>>>> [1] >> > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ >> > >>>>>> Flink+Release+and+Feature+Plan >> > >>>>>> >> > >>>>> >> > >>>> >> > >> >> > >> >> > >> > >> > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi,
I would like to reactivate this old email thread. The initial plan was to release 1.2 before Christmas, have RC1 by end of Nov / December and a feature freeze mid-November. Today (December 5), we did not yet do the feature freeze. I would like to get an overview again of the remaining blockers and what we want to include into the release. From the issues Fabian posted in the beginning, they have the following status: - RESOLVED dynamic Scaling / Key Groups (FLINK-3755) - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379) - UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797) - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984) - UNRESOLVED Secure Data Access (FLINK-3930) - RESOLVED Queryable State (FLINK-3779) - RESOLVED Metrics in Webinterface (FLINK-4389) - RESOLVED Kafka 0.10 support (FLINK-4035) - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11) - RESOLVED Table API: Scalar Functions (FLINK-3097) Added by Stephan: - UNRESOLVED Provide support for asynchronous operations over streams (FLINK-4391) - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) I've also reflected the status in the Wiki: https://cwiki.apache.org/confluence/display/FLINK/1.2+Release @Flavio: I assume PR #2790 is resolved and not a release blocker. Given that we are already behind the schedule, I'm wondering what we can do regarding the remaining UNRESOLVED issues. I would also be willing to release some of them as undocumented, work-in-progress features. We have so many good things like dynamic scaling or Mesos in the release that we should get it out rather soon. It would be nice if those committers / contributors working on the UNRESOLVED issues could give a more detailed status update what they think release-wise. Also, is there anything else we need to consider before forking off the release-1.2 branch from master? Once we've collected the detailed status updates from the last unresolved features, we can discuss a new date for the feature freeze. I'll then also create a RC0 testing release candidate without a VOTE, to have a starting point for the first big release testing. (And yes, I'm volunteering to be the release manager for the 1.2 release ;) ) Regards, Robert On Mon, Nov 21, 2016 at 3:03 PM, Flavio Pompermaier <[hidden email]> wrote: > Hi to all, > we've completed the PR #2790 <https://github.com/apache/flink/pull/2790> > to > implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491>: > we've added the comment that @zentol asked for a week ago and we also fixed > a potential annoying problem we faced during indexing (if a malformed > document generates an error during indexing, the close() method throws an > exception forcing the entire job to fail, killing other threads that are > still indexing). I hope this PR could hep other people working with > Elasticsearch, right from the next release ;) > > On Mon, Nov 14, 2016 at 2:47 PM, Flavio Pompermaier <[hidden email]> > wrote: > > > Hi to all, > > for the 1.2 release do you think it will be possible to include also a PR > > to implement FLINK-4491 <https://issues.apache.org/ > jira/browse/FLINK-4491> > > ? > > The PR is waiting for a review at https://github.com/apache/ > > flink/pull/2790. > > > > Best, > > Flavio > > > > On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske <[hidden email]> > wrote: > > > >> Hi Jark, > >> > >> FLINK-4469: I agree, UDTFs would be a great feature. I don't think we > >> should make it a blocker but I will help to get it the master before the > >> feature freeze. > >> FLINK-4704: Regarding the renaming of the namespace, we have two > positive > >> votes in JIRA, but we should also start a discussion on the dev and user > >> list to collect some more input. The actual change should be simple. > >> > >> @Everybody: I created a wiki page for the 1.2 release [1] and added all > >> features discussed in this thread so far. > >> Please have a look at the page and update it if you know the JIRAs > and/or > >> state of the features. > >> > >> Cheers, Fabian > >> > >> [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > >> > >> 2016-10-28 15:28 GMT+02:00 Kostas Kloudas <[hidden email] > >: > >> > >> > Hello, > >> > > >> > As Aljoshca said, the TriggerDSL depends on another PR, before it can > be > >> > merged. > >> > In addition, this will change also the whole structure of how we test > >> > window/trigger > >> > related functionality, so some refactoring will be required. > >> > > >> > Kostas > >> > > >> > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek <[hidden email]> > >> > wrote: > >> > > > >> > > @Till: For the TriggerDSL we have to merge my open PR [1] for > >> refactoring > >> > > the WindowOperator/Trigger tests and then refactor the tests of the > >> > > TriggerDSL to use the new test facilities introduced there. Plus > then > >> > maybe > >> > > some fixups. > >> > > > >> > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> > >> wrote: > >> > > > >> > >> +1 to the schedule > >> > >> > >> > >> I hope to add > >> > >> > >> > >> 1) Table API: User defined table function (FLINK-4469 < > >> > >> https://issues.apache.org/jira/browse/FLINK-4469>) > >> > >> 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 > < > >> > >> https://issues.apache.org/jira/browse/FLINK-4704>) > >> > >> > >> > >> The 1) has been discussed for a long time and a pull request has > >> been on > >> > >> github. The 2) would be a large change, but I hope it could be > >> included > >> > in > >> > >> 1.2 release, otherwise it will be more difficult to fix it. > >> > >> > >> > >> - Jark Wu > >> > >> > >> > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > >> > >>> > >> > >>> +1 for the features to include. > >> > >>> > >> > >>> What is the state of the Trigger DSL? How much is left to be done > >> > before > >> > >>> merging? > >> > >>> > >> > >>> Cheers, > >> > >>> Till > >> > >>> > >> > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek < > >> [hidden email] > >> > > > >> > >>> wrote: > >> > >>> > >> > >>>> +1 the schedule proposed so far. > >> > >>>> > >> > >>>> Do we also want to get in the "Trigger DSL" that we've had > brewing > >> > for a > >> > >>>> while now? > >> > >>>> > >> > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> > >> wrote: > >> > >>>> > >> > >>>>> I think this sounds very reasonable, +1 to the schedule. > >> > >>>>> > >> > >>>>> I would definitely add > >> > >>>>> - FLIP-10 (unify checkpoints and savepoints) > >> > >>>>> - FLIP-7 (metrics in web UI) > >> > >>>>> - FLIP-12 (async request operators) > >> > >>>>> > >> > >>>>> Those should be all safe bets, as they are basically done. > >> > >>>>> > >> > >>>>> Let's see what else is in shape until the feature freeze day. > >> > >>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske < > [hidden email] > >> > > >> > >>>> wrote: > >> > >>>>> > >> > >>>>>> Hi everybody, > >> > >>>>>> > >> > >>>>>> Flink 1.1.0 was released in August and I think it is time to > kick > >> > off > >> > >> a > >> > >>>>>> discussion about the schedule and scope of Flink 1.2.0. > >> > >>>>>> > >> > >>>>>> == Scope > >> > >>>>>> > >> > >>>>>> We started to collect features for Flink 1.2.0 in the Flink > >> Release > >> > >>>> wiki > >> > >>>>>> page [1]. > >> > >>>>>> I copy the feature list for convenience: > >> > >>>>>> > >> > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design > Doc > >> > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > >> > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > >> > >>>>>> ) > >> > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >> > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> > >> > >>>>>> ) > >> > >>>>>> - Backwards compatible savepoints (FLINK-4797 > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > >> > >>>>>> - > >> > >>>>>> > >> > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design > Doc > >> > >>>>>> <https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFO > >> H9GUJQ > >> > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > >> > >>>>>> ) > >> > >>>>>> - Secure Data Access (FLINK-3930 > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design > Doc > >> > >>>>>> <https://docs.google.com/document/d/1- > >> > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > >> > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > >> > >>>>>> ) > >> > >>>>>> - Queryable State (FLINK-3779 > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > >> > >>>>>> - Metrics in Webinterface (FLIP-7 > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > >> > >>>>>> 7%3A+Expose+metrics+to+WebInterface> > >> > >>>>>> ) > >> > >>>>>> - Kafka 0.10 support (FLINK-4035 > >> > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > >> > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > >> > >>>>>> - Table API: Scalar Functions (FLINK-3097) > >> > >>>>>> > >> > >>>>>> > >> > >>>>>> Is a feature missing or something too ambitious? > >> > >>>>>> > >> > >>>>>> == Schedule > >> > >>>>>> > >> > >>>>>> In my opinion we should try to release 1.2.0 before Christmas. > >> That > >> > >>>>> means, > >> > >>>>>> we should have a RC1 by end November / early December and a > >> feature > >> > >>>>> freeze > >> > >>>>>> two weeks earlier in mid of November. Does that sound > >> appropriate or > >> > >>>>> should > >> > >>>>>> we aim for an earlier or later release? > >> > >>>>>> > >> > >>>>>> What do others think about the scope and schedule? > >> > >>>>>> Please discuss. > >> > >>>>>> > >> > >>>>>> Best, > >> > >>>>>> Fabian > >> > >>>>>> > >> > >>>>>> [1] > >> > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ > >> > >>>>>> Flink+Release+and+Feature+Plan > >> > >>>>>> > >> > >>>>> > >> > >>>> > >> > >> > >> > >> > >> > > >> > > >> > > > > > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi Robert,
our pull request was closed in favor of https://github.com/apache/flink/pull/2861. However that PR still need some review I think (I also added some comment in the JIRA issue[1]). it will be great to have that fix in the next release :) [1] https://issues.apache.org/jira/browse/FLINK-5122 Best, Flavio On Mon, Dec 5, 2016 at 6:28 PM, Robert Metzger <[hidden email]> wrote: > Hi, > > I would like to reactivate this old email thread. The initial plan was to > release 1.2 before Christmas, have RC1 by end of Nov / December and a > feature freeze mid-November. > Today (December 5), we did not yet do the feature freeze. I would like to > get an overview again of the remaining blockers and what we want to include > into the release. > > From the issues Fabian posted in the beginning, they have the following > status: > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755) > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379) > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797) > - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984) > - UNRESOLVED Secure Data Access (FLINK-3930) > - RESOLVED Queryable State (FLINK-3779) > - RESOLVED Metrics in Webinterface (FLINK-4389) > - RESOLVED Kafka 0.10 support (FLINK-4035) > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > - RESOLVED Table API: Scalar Functions (FLINK-3097) > Added by Stephan: > - UNRESOLVED Provide support for asynchronous operations over streams > (FLINK-4391) > - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) > > I've also reflected the status in the Wiki: > https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > > @Flavio: I assume PR #2790 is resolved and not a release blocker. > > Given that we are already behind the schedule, I'm wondering what we can do > regarding the remaining UNRESOLVED issues. > I would also be willing to release some of them as undocumented, > work-in-progress features. We have so many good things like dynamic scaling > or Mesos in the release that we should get it out rather soon. > > It would be nice if those committers / contributors working on the > UNRESOLVED issues could give a more detailed status update what they think > release-wise. > Also, is there anything else we need to consider before forking off the > release-1.2 branch from master? > > Once we've collected the detailed status updates from the last unresolved > features, we can discuss a new date for the feature freeze. > I'll then also create a RC0 testing release candidate without a VOTE, to > have a starting point for the first big release testing. (And yes, I'm > volunteering to be the release manager for the 1.2 release ;) ) > > > Regards, > Robert > > > > > On Mon, Nov 21, 2016 at 3:03 PM, Flavio Pompermaier <[hidden email]> > wrote: > > > Hi to all, > > we've completed the PR #2790 <https://github.com/apache/flink/pull/2790> > > to > > implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491>: > > we've added the comment that @zentol asked for a week ago and we also > fixed > > a potential annoying problem we faced during indexing (if a malformed > > document generates an error during indexing, the close() method throws an > > exception forcing the entire job to fail, killing other threads that are > > still indexing). I hope this PR could hep other people working with > > Elasticsearch, right from the next release ;) > > > > On Mon, Nov 14, 2016 at 2:47 PM, Flavio Pompermaier < > [hidden email]> > > wrote: > > > > > Hi to all, > > > for the 1.2 release do you think it will be possible to include also a > PR > > > to implement FLINK-4491 <https://issues.apache.org/ > > jira/browse/FLINK-4491> > > > ? > > > The PR is waiting for a review at https://github.com/apache/ > > > flink/pull/2790. > > > > > > Best, > > > Flavio > > > > > > On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske <[hidden email]> > > wrote: > > > > > >> Hi Jark, > > >> > > >> FLINK-4469: I agree, UDTFs would be a great feature. I don't think we > > >> should make it a blocker but I will help to get it the master before > the > > >> feature freeze. > > >> FLINK-4704: Regarding the renaming of the namespace, we have two > > positive > > >> votes in JIRA, but we should also start a discussion on the dev and > user > > >> list to collect some more input. The actual change should be simple. > > >> > > >> @Everybody: I created a wiki page for the 1.2 release [1] and added > all > > >> features discussed in this thread so far. > > >> Please have a look at the page and update it if you know the JIRAs > > and/or > > >> state of the features. > > >> > > >> Cheers, Fabian > > >> > > >> [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > > >> > > >> 2016-10-28 15:28 GMT+02:00 Kostas Kloudas < > [hidden email] > > >: > > >> > > >> > Hello, > > >> > > > >> > As Aljoshca said, the TriggerDSL depends on another PR, before it > can > > be > > >> > merged. > > >> > In addition, this will change also the whole structure of how we > test > > >> > window/trigger > > >> > related functionality, so some refactoring will be required. > > >> > > > >> > Kostas > > >> > > > >> > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek < > [hidden email]> > > >> > wrote: > > >> > > > > >> > > @Till: For the TriggerDSL we have to merge my open PR [1] for > > >> refactoring > > >> > > the WindowOperator/Trigger tests and then refactor the tests of > the > > >> > > TriggerDSL to use the new test facilities introduced there. Plus > > then > > >> > maybe > > >> > > some fixups. > > >> > > > > >> > > On Fri, 28 Oct 2016 at 11:17 Jark Wu <[hidden email]> > > >> wrote: > > >> > > > > >> > >> +1 to the schedule > > >> > >> > > >> > >> I hope to add > > >> > >> > > >> > >> 1) Table API: User defined table function (FLINK-4469 < > > >> > >> https://issues.apache.org/jira/browse/FLINK-4469>) > > >> > >> 2) Table API: Move Table API to org.apache.flink.table > (FLINK-4704 > > < > > >> > >> https://issues.apache.org/jira/browse/FLINK-4704>) > > >> > >> > > >> > >> The 1) has been discussed for a long time and a pull request has > > >> been on > > >> > >> github. The 2) would be a large change, but I hope it could be > > >> included > > >> > in > > >> > >> 1.2 release, otherwise it will be more difficult to fix it. > > >> > >> > > >> > >> - Jark Wu > > >> > >> > > >> > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > > >> > >>> > > >> > >>> +1 for the features to include. > > >> > >>> > > >> > >>> What is the state of the Trigger DSL? How much is left to be > done > > >> > before > > >> > >>> merging? > > >> > >>> > > >> > >>> Cheers, > > >> > >>> Till > > >> > >>> > > >> > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek < > > >> [hidden email] > > >> > > > > >> > >>> wrote: > > >> > >>> > > >> > >>>> +1 the schedule proposed so far. > > >> > >>>> > > >> > >>>> Do we also want to get in the "Trigger DSL" that we've had > > brewing > > >> > for a > > >> > >>>> while now? > > >> > >>>> > > >> > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> > > >> wrote: > > >> > >>>> > > >> > >>>>> I think this sounds very reasonable, +1 to the schedule. > > >> > >>>>> > > >> > >>>>> I would definitely add > > >> > >>>>> - FLIP-10 (unify checkpoints and savepoints) > > >> > >>>>> - FLIP-7 (metrics in web UI) > > >> > >>>>> - FLIP-12 (async request operators) > > >> > >>>>> > > >> > >>>>> Those should be all safe bets, as they are basically done. > > >> > >>>>> > > >> > >>>>> Let's see what else is in shape until the feature freeze day. > > >> > >>>>> > > >> > >>>>> > > >> > >>>>> > > >> > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske < > > [hidden email] > > >> > > > >> > >>>> wrote: > > >> > >>>>> > > >> > >>>>>> Hi everybody, > > >> > >>>>>> > > >> > >>>>>> Flink 1.1.0 was released in August and I think it is time to > > kick > > >> > off > > >> > >> a > > >> > >>>>>> discussion about the schedule and scope of Flink 1.2.0. > > >> > >>>>>> > > >> > >>>>>> == Scope > > >> > >>>>>> > > >> > >>>>>> We started to collect features for Flink 1.2.0 in the Flink > > >> Release > > >> > >>>> wiki > > >> > >>>>>> page [1]. > > >> > >>>>>> I copy the feature list for convenience: > > >> > >>>>>> > > >> > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) (Design > > Doc > > >> > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > > >> > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > > >> > >>>>>> ) > > >> > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 > > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > >> > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> > > >> > >>>>>> ) > > >> > >>>>>> - Backwards compatible savepoints (FLINK-4797 > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > > >> > >>>>>> - > > >> > >>>>>> > > >> > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) (Design > > Doc > > >> > >>>>>> <https://docs.google.com/document/d/ > 1WItafBmGbjlaBbP8Of5PAFO > > >> H9GUJQ > > >> > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > > >> > >>>>>> ) > > >> > >>>>>> - Secure Data Access (FLINK-3930 > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) (Design > > Doc > > >> > >>>>>> <https://docs.google.com/document/d/1- > > >> > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > > >> > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > > >> > >>>>>> ) > > >> > >>>>>> - Queryable State (FLINK-3779 > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > > >> > >>>>>> - Metrics in Webinterface (FLIP-7 > > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > >> > >>>>>> 7%3A+Expose+metrics+to+WebInterface> > > >> > >>>>>> ) > > >> > >>>>>> - Kafka 0.10 support (FLINK-4035 > > >> > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > > >> > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > >> > >>>>>> - Table API: Scalar Functions (FLINK-3097) > > >> > >>>>>> > > >> > >>>>>> > > >> > >>>>>> Is a feature missing or something too ambitious? > > >> > >>>>>> > > >> > >>>>>> == Schedule > > >> > >>>>>> > > >> > >>>>>> In my opinion we should try to release 1.2.0 before > Christmas. > > >> That > > >> > >>>>> means, > > >> > >>>>>> we should have a RC1 by end November / early December and a > > >> feature > > >> > >>>>> freeze > > >> > >>>>>> two weeks earlier in mid of November. Does that sound > > >> appropriate or > > >> > >>>>> should > > >> > >>>>>> we aim for an earlier or later release? > > >> > >>>>>> > > >> > >>>>>> What do others think about the scope and schedule? > > >> > >>>>>> Please discuss. > > >> > >>>>>> > > >> > >>>>>> Best, > > >> > >>>>>> Fabian > > >> > >>>>>> > > >> > >>>>>> [1] > > >> > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ > > >> > >>>>>> Flink+Release+and+Feature+Plan > > >> > >>>>>> > > >> > >>>>> > > >> > >>>> > > >> > >> > > >> > >> > > >> > > > >> > > > >> > > > > > > > > > > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Thanks Robert for reactivating the discussion.
There are a few Table API related refactorings that I would like to have in 1.2 because they are API breaking: - FLINK-4704 Clean up the packages of the Table API and move it to the org.apache.flink.table package space. This is an API breaking change that we should do rather earlier than later. - FLINK-5186 Move Row to flink-core. Row is used by other modules as a flexible and wider Tuple (e.g., flink-jdbc, RowCsvInputFormat). Several users requested to move it to flink-core, also to avoid Calcite which comes with flink-table. Both changes are mostly refactoring and should be solvable with rather little effort. @Flavio: I completely agree that we need a fix for FLINK-5122 in 1.2.0. Since this is not a new feature but rather a bugfix it won't be affected by the feature freeze, IMO. It makes sense to fix it for RC1 though. Best, Fabian 2016-12-05 18:35 GMT+01:00 Flavio Pompermaier <[hidden email]>: > Hi Robert, > our pull request was closed in favor of > https://github.com/apache/flink/pull/2861. > However that PR still need some review I think (I also added some comment > in the JIRA issue[1]). > it will be great to have that fix in the next release :) > > [1] https://issues.apache.org/jira/browse/FLINK-5122 > > Best, > Flavio > > On Mon, Dec 5, 2016 at 6:28 PM, Robert Metzger <[hidden email]> > wrote: > > > Hi, > > > > I would like to reactivate this old email thread. The initial plan was to > > release 1.2 before Christmas, have RC1 by end of Nov / December and a > > feature freeze mid-November. > > Today (December 5), we did not yet do the feature freeze. I would like to > > get an overview again of the remaining blockers and what we want to > include > > into the release. > > > > From the issues Fabian posted in the beginning, they have the following > > status: > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755) > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379) > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797) > > - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984) > > - UNRESOLVED Secure Data Access (FLINK-3930) > > - RESOLVED Queryable State (FLINK-3779) > > - RESOLVED Metrics in Webinterface (FLINK-4389) > > - RESOLVED Kafka 0.10 support (FLINK-4035) > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > - RESOLVED Table API: Scalar Functions (FLINK-3097) > > Added by Stephan: > > - UNRESOLVED Provide support for asynchronous operations over streams > > (FLINK-4391) > > - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) > > > > I've also reflected the status in the Wiki: > > https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > > > > @Flavio: I assume PR #2790 is resolved and not a release blocker. > > > > Given that we are already behind the schedule, I'm wondering what we can > do > > regarding the remaining UNRESOLVED issues. > > I would also be willing to release some of them as undocumented, > > work-in-progress features. We have so many good things like dynamic > scaling > > or Mesos in the release that we should get it out rather soon. > > > > It would be nice if those committers / contributors working on the > > UNRESOLVED issues could give a more detailed status update what they > think > > release-wise. > > Also, is there anything else we need to consider before forking off the > > release-1.2 branch from master? > > > > Once we've collected the detailed status updates from the last unresolved > > features, we can discuss a new date for the feature freeze. > > I'll then also create a RC0 testing release candidate without a VOTE, to > > have a starting point for the first big release testing. (And yes, I'm > > volunteering to be the release manager for the 1.2 release ;) ) > > > > > > Regards, > > Robert > > > > > > > > > > On Mon, Nov 21, 2016 at 3:03 PM, Flavio Pompermaier < > [hidden email]> > > wrote: > > > > > Hi to all, > > > we've completed the PR #2790 <https://github.com/apache/ > flink/pull/2790> > > > to > > > implement FLINK-4491 <https://issues.apache.org/jira/browse/FLINK-4491 > >: > > > we've added the comment that @zentol asked for a week ago and we also > > fixed > > > a potential annoying problem we faced during indexing (if a malformed > > > document generates an error during indexing, the close() method throws > an > > > exception forcing the entire job to fail, killing other threads that > are > > > still indexing). I hope this PR could hep other people working with > > > Elasticsearch, right from the next release ;) > > > > > > On Mon, Nov 14, 2016 at 2:47 PM, Flavio Pompermaier < > > [hidden email]> > > > wrote: > > > > > > > Hi to all, > > > > for the 1.2 release do you think it will be possible to include also > a > > PR > > > > to implement FLINK-4491 <https://issues.apache.org/ > > > jira/browse/FLINK-4491> > > > > ? > > > > The PR is waiting for a review at https://github.com/apache/ > > > > flink/pull/2790. > > > > > > > > Best, > > > > Flavio > > > > > > > > On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske <[hidden email]> > > > wrote: > > > > > > > >> Hi Jark, > > > >> > > > >> FLINK-4469: I agree, UDTFs would be a great feature. I don't think > we > > > >> should make it a blocker but I will help to get it the master before > > the > > > >> feature freeze. > > > >> FLINK-4704: Regarding the renaming of the namespace, we have two > > > positive > > > >> votes in JIRA, but we should also start a discussion on the dev and > > user > > > >> list to collect some more input. The actual change should be simple. > > > >> > > > >> @Everybody: I created a wiki page for the 1.2 release [1] and added > > all > > > >> features discussed in this thread so far. > > > >> Please have a look at the page and update it if you know the JIRAs > > > and/or > > > >> state of the features. > > > >> > > > >> Cheers, Fabian > > > >> > > > >> [1] https://cwiki.apache.org/confluence/display/FLINK/1.2+Release > > > >> > > > >> 2016-10-28 15:28 GMT+02:00 Kostas Kloudas < > > [hidden email] > > > >: > > > >> > > > >> > Hello, > > > >> > > > > >> > As Aljoshca said, the TriggerDSL depends on another PR, before it > > can > > > be > > > >> > merged. > > > >> > In addition, this will change also the whole structure of how we > > test > > > >> > window/trigger > > > >> > related functionality, so some refactoring will be required. > > > >> > > > > >> > Kostas > > > >> > > > > >> > > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek < > > [hidden email]> > > > >> > wrote: > > > >> > > > > > >> > > @Till: For the TriggerDSL we have to merge my open PR [1] for > > > >> refactoring > > > >> > > the WindowOperator/Trigger tests and then refactor the tests of > > the > > > >> > > TriggerDSL to use the new test facilities introduced there. Plus > > > then > > > >> > maybe > > > >> > > some fixups. > > > >> > > > > > >> > > On Fri, 28 Oct 2016 at 11:17 Jark Wu < > [hidden email]> > > > >> wrote: > > > >> > > > > > >> > >> +1 to the schedule > > > >> > >> > > > >> > >> I hope to add > > > >> > >> > > > >> > >> 1) Table API: User defined table function (FLINK-4469 < > > > >> > >> https://issues.apache.org/jira/browse/FLINK-4469>) > > > >> > >> 2) Table API: Move Table API to org.apache.flink.table > > (FLINK-4704 > > > < > > > >> > >> https://issues.apache.org/jira/browse/FLINK-4704>) > > > >> > >> > > > >> > >> The 1) has been discussed for a long time and a pull request > has > > > >> been on > > > >> > >> github. The 2) would be a large change, but I hope it could be > > > >> included > > > >> > in > > > >> > >> 1.2 release, otherwise it will be more difficult to fix it. > > > >> > >> > > > >> > >> - Jark Wu > > > >> > >> > > > >> > >>> 在 2016年10月26日，下午8:09，Till Rohrmann <[hidden email]> 写道： > > > >> > >>> > > > >> > >>> +1 for the features to include. > > > >> > >>> > > > >> > >>> What is the state of the Trigger DSL? How much is left to be > > done > > > >> > before > > > >> > >>> merging? > > > >> > >>> > > > >> > >>> Cheers, > > > >> > >>> Till > > > >> > >>> > > > >> > >>> On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek < > > > >> [hidden email] > > > >> > > > > > >> > >>> wrote: > > > >> > >>> > > > >> > >>>> +1 the schedule proposed so far. > > > >> > >>>> > > > >> > >>>> Do we also want to get in the "Trigger DSL" that we've had > > > brewing > > > >> > for a > > > >> > >>>> while now? > > > >> > >>>> > > > >> > >>>> On Mon, 17 Oct 2016 at 16:17 Stephan Ewen <[hidden email]> > > > >> wrote: > > > >> > >>>> > > > >> > >>>>> I think this sounds very reasonable, +1 to the schedule. > > > >> > >>>>> > > > >> > >>>>> I would definitely add > > > >> > >>>>> - FLIP-10 (unify checkpoints and savepoints) > > > >> > >>>>> - FLIP-7 (metrics in web UI) > > > >> > >>>>> - FLIP-12 (async request operators) > > > >> > >>>>> > > > >> > >>>>> Those should be all safe bets, as they are basically done. > > > >> > >>>>> > > > >> > >>>>> Let's see what else is in shape until the feature freeze > day. > > > >> > >>>>> > > > >> > >>>>> > > > >> > >>>>> > > > >> > >>>>> On Mon, Oct 17, 2016 at 2:18 PM, Fabian Hueske < > > > [hidden email] > > > >> > > > > >> > >>>> wrote: > > > >> > >>>>> > > > >> > >>>>>> Hi everybody, > > > >> > >>>>>> > > > >> > >>>>>> Flink 1.1.0 was released in August and I think it is time > to > > > kick > > > >> > off > > > >> > >> a > > > >> > >>>>>> discussion about the schedule and scope of Flink 1.2.0. > > > >> > >>>>>> > > > >> > >>>>>> == Scope > > > >> > >>>>>> > > > >> > >>>>>> We started to collect features for Flink 1.2.0 in the Flink > > > >> Release > > > >> > >>>> wiki > > > >> > >>>>>> page [1]. > > > >> > >>>>>> I copy the feature list for convenience: > > > >> > >>>>>> > > > >> > >>>>>> - Dynamic Scaling / Key Groups (FLINK-3755 > > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3755>) > (Design > > > Doc > > > >> > >>>>>> <https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu- > > > >> > >>>>>> LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4> > > > >> > >>>>>> ) > > > >> > >>>>>> - Rescalable Non-Partitioned State (FLIP-8 > > > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > > >> > >>>>>> 8%3A+Rescalable+Non-Partitioned+State> > > > >> > >>>>>> ) > > > >> > >>>>>> - Backwards compatible savepoints (FLINK-4797 > > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-4797>) > > > >> > >>>>>> - > > > >> > >>>>>> > > > >> > >>>>>> Integrate Flink with Apache Mesos (FLINK-1984 > > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-1984>) > (Design > > > Doc > > > >> > >>>>>> <https://docs.google.com/document/d/ > > 1WItafBmGbjlaBbP8Of5PAFO > > > >> H9GUJQ > > > >> > >>>>>> xf5S4hjEuPchuU/edit#heading=h.33uxp8abvuf0> > > > >> > >>>>>> ) > > > >> > >>>>>> - Secure Data Access (FLINK-3930 > > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3930>) > (Design > > > Doc > > > >> > >>>>>> <https://docs.google.com/document/d/1- > > > >> > >>>> GQB6uVOyoaXGwtqwqLV8BHDxWiMO2W > > > >> > >>>>>> nVzBoJ8oPaAs/edit#heading=h.dr22iualfm0h> > > > >> > >>>>>> ) > > > >> > >>>>>> - Queryable State (FLINK-3779 > > > >> > >>>>>> <https://issues.apache.org/jira/browse/FLINK-3779>) > > > >> > >>>>>> - Metrics in Webinterface (FLIP-7 > > > >> > >>>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP- > > > >> > >>>>>> 7%3A+Expose+metrics+to+WebInterface> > > > >> > >>>>>> ) > > > >> > >>>>>> - Kafka 0.10 support (FLINK-4035 > > > >> > >>>>>> <http://issues.apache.org/jira/browse/FLINK-4035>) > > > >> > >>>>>> - Table API: Group Window Aggregates (FLINK-4691, FLIP-11) > > > >> > >>>>>> - Table API: Scalar Functions (FLINK-3097) > > > >> > >>>>>> > > > >> > >>>>>> > > > >> > >>>>>> Is a feature missing or something too ambitious? > > > >> > >>>>>> > > > >> > >>>>>> == Schedule > > > >> > >>>>>> > > > >> > >>>>>> In my opinion we should try to release 1.2.0 before > > Christmas. > > > >> That > > > >> > >>>>> means, > > > >> > >>>>>> we should have a RC1 by end November / early December and a > > > >> feature > > > >> > >>>>> freeze > > > >> > >>>>>> two weeks earlier in mid of November. Does that sound > > > >> appropriate or > > > >> > >>>>> should > > > >> > >>>>>> we aim for an earlier or later release? > > > >> > >>>>>> > > > >> > >>>>>> What do others think about the scope and schedule? > > > >> > >>>>>> Please discuss. > > > >> > >>>>>> > > > >> > >>>>>> Best, > > > >> > >>>>>> Fabian > > > >> > >>>>>> > > > >> > >>>>>> [1] > > > >> > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/ > > > >> > >>>>>> Flink+Release+and+Feature+Plan > > > >> > >>>>>> > > > >> > >>>>> > > > >> > >>>> > > > >> > >> > > > >> > >> > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
In reply to this post by Robert Metzger
Hey Robert,
thanks for bringing the discussion up again. On Mon, Dec 5, 2016 at 6:28 PM, Robert Metzger <[hidden email]> wrote: > - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) This issue/FLIP contained two parts: 1. Write periodic checkpoints out to external storage 2. Allow periodic scheduling of savepoints (1) is done. (2) is aimed for after 1.2. There are two small non-blocking PRs for (1) that would be nice to get in before 1.2: https://github.com/apache/flink/pull/2750 https://github.com/apache/flink/pull/2752 – Ufuk |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Thanks Robert for bringing this up again.
Would be great to come to that @use: I'll take a look at https://github.com/apache/flink/pull/2752 Considering "UNRESOLVED Provide support for asynchronous operations over streams (FLINK-4391)" - that is currently in the hands of the contributor. I would like it if it went into the release, but would not block the release on that, if it is not ready in time. Best, Stephan On Tue, Dec 6, 2016 at 2:32 PM, Ufuk Celebi <[hidden email]> wrote: > Hey Robert, > > thanks for bringing the discussion up again. > > On Mon, Dec 5, 2016 at 6:28 PM, Robert Metzger <[hidden email]> > wrote: > > - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) > > This issue/FLIP contained two parts: > 1. Write periodic checkpoints out to external storage > 2. Allow periodic scheduling of savepoints > > (1) is done. (2) is aimed for after 1.2. > > There are two small non-blocking PRs for (1) that would be nice to get > in before 1.2: > https://github.com/apache/flink/pull/2750 > https://github.com/apache/flink/pull/2752 > > – Ufuk > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
In reply to this post by Robert Metzger
Hi Robert,
The state of (FLINK-4797) Add Flink 1.1 savepoint backwards compatibility is as follows: We have identified 5 major steps: 1) Convert savepoint format, task state, state handles, etc. 2) Convert binary format of keyed state backends (Rocks, Heap, Fs) 3) Introduce a first step for for being able to upgrade type serializers in the future, in particular versioning and making them part of a check/savepoint 4) Ensure current operators are compatible with their old format 5) Intensive testing 1) and 2) are finished and reviewed, they will be merged today or tomorrow. 3) Is almost done, then must be reviewed. My estimate is end of this week. 4) Kostas is working on this, but my feeling is that we are pretty close to completion. We already restored a job with general window operator and kafka source in an end-to-end test. My estimate for this would be end of this week or beginning of next week. 5) While we already tested a lo during development, I think this needs some thorough end-to-end testing after steps 1-4 are completed. Best, Stefan |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Hi Robert,
I agree with Stefan on the estimation about 4. As for the testing, this can already start, as some basic operators have already been ported. Kostas > On Dec 6, 2016, at 5:50 PM, Stefan Richter <[hidden email]> wrote: > > e already restored a job with general window operator and kafka source in an end-to-en |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
Thank you all for your responses so far. The situation is better than I
expected :) Let me summarize, and please object if you disagree: - RESOLVED dynamic Scaling / Key Groups (FLINK-3755) - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379) - RESOLVED [Split for 1.3] Add Flink 1.1 savepoint backwards compatability (FLINK-4797) - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984) - UNRESOLVED Secure Data Access (FLINK-3930) - RESOLVED Queryable State (FLINK-3779) - RESOLVED Metrics in Webinterface (FLINK-4389) - RESOLVED Kafka 0.10 support (FLINK-4035) - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11) - RESOLVED Table API: Scalar Functions (FLINK-3097) Added by Stephan: - NON-BLOCKING [Pending PR] Provide support for asynchronous operations over streams (FLINK-4391) - ONGOING [beginning of next week] Unify Savepoints and Checkpoints (FLINK-4484) Added by Fabian: - UNRESOLVED Clean up the packages of the Table API (FLINK-4704) - UNRESOLVED Move Row to flink-core (FLINK-5186) On Tue, Dec 6, 2016 at 5:55 PM, Kostas Kloudas <[hidden email]> wrote: > Hi Robert, > > I agree with Stefan on the estimation about 4. > As for the testing, this can already start, as some basic operators have > already been ported. > > Kostas > > > On Dec 6, 2016, at 5:50 PM, Stefan Richter <[hidden email]> > wrote: > > > > e already restored a job with general window operator and kafka source > in an end-to-en > > |
Re: [DISCUSS] Schedule and Scope for Flink 1.2
|
|
>>Secure Data Access (FLINK-3930)
The PR for the work is still under review and I hope this could be included in the release. Regards, Vijay Sent from my iPhone > On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]> wrote: > > UNRESOLVED Secure Data Access (FLINK-3930) |
|
|
> - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984)
The initial integration is already completed with the last issues being resolved in the Mesos component: https://issues.apache.org/jira/browse/FLINK/component/12331068/ The implementation will be further refined after the next release and with the merge of FLIP-6. We're missing documentation on how to deploy a Flink Mesos cluster. > - UNRESOLVED Secure Data Access (FLINK-3930) We have support for Kerberos authentication with Haddop, Kafka, Zookeper, and all services supporting JAAS. Additionally, we implemented SSL encryption for all communications paths, i.e. web interface, Akka, Netty, BlobServer. We still lack support for authorization: Vijay's PR is blocked because we haven't found time to properly review the sensitive network changes. I'd like to add the Akka changes for containered environments which should be ready by the end of the week: https://issues.apache.org/jira/browse/FLINK-2821 -Max On Tue, Dec 6, 2016 at 8:57 PM, Vijay <[hidden email]> wrote: >>>Secure Data Access (FLINK-3930) > > The PR for the work is still under review and I hope this could be included in the release. > > Regards, > Vijay > > Sent from my iPhone > >> On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]> wrote: >> >> UNRESOLVED Secure Data Access (FLINK-3930) > |
«
Return to (DEPRECATED) Apache Flink Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |