[DISCUSS] Schedule and Scope for Flink 1.2

[Robert Metzger]

Thank you for your responses Max and Vijay.
So I understand that Mesos is basically ready for the 1.2 release.

Regarding the security changes: Having Hadoop, Kafka and Zookeeper
integration is a big improvement and a much requested feature. I'm super
excited to have that in :)
Are all the other security changes useless without authorization, or could
we consider releasing 1.2 without it? (Another way to think about it: How
close is the PR to being merged. If its just a final review & we are done,
I would actually try to get it in. But if there's a lot of uncertainty, I
would prefer to move it to the next release)

I agree regarding FLINK-2821, that's important for many deployments.


The updated list:
- RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
- RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
- RESOLVED [Split for 1.3] Add Flink 1.1 savepoint backwards compatability
(FLINK-4797)
- RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
- UNRESOLVED Secure Data Access (FLINK-3930)
- RESOLVED Queryable State (FLINK-3779)
- RESOLVED Metrics in Webinterface (FLINK-4389)
- RESOLVED Kafka 0.10 support (FLINK-4035)
- RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
- RESOLVED Table API: Scalar Functions (FLINK-3097)
Added by Stephan:
- NON-BLOCKING [Pending PR] Provide support for asynchronous operations
over streams (FLINK-4391)
- ONGOING [beginning of next week] Unify Savepoints and Checkpoints
(FLINK-4484)
Added by Fabian:
- ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
- UNRESOLVED Move Row to flink-core (FLINK-5186)
Added by Max:
- ONGOING [Pending PR] Change Akka configuration to allow accessing actors
from different URLs (FLINK-2821)


On Wed, Dec 7, 2016 at 12:40 PM, Maximilian Michels <[hidden email]> wrote:

> > - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984)
>
> The initial integration is already completed with the last issues
> being resolved in the Mesos component:
> https://issues.apache.org/jira/browse/FLINK/component/12331068/ The
> implementation will be further refined after the next release and with
> the merge of FLIP-6. We're missing documentation on how to deploy a
> Flink Mesos cluster.
>
> > - UNRESOLVED Secure Data Access (FLINK-3930)
>
> We have support for Kerberos authentication with Haddop, Kafka,
> Zookeper, and all services supporting JAAS. Additionally, we
> implemented SSL encryption for all communications paths, i.e. web
> interface, Akka, Netty, BlobServer. We still lack support for
> authorization: Vijay's PR is blocked because we haven't found time to
> properly review the sensitive network changes.
>
> I'd like to add the Akka changes for containered environments which
> should be ready by the end of the week:
> https://issues.apache.org/jira/browse/FLINK-2821
>
> -Max
>
> On Tue, Dec 6, 2016 at 8:57 PM, Vijay <[hidden email]>
> wrote:
> >>>Secure Data Access (FLINK-3930)
> >
> > The PR for the work is still under review and I hope this could be
> included in the release.
> >
> > Regards,
> > Vijay
> >
> > Sent from my iPhone
> >
> >> On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]>
> wrote:
> >>
> >> UNRESOLVED Secure Data Access (FLINK-3930)
> >
>

[Till Rohrmann]

We might also think about addressing:

Relocate Flink's Hadoop dependency and its transitive dependencies
(FLINK-5297),

because a user reported that they cannot use the system due to a dependency
issue.

Cheers,
Till

On Thu, Dec 8, 2016 at 10:17 AM, Robert Metzger <[hidden email]> wrote:

> Thank you for your responses Max and Vijay.
> So I understand that Mesos is basically ready for the 1.2 release.
>
> Regarding the security changes: Having Hadoop, Kafka and Zookeeper
> integration is a big improvement and a much requested feature. I'm super
> excited to have that in :)
> Are all the other security changes useless without authorization, or could
> we consider releasing 1.2 without it? (Another way to think about it: How
> close is the PR to being merged. If its just a final review & we are done,
> I would actually try to get it in. But if there's a lot of uncertainty, I
> would prefer to move it to the next release)
>
> I agree regarding FLINK-2821, that's important for many deployments.
>
>
> The updated list:
> - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> - RESOLVED [Split for 1.3] Add Flink 1.1 savepoint backwards compatability
> (FLINK-4797)
> - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
> - UNRESOLVED Secure Data Access (FLINK-3930)
> - RESOLVED Queryable State (FLINK-3779)
> - RESOLVED Metrics in Webinterface (FLINK-4389)
> - RESOLVED Kafka 0.10 support (FLINK-4035)
> - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> - RESOLVED Table API: Scalar Functions (FLINK-3097)
> Added by Stephan:
> - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
> over streams (FLINK-4391)
> - ONGOING [beginning of next week] Unify Savepoints and Checkpoints
> (FLINK-4484)
> Added by Fabian:
> - ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
> - UNRESOLVED Move Row to flink-core (FLINK-5186)
> Added by Max:
> - ONGOING [Pending PR] Change Akka configuration to allow accessing actors
> from different URLs (FLINK-2821)
>
>
> On Wed, Dec 7, 2016 at 12:40 PM, Maximilian Michels <[hidden email]>
> wrote:
>
> > > - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984)
> >
> > The initial integration is already completed with the last issues
> > being resolved in the Mesos component:
> > https://issues.apache.org/jira/browse/FLINK/component/12331068/ The
> > implementation will be further refined after the next release and with
> > the merge of FLIP-6. We're missing documentation on how to deploy a
> > Flink Mesos cluster.
> >
> > > - UNRESOLVED Secure Data Access (FLINK-3930)
> >
> > We have support for Kerberos authentication with Haddop, Kafka,
> > Zookeper, and all services supporting JAAS. Additionally, we
> > implemented SSL encryption for all communications paths, i.e. web
> > interface, Akka, Netty, BlobServer. We still lack support for
> > authorization: Vijay's PR is blocked because we haven't found time to
> > properly review the sensitive network changes.
> >
> > I'd like to add the Akka changes for containered environments which
> > should be ready by the end of the week:
> > https://issues.apache.org/jira/browse/FLINK-2821
> >
> > -Max
> >
> > On Tue, Dec 6, 2016 at 8:57 PM, Vijay <[hidden email]>
> > wrote:
> > >>>Secure Data Access (FLINK-3930)
> > >
> > > The PR for the work is still under review and I hope this could be
> > included in the release.
> > >
> > > Regards,
> > > Vijay
> > >
> > > Sent from my iPhone
> > >
> > >> On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]>
> > wrote:
> > >>
> > >> UNRESOLVED Secure Data Access (FLINK-3930)
> > >
> >
>

[Vijay Srinivasaraghavan]

On FLINK-3930, almost all of the feedback has been addressed. The only pending review is Netty cookie authorization part which I have moved the cookie validation from message level to a separate channel handler. I have just rebased the code with master for final review.

Regards,
Vijay

Sent from my iPhone

> On Dec 8, 2016, at 1:17 AM, Robert Metzger <[hidden email]> wrote:
>
> Thank you for your responses Max and Vijay.
> So I understand that Mesos is basically ready for the 1.2 release.
>
> Regarding the security changes: Having Hadoop, Kafka and Zookeeper
> integration is a big improvement and a much requested feature. I'm super
> excited to have that in :)
> Are all the other security changes useless without authorization, or could
> we consider releasing 1.2 without it? (Another way to think about it: How
> close is the PR to being merged. If its just a final review & we are done,
> I would actually try to get it in. But if there's a lot of uncertainty, I
> would prefer to move it to the next release)
>
> I agree regarding FLINK-2821, that's important for many deployments.
>
>
> The updated list:
> - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> - RESOLVED [Split for 1.3] Add Flink 1.1 savepoint backwards compatability
> (FLINK-4797)
> - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
> - UNRESOLVED Secure Data Access (FLINK-3930)
> - RESOLVED Queryable State (FLINK-3779)
> - RESOLVED Metrics in Webinterface (FLINK-4389)
> - RESOLVED Kafka 0.10 support (FLINK-4035)
> - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> - RESOLVED Table API: Scalar Functions (FLINK-3097)
> Added by Stephan:
> - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
> over streams (FLINK-4391)
> - ONGOING [beginning of next week] Unify Savepoints and Checkpoints
> (FLINK-4484)
> Added by Fabian:
> - ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
> - UNRESOLVED Move Row to flink-core (FLINK-5186)
> Added by Max:
> - ONGOING [Pending PR] Change Akka configuration to allow accessing actors
> from different URLs (FLINK-2821)
>
>
> On Wed, Dec 7, 2016 at 12:40 PM, Maximilian Michels <[hidden email]> wrote:
>
>>> - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984)
>>
>> The initial integration is already completed with the last issues
>> being resolved in the Mesos component:
>> https://issues.apache.org/jira/browse/FLINK/component/12331068/ The
>> implementation will be further refined after the next release and with
>> the merge of FLIP-6. We're missing documentation on how to deploy a
>> Flink Mesos cluster.
>>
>>> - UNRESOLVED Secure Data Access (FLINK-3930)
>>
>> We have support for Kerberos authentication with Haddop, Kafka,
>> Zookeper, and all services supporting JAAS. Additionally, we
>> implemented SSL encryption for all communications paths, i.e. web
>> interface, Akka, Netty, BlobServer. We still lack support for
>> authorization: Vijay's PR is blocked because we haven't found time to
>> properly review the sensitive network changes.
>>
>> I'd like to add the Akka changes for containered environments which
>> should be ready by the end of the week:
>> https://issues.apache.org/jira/browse/FLINK-2821
>>
>> -Max
>>
>> On Tue, Dec 6, 2016 at 8:57 PM, Vijay <[hidden email]>
>> wrote:
>>>>> Secure Data Access (FLINK-3930)
>>>
>>> The PR for the work is still under review and I hope this could be
>> included in the release.
>>>
>>> Regards,
>>> Vijay
>>>
>>> Sent from my iPhone
>>>
>>>> On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]>
>> wrote:
>>>>
>>>> UNRESOLVED Secure Data Access (FLINK-3930)
>>>
>>

[mxm]

Hi Vijay,

Thank you for updating the pull request. I appreciate your work in the
security realm of Flink and value your contributions so far. It would
be great to merge the authorization pull request for the release.
However, I don't feel comfortable about the network stack (i.e. Netty)
related changes because they touch a very critical part of the Flink
engine. While the code has already been iterated over in the lifetime
of the pull request, I still think somebody extremely familiar with
Netty and the Flink network layer should check out the changes
(thinking about Ufuk here). Apart from that, I would like to further
test the changes to harden the code.

It seems like we lack the resources for now to properly to take care
of your pull request before the release. Unless someone from the
community is really eager to help out here, I would be in favor of
merging the pull request to the master after the release branch has
been forked off. We should make sure it gets the attention it deserves
then.

Thanks,
Max

On Mon, Dec 12, 2016 at 8:09 AM, Vijay <[hidden email]> wrote:

> On FLINK-3930, almost all of the feedback has been addressed. The only pending review is Netty cookie authorization part which I have moved the cookie validation from message level to a separate channel handler. I have just rebased the code with master for final review.
>
> Regards,
> Vijay
>
> Sent from my iPhone
>
>> On Dec 8, 2016, at 1:17 AM, Robert Metzger <[hidden email]> wrote:
>>
>> Thank you for your responses Max and Vijay.
>> So I understand that Mesos is basically ready for the 1.2 release.
>>
>> Regarding the security changes: Having Hadoop, Kafka and Zookeeper
>> integration is a big improvement and a much requested feature. I'm super
>> excited to have that in :)
>> Are all the other security changes useless without authorization, or could
>> we consider releasing 1.2 without it? (Another way to think about it: How
>> close is the PR to being merged. If its just a final review & we are done,
>> I would actually try to get it in. But if there's a lot of uncertainty, I
>> would prefer to move it to the next release)
>>
>> I agree regarding FLINK-2821, that's important for many deployments.
>>
>>
>> The updated list:
>> - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
>> - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
>> - RESOLVED [Split for 1.3] Add Flink 1.1 savepoint backwards compatability
>> (FLINK-4797)
>> - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
>> - UNRESOLVED Secure Data Access (FLINK-3930)
>> - RESOLVED Queryable State (FLINK-3779)
>> - RESOLVED Metrics in Webinterface (FLINK-4389)
>> - RESOLVED Kafka 0.10 support (FLINK-4035)
>> - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
>> - RESOLVED Table API: Scalar Functions (FLINK-3097)
>> Added by Stephan:
>> - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
>> over streams (FLINK-4391)
>> - ONGOING [beginning of next week] Unify Savepoints and Checkpoints
>> (FLINK-4484)
>> Added by Fabian:
>> - ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
>> - UNRESOLVED Move Row to flink-core (FLINK-5186)
>> Added by Max:
>> - ONGOING [Pending PR] Change Akka configuration to allow accessing actors
>> from different URLs (FLINK-2821)
>>
>>
>> On Wed, Dec 7, 2016 at 12:40 PM, Maximilian Michels <[hidden email]> wrote:
>>
>>>> - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984)
>>>
>>> The initial integration is already completed with the last issues
>>> being resolved in the Mesos component:
>>> https://issues.apache.org/jira/browse/FLINK/component/12331068/ The
>>> implementation will be further refined after the next release and with
>>> the merge of FLIP-6. We're missing documentation on how to deploy a
>>> Flink Mesos cluster.
>>>
>>>> - UNRESOLVED Secure Data Access (FLINK-3930)
>>>
>>> We have support for Kerberos authentication with Haddop, Kafka,
>>> Zookeper, and all services supporting JAAS. Additionally, we
>>> implemented SSL encryption for all communications paths, i.e. web
>>> interface, Akka, Netty, BlobServer. We still lack support for
>>> authorization: Vijay's PR is blocked because we haven't found time to
>>> properly review the sensitive network changes.
>>>
>>> I'd like to add the Akka changes for containered environments which
>>> should be ready by the end of the week:
>>> https://issues.apache.org/jira/browse/FLINK-2821
>>>
>>> -Max
>>>
>>> On Tue, Dec 6, 2016 at 8:57 PM, Vijay <[hidden email]>
>>> wrote:
>>>>>> Secure Data Access (FLINK-3930)
>>>>
>>>> The PR for the work is still under review and I hope this could be
>>> included in the release.
>>>>
>>>> Regards,
>>>> Vijay
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On Dec 6, 2016, at 11:51 AM, Robert Metzger <[hidden email]>
>>> wrote:
>>>>>
>>>>> UNRESOLVED Secure Data Access (FLINK-3930)
>>>>
>>>
>

[Ufuk Celebi-2]

On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email]) wrote:

> > It seems like we lack the resources for now to properly to take  
> care
> of your pull request before the release. Unless someone from  
> the
> community is really eager to help out here, I would be in favor  
> of
> merging the pull request to the master after the release branch  
> has
> been forked off. We should make sure it gets the attention it deserves  
> then.

Thanks Max! I fully agree with your reasoning. +1 to not include this in 1.2 now, but look at it afterwards. I hope that OK with you Vijay.

– Ufuk

[Vijay Srinivasaraghavan]

Max and Ufuk, I respect your concerns and fully understand the importance of the network layer stack in Flink code base. Will you be comfortable to merge the code if I remove the Netty layer changes and leave the rest of the code. We can address the Netty code changes post 1.2 release?

Regards,
Vijay

Sent from my iPhone

> On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
>
> On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email]) wrote:
>>> It seems like we lack the resources for now to properly to take  
>> care
>> of your pull request before the release. Unless someone from  
>> the
>> community is really eager to help out here, I would be in favor  
>> of
>> merging the pull request to the master after the release branch  
>> has
>> been forked off. We should make sure it gets the attention it deserves  
>> then.
>
> Thanks Max! I fully agree with your reasoning. +1 to not include this in 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
>
> – Ufuk
>
>

[Stephan Ewen]

Hi Vijay!

The workaround you suggest may be doable, but I am wondering how much that
helps, because the authorization feature would be incomplete like that and
thus of limited use.

I would also assume that merging it properly and in full use after the 1.2
release would be a bit better - in general, we have often avoided last
minute additions of sensitive and complex features.

Do you think it is more urgent to have this in Flink?

Best,
Stephan


On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]> wrote:

> Max and Ufuk, I respect your concerns and fully understand the importance
> of the network layer stack in Flink code base. Will you be comfortable to
> merge the code if I remove the Netty layer changes and leave the rest of
> the code. We can address the Netty code changes post 1.2 release?
>
> Regards,
> Vijay
>
> Sent from my iPhone
>
> > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> >
> > On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email])
> wrote:
> >>> It seems like we lack the resources for now to properly to take
> >> care
> >> of your pull request before the release. Unless someone from
> >> the
> >> community is really eager to help out here, I would be in favor
> >> of
> >> merging the pull request to the master after the release branch
> >> has
> >> been forked off. We should make sure it gets the attention it deserves
> >> then.
> >
> > Thanks Max! I fully agree with your reasoning. +1 to not include this in
> 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> >
> > – Ufuk
> >
> >
>
>

[Vijay Srinivasaraghavan]

Hi Stephan,

The new handler changes for Netty layer is straightforward and it is not really an workaround. Moreover if we think there is a better way to handle it in future we can easily unwind it as it is just a pluggable handler. Pushing these changes before FLIP-6 merge certainly avoids lot of conflicts and also we already spent cycles in reviewing and fixing rest of the authorization changes for other layers.

Regards,
Vijay

Sent from my iPhone

> On Dec 12, 2016, at 8:43 AM, Stephan Ewen <[hidden email]> wrote:
>
> Hi Vijay!
>
> The workaround you suggest may be doable, but I am wondering how much that
> helps, because the authorization feature would be incomplete like that and
> thus of limited use.
>
> I would also assume that merging it properly and in full use after the 1.2
> release would be a bit better - in general, we have often avoided last
> minute additions of sensitive and complex features.
>
> Do you think it is more urgent to have this in Flink?
>
> Best,
> Stephan
>
>
>> On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]> wrote:
>>
>> Max and Ufuk, I respect your concerns and fully understand the importance
>> of the network layer stack in Flink code base. Will you be comfortable to
>> merge the code if I remove the Netty layer changes and leave the rest of
>> the code. We can address the Netty code changes post 1.2 release?
>>
>> Regards,
>> Vijay
>>
>> Sent from my iPhone
>>
>>> On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
>>>
>>> On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email])
>> wrote:
>>>>> It seems like we lack the resources for now to properly to take
>>>> care
>>>> of your pull request before the release. Unless someone from
>>>> the
>>>> community is really eager to help out here, I would be in favor
>>>> of
>>>> merging the pull request to the master after the release branch
>>>> has
>>>> been forked off. We should make sure it gets the attention it deserves
>>>> then.
>>>
>>> Thanks Max! I fully agree with your reasoning. +1 to not include this in
>> 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
>>>
>>> – Ufuk
>>>
>>>
>>
>>

[Robert Metzger]

Thank you all for figuring out a solution for the security pull request.


Lets try to get 1.2 feature freezed as fast as possible so that we can
"unblock" waiting features like FLIP-6 and the remaining security changes.

*What do you think about Friday evening (6pm Berlin, 9am US west coast) for
feature freezing Flink 1.2?* (only bugfixes are allowed in afterwards)
I'll then fork-off a "release-1.2" branch and update the version in
"master" to 1.3-SNAPSHOT.
Please object if you have a bigger change or any other reservations
regarding the feature freeze date!

This is my current view of things on the release:

- RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
- RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
- UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797)
- RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
- UNDER DISCUSSION Secure Data Access (FLINK-3930)
- RESOLVED Queryable State (FLINK-3779)
- RESOLVED Metrics in Webinterface (FLINK-4389)
- RESOLVED Kafka 0.10 support (FLINK-4035)
- RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
- RESOLVED Table API: Scalar Functions (FLINK-3097)
Added by Stephan:
- NON-BLOCKING [Pending PR] Provide support for asynchronous operations
over streams (FLINK-4391)
- NON-BLOCKING [beginning of next week] Unify Savepoints and Checkpoints
(FLINK-4484)
Added by Fabian:
- ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
 Move Row to flink-core (
Added by Max:
- ONGOING [Pending PR] Change Akka configuration to allow accessing actors
from different URLs (FLINK-2821)


On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]> wrote:

> Hi Vijay!
>
> The workaround you suggest may be doable, but I am wondering how much that
> helps, because the authorization feature would be incomplete like that and
> thus of limited use.
>
> I would also assume that merging it properly and in full use after the 1.2
> release would be a bit better - in general, we have often avoided last
> minute additions of sensitive and complex features.
>
> Do you think it is more urgent to have this in Flink?
>
> Best,
> Stephan
>
>
> On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]>
> wrote:
>
> > Max and Ufuk, I respect your concerns and fully understand the importance
> > of the network layer stack in Flink code base. Will you be comfortable to
> > merge the code if I remove the Netty layer changes and leave the rest of
> > the code. We can address the Netty code changes post 1.2 release?
> >
> > Regards,
> > Vijay
> >
> > Sent from my iPhone
> >
> > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> > >
> > > On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email])
> > wrote:
> > >>> It seems like we lack the resources for now to properly to take
> > >> care
> > >> of your pull request before the release. Unless someone from
> > >> the
> > >> community is really eager to help out here, I would be in favor
> > >> of
> > >> merging the pull request to the master after the release branch
> > >> has
> > >> been forked off. We should make sure it gets the attention it deserves
> > >> then.
> > >
> > > Thanks Max! I fully agree with your reasoning. +1 to not include this
> in
> > 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> > >
> > > – Ufuk
> > >
> > >
> >
> >
>

[Feng Wang]

It will be pretty good if 1.2 branch could be forked off within this week, and our guys working on FLIP-6  hope FLIP-6 branch could be merged into master as soon as possible.

Best Regards,

Feng Wang

Alibaba

________________________________
From: Robert Metzger <[hidden email]>
Sent: Tuesday, December 13, 2016 4:58 AM
To: [hidden email]
Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thank you all for figuring out a solution for the security pull request.


Lets try to get 1.2 feature freezed as fast as possible so that we can
"unblock" waiting features like FLIP-6 and the remaining security changes.

*What do you think about Friday evening (6pm Berlin, 9am US west coast) for
feature freezing Flink 1.2?* (only bugfixes are allowed in afterwards)
I'll then fork-off a "release-1.2" branch and update the version in
"master" to 1.3-SNAPSHOT.
Please object if you have a bigger change or any other reservations
regarding the feature freeze date!

This is my current view of things on the release:

- RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
- RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
- UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797)
- RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
- UNDER DISCUSSION Secure Data Access (FLINK-3930)
- RESOLVED Queryable State (FLINK-3779)
- RESOLVED Metrics in Webinterface (FLINK-4389)
- RESOLVED Kafka 0.10 support (FLINK-4035)
- RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
- RESOLVED Table API: Scalar Functions (FLINK-3097)
Added by Stephan:
- NON-BLOCKING [Pending PR] Provide support for asynchronous operations
over streams (FLINK-4391)
- NON-BLOCKING [beginning of next week] Unify Savepoints and Checkpoints
(FLINK-4484)
Added by Fabian:
- ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
 Move Row to flink-core (
Added by Max:
- ONGOING [Pending PR] Change Akka configuration to allow accessing actors
from different URLs (FLINK-2821)


On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]> wrote:

> Hi Vijay!
>
> The workaround you suggest may be doable, but I am wondering how much that
> helps, because the authorization feature would be incomplete like that and
> thus of limited use.
>
> I would also assume that merging it properly and in full use after the 1.2
> release would be a bit better - in general, we have often avoided last
> minute additions of sensitive and complex features.
>
> Do you think it is more urgent to have this in Flink?
>
> Best,
> Stephan
>
>
> On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]>
> wrote:
>
> > Max and Ufuk, I respect your concerns and fully understand the importance
> > of the network layer stack in Flink code base. Will you be comfortable to
> > merge the code if I remove the Netty layer changes and leave the rest of
> > the code. We can address the Netty code changes post 1.2 release?
> >
> > Regards,
> > Vijay
> >
> > Sent from my iPhone
> >
> > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> > >
> > > On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email])
> > wrote:
> > >>> It seems like we lack the resources for now to properly to take
> > >> care
> > >> of your pull request before the release. Unless someone from
> > >> the
> > >> community is really eager to help out here, I would be in favor
> > >> of
> > >> merging the pull request to the master after the release branch
> > >> has
> > >> been forked off. We should make sure it gets the attention it deserves
> > >> then.
> > >
> > > Thanks Max! I fully agree with your reasoning. +1 to not include this
> in
> > 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> > >
> > > - Ufuk
> > >
> > >
> >
> >
>

[Aljoscha Krettek-2]

Hi,
we're still working on making the backwards compatibility from 1.1
savepoints a reality. We have most of the code and some tests now but it
still needs some work. This is the issue that tracks the progress on the
operators that we would like to make backwards compatible:
https://issues.apache.org/jira/browse/FLINK-5292

Cheers,
Aljoscha

On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]> wrote:

> It will be pretty good if 1.2 branch could be forked off within this week,
> and our guys working on FLIP-6  hope FLIP-6 branch could be merged into
> master as soon as possible.
>
> Best Regards,
>
> Feng Wang
>
> Alibaba
>
> ________________________________
> From: Robert Metzger <[hidden email]>
> Sent: Tuesday, December 13, 2016 4:58 AM
> To: [hidden email]
> Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
>
> Thank you all for figuring out a solution for the security pull request.
>
>
> Lets try to get 1.2 feature freezed as fast as possible so that we can
> "unblock" waiting features like FLIP-6 and the remaining security changes.
>
> *What do you think about Friday evening (6pm Berlin, 9am US west coast) for
> feature freezing Flink 1.2?* (only bugfixes are allowed in afterwards)
> I'll then fork-off a "release-1.2" branch and update the version in
> "master" to 1.3-SNAPSHOT.
> Please object if you have a bigger change or any other reservations
> regarding the feature freeze date!
>
> This is my current view of things on the release:
>
> - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> - UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797)
> - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
> - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> - RESOLVED Queryable State (FLINK-3779)
> - RESOLVED Metrics in Webinterface (FLINK-4389)
> - RESOLVED Kafka 0.10 support (FLINK-4035)
> - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> - RESOLVED Table API: Scalar Functions (FLINK-3097)
> Added by Stephan:
> - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
> over streams (FLINK-4391)
> - NON-BLOCKING [beginning of next week] Unify Savepoints and Checkpoints
> (FLINK-4484)
> Added by Fabian:
> - ONGOING [Pending PR] Clean up the packages of the Table API (FLINK-4704)
>  Move Row to flink-core (
> Added by Max:
> - ONGOING [Pending PR] Change Akka configuration to allow accessing actors
> from different URLs (FLINK-2821)
>
>
> On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]> wrote:
>
> > Hi Vijay!
> >
> > The workaround you suggest may be doable, but I am wondering how much
> that
> > helps, because the authorization feature would be incomplete like that
> and
> > thus of limited use.
> >
> > I would also assume that merging it properly and in full use after the
> 1.2
> > release would be a bit better - in general, we have often avoided last
> > minute additions of sensitive and complex features.
> >
> > Do you think it is more urgent to have this in Flink?
> >
> > Best,
> > Stephan
> >
> >
> > On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]>
> > wrote:
> >
> > > Max and Ufuk, I respect your concerns and fully understand the
> importance
> > > of the network layer stack in Flink code base. Will you be comfortable
> to
> > > merge the code if I remove the Netty layer changes and leave the rest
> of
> > > the code. We can address the Netty code changes post 1.2 release?
> > >
> > > Regards,
> > > Vijay
> > >
> > > Sent from my iPhone
> > >
> > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> > > >
> > > > On 12 December 2016 at 12:30:31, Maximilian Michels ([hidden email])
> > > wrote:
> > > >>> It seems like we lack the resources for now to properly to take
> > > >> care
> > > >> of your pull request before the release. Unless someone from
> > > >> the
> > > >> community is really eager to help out here, I would be in favor
> > > >> of
> > > >> merging the pull request to the master after the release branch
> > > >> has
> > > >> been forked off. We should make sure it gets the attention it
> deserves
> > > >> then.
> > > >
> > > > Thanks Max! I fully agree with your reasoning. +1 to not include this
> > in
> > > 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> > > >
> > > > - Ufuk
> > > >
> > > >
> > >
> > >
> >
>

[Robert Metzger]

Thank you for the update. Do you think you get it done until Monday evening?

On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
> we're still working on making the backwards compatibility from 1.1
> savepoints a reality. We have most of the code and some tests now but it
> still needs some work. This is the issue that tracks the progress on the
> operators that we would like to make backwards compatible:
> https://issues.apache.org/jira/browse/FLINK-5292
>
> Cheers,
> Aljoscha
>
> On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]> wrote:
>
> > It will be pretty good if 1.2 branch could be forked off within this
> week,
> > and our guys working on FLIP-6  hope FLIP-6 branch could be merged into
> > master as soon as possible.
> >
> > Best Regards,
> >
> > Feng Wang
> >
> > Alibaba
> >
> > ________________________________
> > From: Robert Metzger <[hidden email]>
> > Sent: Tuesday, December 13, 2016 4:58 AM
> > To: [hidden email]
> > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> >
> > Thank you all for figuring out a solution for the security pull request.
> >
> >
> > Lets try to get 1.2 feature freezed as fast as possible so that we can
> > "unblock" waiting features like FLIP-6 and the remaining security
> changes.
> >
> > *What do you think about Friday evening (6pm Berlin, 9am US west coast)
> for
> > feature freezing Flink 1.2?* (only bugfixes are allowed in afterwards)
> > I'll then fork-off a "release-1.2" branch and update the version in
> > "master" to 1.3-SNAPSHOT.
> > Please object if you have a bigger change or any other reservations
> > regarding the feature freeze date!
> >
> > This is my current view of things on the release:
> >
> > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability (FLINK-4797)
> > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos (FLINK-1984)
> > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> > - RESOLVED Queryable State (FLINK-3779)
> > - RESOLVED Metrics in Webinterface (FLINK-4389)
> > - RESOLVED Kafka 0.10 support (FLINK-4035)
> > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> > Added by Stephan:
> > - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
> > over streams (FLINK-4391)
> > - NON-BLOCKING [beginning of next week] Unify Savepoints and Checkpoints
> > (FLINK-4484)
> > Added by Fabian:
> > - ONGOING [Pending PR] Clean up the packages of the Table API
> (FLINK-4704)
> >  Move Row to flink-core (
> > Added by Max:
> > - ONGOING [Pending PR] Change Akka configuration to allow accessing
> actors
> > from different URLs (FLINK-2821)
> >
> >
> > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]> wrote:
> >
> > > Hi Vijay!
> > >
> > > The workaround you suggest may be doable, but I am wondering how much
> > that
> > > helps, because the authorization feature would be incomplete like that
> > and
> > > thus of limited use.
> > >
> > > I would also assume that merging it properly and in full use after the
> > 1.2
> > > release would be a bit better - in general, we have often avoided last
> > > minute additions of sensitive and complex features.
> > >
> > > Do you think it is more urgent to have this in Flink?
> > >
> > > Best,
> > > Stephan
> > >
> > >
> > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]>
> > > wrote:
> > >
> > > > Max and Ufuk, I respect your concerns and fully understand the
> > importance
> > > > of the network layer stack in Flink code base. Will you be
> comfortable
> > to
> > > > merge the code if I remove the Netty layer changes and leave the rest
> > of
> > > > the code. We can address the Netty code changes post 1.2 release?
> > > >
> > > > Regards,
> > > > Vijay
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> > > > >
> > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> [hidden email])
> > > > wrote:
> > > > >>> It seems like we lack the resources for now to properly to take
> > > > >> care
> > > > >> of your pull request before the release. Unless someone from
> > > > >> the
> > > > >> community is really eager to help out here, I would be in favor
> > > > >> of
> > > > >> merging the pull request to the master after the release branch
> > > > >> has
> > > > >> been forked off. We should make sure it gets the attention it
> > deserves
> > > > >> then.
> > > > >
> > > > > Thanks Max! I fully agree with your reasoning. +1 to not include
> this
> > > in
> > > > 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> > > > >
> > > > > - Ufuk
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

[Aljoscha Krettek-2]

Yes, I'm confident that we can finish the tests until then and merge the
code.

On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]> wrote:

> Thank you for the update. Do you think you get it done until Monday
> evening?
>
> On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > Hi,
> > we're still working on making the backwards compatibility from 1.1
> > savepoints a reality. We have most of the code and some tests now but it
> > still needs some work. This is the issue that tracks the progress on the
> > operators that we would like to make backwards compatible:
> > https://issues.apache.org/jira/browse/FLINK-5292
> >
> > Cheers,
> > Aljoscha
> >
> > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]> wrote:
> >
> > > It will be pretty good if 1.2 branch could be forked off within this
> > week,
> > > and our guys working on FLIP-6  hope FLIP-6 branch could be merged into
> > > master as soon as possible.
> > >
> > > Best Regards,
> > >
> > > Feng Wang
> > >
> > > Alibaba
> > >
> > > ________________________________
> > > From: Robert Metzger <[hidden email]>
> > > Sent: Tuesday, December 13, 2016 4:58 AM
> > > To: [hidden email]
> > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> > >
> > > Thank you all for figuring out a solution for the security pull
> request.
> > >
> > >
> > > Lets try to get 1.2 feature freezed as fast as possible so that we can
> > > "unblock" waiting features like FLIP-6 and the remaining security
> > changes.
> > >
> > > *What do you think about Friday evening (6pm Berlin, 9am US west coast)
> > for
> > > feature freezing Flink 1.2?* (only bugfixes are allowed in afterwards)
> > > I'll then fork-off a "release-1.2" branch and update the version in
> > > "master" to 1.3-SNAPSHOT.
> > > Please object if you have a bigger change or any other reservations
> > > regarding the feature freeze date!
> > >
> > > This is my current view of things on the release:
> > >
> > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
> (FLINK-4797)
> > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
> (FLINK-1984)
> > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> > > - RESOLVED Queryable State (FLINK-3779)
> > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> > > Added by Stephan:
> > > - NON-BLOCKING [Pending PR] Provide support for asynchronous operations
> > > over streams (FLINK-4391)
> > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
> Checkpoints
> > > (FLINK-4484)
> > > Added by Fabian:
> > > - ONGOING [Pending PR] Clean up the packages of the Table API
> > (FLINK-4704)
> > >  Move Row to flink-core (
> > > Added by Max:
> > > - ONGOING [Pending PR] Change Akka configuration to allow accessing
> > actors
> > > from different URLs (FLINK-2821)
> > >
> > >
> > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]>
> wrote:
> > >
> > > > Hi Vijay!
> > > >
> > > > The workaround you suggest may be doable, but I am wondering how much
> > > that
> > > > helps, because the authorization feature would be incomplete like
> that
> > > and
> > > > thus of limited use.
> > > >
> > > > I would also assume that merging it properly and in full use after
> the
> > > 1.2
> > > > release would be a bit better - in general, we have often avoided
> last
> > > > minute additions of sensitive and complex features.
> > > >
> > > > Do you think it is more urgent to have this in Flink?
> > > >
> > > > Best,
> > > > Stephan
> > > >
> > > >
> > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Max and Ufuk, I respect your concerns and fully understand the
> > > importance
> > > > > of the network layer stack in Flink code base. Will you be
> > comfortable
> > > to
> > > > > merge the code if I remove the Netty layer changes and leave the
> rest
> > > of
> > > > > the code. We can address the Netty code changes post 1.2 release?
> > > > >
> > > > > Regards,
> > > > > Vijay
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]> wrote:
> > > > > >
> > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> > [hidden email])
> > > > > wrote:
> > > > > >>> It seems like we lack the resources for now to properly to take
> > > > > >> care
> > > > > >> of your pull request before the release. Unless someone from
> > > > > >> the
> > > > > >> community is really eager to help out here, I would be in favor
> > > > > >> of
> > > > > >> merging the pull request to the master after the release branch
> > > > > >> has
> > > > > >> been forked off. We should make sure it gets the attention it
> > > deserves
> > > > > >> then.
> > > > > >
> > > > > > Thanks Max! I fully agree with your reasoning. +1 to not include
> > this
> > > > in
> > > > > 1.2 now, but look at it afterwards. I hope that OK with you Vijay.
> > > > > >
> > > > > > - Ufuk
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

[Fabian Hueske-2]

Hi,

I merged the Table API refactoring changes:

- RESOLVED Clean up the packages of the Table API (FLINK-4704)
- RESOLVED Move Row to flink-core (FLINK-5186)

No blockers left from my side.

Cheers, Fabian

2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:

> Yes, I'm confident that we can finish the tests until then and merge the
> code.
>
> On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]> wrote:
>
> > Thank you for the update. Do you think you get it done until Monday
> > evening?
> >
> > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> > > Hi,
> > > we're still working on making the backwards compatibility from 1.1
> > > savepoints a reality. We have most of the code and some tests now but
> it
> > > still needs some work. This is the issue that tracks the progress on
> the
> > > operators that we would like to make backwards compatible:
> > > https://issues.apache.org/jira/browse/FLINK-5292
> > >
> > > Cheers,
> > > Aljoscha
> > >
> > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]> wrote:
> > >
> > > > It will be pretty good if 1.2 branch could be forked off within this
> > > week,
> > > > and our guys working on FLIP-6  hope FLIP-6 branch could be merged
> into
> > > > master as soon as possible.
> > > >
> > > > Best Regards,
> > > >
> > > > Feng Wang
> > > >
> > > > Alibaba
> > > >
> > > > ________________________________
> > > > From: Robert Metzger <[hidden email]>
> > > > Sent: Tuesday, December 13, 2016 4:58 AM
> > > > To: [hidden email]
> > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> > > >
> > > > Thank you all for figuring out a solution for the security pull
> > request.
> > > >
> > > >
> > > > Lets try to get 1.2 feature freezed as fast as possible so that we
> can
> > > > "unblock" waiting features like FLIP-6 and the remaining security
> > > changes.
> > > >
> > > > *What do you think about Friday evening (6pm Berlin, 9am US west
> coast)
> > > for
> > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
> afterwards)
> > > > I'll then fork-off a "release-1.2" branch and update the version in
> > > > "master" to 1.3-SNAPSHOT.
> > > > Please object if you have a bigger change or any other reservations
> > > > regarding the feature freeze date!
> > > >
> > > > This is my current view of things on the release:
> > > >
> > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
> > (FLINK-4797)
> > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
> > (FLINK-1984)
> > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> > > > - RESOLVED Queryable State (FLINK-3779)
> > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> > > > Added by Stephan:
> > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
> operations
> > > > over streams (FLINK-4391)
> > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
> > Checkpoints
> > > > (FLINK-4484)
> > > > Added by Fabian:
> > > > - ONGOING [Pending PR] Clean up the packages of the Table API
> > > (FLINK-4704)
> > > >  Move Row to flink-core (
> > > > Added by Max:
> > > > - ONGOING [Pending PR] Change Akka configuration to allow accessing
> > > actors
> > > > from different URLs (FLINK-2821)
> > > >
> > > >
> > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]>
> > wrote:
> > > >
> > > > > Hi Vijay!
> > > > >
> > > > > The workaround you suggest may be doable, but I am wondering how
> much
> > > > that
> > > > > helps, because the authorization feature would be incomplete like
> > that
> > > > and
> > > > > thus of limited use.
> > > > >
> > > > > I would also assume that merging it properly and in full use after
> > the
> > > > 1.2
> > > > > release would be a bit better - in general, we have often avoided
> > last
> > > > > minute additions of sensitive and complex features.
> > > > >
> > > > > Do you think it is more urgent to have this in Flink?
> > > > >
> > > > > Best,
> > > > > Stephan
> > > > >
> > > > >
> > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
> <[hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > Max and Ufuk, I respect your concerns and fully understand the
> > > > importance
> > > > > > of the network layer stack in Flink code base. Will you be
> > > comfortable
> > > > to
> > > > > > merge the code if I remove the Netty layer changes and leave the
> > rest
> > > > of
> > > > > > the code. We can address the Netty code changes post 1.2 release?
> > > > > >
> > > > > > Regards,
> > > > > > Vijay
> > > > > >
> > > > > > Sent from my iPhone
> > > > > >
> > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]>
> wrote:
> > > > > > >
> > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> > > [hidden email])
> > > > > > wrote:
> > > > > > >>> It seems like we lack the resources for now to properly to
> take
> > > > > > >> care
> > > > > > >> of your pull request before the release. Unless someone from
> > > > > > >> the
> > > > > > >> community is really eager to help out here, I would be in
> favor
> > > > > > >> of
> > > > > > >> merging the pull request to the master after the release
> branch
> > > > > > >> has
> > > > > > >> been forked off. We should make sure it gets the attention it
> > > > deserves
> > > > > > >> then.
> > > > > > >
> > > > > > > Thanks Max! I fully agree with your reasoning. +1 to not
> include
> > > this
> > > > > in
> > > > > > 1.2 now, but look at it afterwards. I hope that OK with you
> Vijay.
> > > > > > >
> > > > > > > - Ufuk
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Robert Metzger]

Thank you Aljoscha and Fabian for the updates.
I propose *Monday evening (6pm Berlin, 9am US west coast) for feature
freezing Flink 1.2 *then. This means that I'll create a release-1.2 fork
and create a 1.2 RC0 (non-voting) release candidate for testing.

I don't think that I'll create the first (voting) RC until January because
of christmas and new years activities. Most of the committers I know are
out of office during these 1,5 weeks.



On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]> wrote:

> Hi,
>
> I merged the Table API refactoring changes:
>
> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
> - RESOLVED Move Row to flink-core (FLINK-5186)
>
> No blockers left from my side.
>
> Cheers, Fabian
>
> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
>
> > Yes, I'm confident that we can finish the tests until then and merge the
> > code.
> >
> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]> wrote:
> >
> > > Thank you for the update. Do you think you get it done until Monday
> > > evening?
> > >
> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <[hidden email]
> >
> > > wrote:
> > >
> > > > Hi,
> > > > we're still working on making the backwards compatibility from 1.1
> > > > savepoints a reality. We have most of the code and some tests now but
> > it
> > > > still needs some work. This is the issue that tracks the progress on
> > the
> > > > operators that we would like to make backwards compatible:
> > > > https://issues.apache.org/jira/browse/FLINK-5292
> > > >
> > > > Cheers,
> > > > Aljoscha
> > > >
> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]>
> wrote:
> > > >
> > > > > It will be pretty good if 1.2 branch could be forked off within
> this
> > > > week,
> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be merged
> > into
> > > > > master as soon as possible.
> > > > >
> > > > > Best Regards,
> > > > >
> > > > > Feng Wang
> > > > >
> > > > > Alibaba
> > > > >
> > > > > ________________________________
> > > > > From: Robert Metzger <[hidden email]>
> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
> > > > > To: [hidden email]
> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> > > > >
> > > > > Thank you all for figuring out a solution for the security pull
> > > request.
> > > > >
> > > > >
> > > > > Lets try to get 1.2 feature freezed as fast as possible so that we
> > can
> > > > > "unblock" waiting features like FLIP-6 and the remaining security
> > > > changes.
> > > > >
> > > > > *What do you think about Friday evening (6pm Berlin, 9am US west
> > coast)
> > > > for
> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
> > afterwards)
> > > > > I'll then fork-off a "release-1.2" branch and update the version in
> > > > > "master" to 1.3-SNAPSHOT.
> > > > > Please object if you have a bigger change or any other reservations
> > > > > regarding the feature freeze date!
> > > > >
> > > > > This is my current view of things on the release:
> > > > >
> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> > > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
> > > (FLINK-4797)
> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
> > > (FLINK-1984)
> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> > > > > - RESOLVED Queryable State (FLINK-3779)
> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691, FLIP-11)
> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> > > > > Added by Stephan:
> > > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
> > operations
> > > > > over streams (FLINK-4391)
> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
> > > Checkpoints
> > > > > (FLINK-4484)
> > > > > Added by Fabian:
> > > > > - ONGOING [Pending PR] Clean up the packages of the Table API
> > > > (FLINK-4704)
> > > > >  Move Row to flink-core (
> > > > > Added by Max:
> > > > > - ONGOING [Pending PR] Change Akka configuration to allow accessing
> > > > actors
> > > > > from different URLs (FLINK-2821)
> > > > >
> > > > >
> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Hi Vijay!
> > > > > >
> > > > > > The workaround you suggest may be doable, but I am wondering how
> > much
> > > > > that
> > > > > > helps, because the authorization feature would be incomplete like
> > > that
> > > > > and
> > > > > > thus of limited use.
> > > > > >
> > > > > > I would also assume that merging it properly and in full use
> after
> > > the
> > > > > 1.2
> > > > > > release would be a bit better - in general, we have often avoided
> > > last
> > > > > > minute additions of sensitive and complex features.
> > > > > >
> > > > > > Do you think it is more urgent to have this in Flink?
> > > > > >
> > > > > > Best,
> > > > > > Stephan
> > > > > >
> > > > > >
> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
> > <[hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Max and Ufuk, I respect your concerns and fully understand the
> > > > > importance
> > > > > > > of the network layer stack in Flink code base. Will you be
> > > > comfortable
> > > > > to
> > > > > > > merge the code if I remove the Netty layer changes and leave
> the
> > > rest
> > > > > of
> > > > > > > the code. We can address the Netty code changes post 1.2
> release?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Vijay
> > > > > > >
> > > > > > > Sent from my iPhone
> > > > > > >
> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]>
> > wrote:
> > > > > > > >
> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> > > > [hidden email])
> > > > > > > wrote:
> > > > > > > >>> It seems like we lack the resources for now to properly to
> > take
> > > > > > > >> care
> > > > > > > >> of your pull request before the release. Unless someone from
> > > > > > > >> the
> > > > > > > >> community is really eager to help out here, I would be in
> > favor
> > > > > > > >> of
> > > > > > > >> merging the pull request to the master after the release
> > branch
> > > > > > > >> has
> > > > > > > >> been forked off. We should make sure it gets the attention
> it
> > > > > deserves
> > > > > > > >> then.
> > > > > > > >
> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to not
> > include
> > > > this
> > > > > > in
> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with you
> > Vijay.
> > > > > > > >
> > > > > > > > - Ufuk
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Robert Metzger]

Quick update here: I talked to Aljoscha offline, and the backwards
compatibility is still being tested (there were some bugs identified while
writing the tests).

Also, Stephan made some fixes to the build infrastructure (
https://github.com/apache/flink/pull/3029) that would be good to be
included into the release branch.
To finally get the FLIP-6 branch merged to master, I'm considering
branching off the 1.2 release later today. It will be a little bit more
overhead for Stephan and Aljoscha, but it will unblock all features waiting
for a Flink 1.3 master.




On Fri, Dec 16, 2016 at 6:05 PM, Robert Metzger <[hidden email]> wrote:

> Thank you Aljoscha and Fabian for the updates.
> I propose *Monday evening (6pm Berlin, 9am US west coast) for feature
> freezing Flink 1.2 *then. This means that I'll create a release-1.2 fork
> and create a 1.2 RC0 (non-voting) release candidate for testing.
>
> I don't think that I'll create the first (voting) RC until January because
> of christmas and new years activities. Most of the committers I know are
> out of office during these 1,5 weeks.
>
>
>
> On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]> wrote:
>
>> Hi,
>>
>> I merged the Table API refactoring changes:
>>
>> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
>> - RESOLVED Move Row to flink-core (FLINK-5186)
>>
>> No blockers left from my side.
>>
>> Cheers, Fabian
>>
>> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
>>
>> > Yes, I'm confident that we can finish the tests until then and merge the
>> > code.
>> >
>> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]> wrote:
>> >
>> > > Thank you for the update. Do you think you get it done until Monday
>> > > evening?
>> > >
>> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <
>> [hidden email]>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > > we're still working on making the backwards compatibility from 1.1
>> > > > savepoints a reality. We have most of the code and some tests now
>> but
>> > it
>> > > > still needs some work. This is the issue that tracks the progress on
>> > the
>> > > > operators that we would like to make backwards compatible:
>> > > > https://issues.apache.org/jira/browse/FLINK-5292
>> > > >
>> > > > Cheers,
>> > > > Aljoscha
>> > > >
>> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]>
>> wrote:
>> > > >
>> > > > > It will be pretty good if 1.2 branch could be forked off within
>> this
>> > > > week,
>> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be merged
>> > into
>> > > > > master as soon as possible.
>> > > > >
>> > > > > Best Regards,
>> > > > >
>> > > > > Feng Wang
>> > > > >
>> > > > > Alibaba
>> > > > >
>> > > > > ________________________________
>> > > > > From: Robert Metzger <[hidden email]>
>> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
>> > > > > To: [hidden email]
>> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
>> > > > >
>> > > > > Thank you all for figuring out a solution for the security pull
>> > > request.
>> > > > >
>> > > > >
>> > > > > Lets try to get 1.2 feature freezed as fast as possible so that we
>> > can
>> > > > > "unblock" waiting features like FLIP-6 and the remaining security
>> > > > changes.
>> > > > >
>> > > > > *What do you think about Friday evening (6pm Berlin, 9am US west
>> > coast)
>> > > > for
>> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
>> > afterwards)
>> > > > > I'll then fork-off a "release-1.2" branch and update the version
>> in
>> > > > > "master" to 1.3-SNAPSHOT.
>> > > > > Please object if you have a bigger change or any other
>> reservations
>> > > > > regarding the feature freeze date!
>> > > > >
>> > > > > This is my current view of things on the release:
>> > > > >
>> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
>> > > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
>> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
>> > > (FLINK-4797)
>> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
>> > > (FLINK-1984)
>> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
>> > > > > - RESOLVED Queryable State (FLINK-3779)
>> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
>> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
>> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691,
>> FLIP-11)
>> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
>> > > > > Added by Stephan:
>> > > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
>> > operations
>> > > > > over streams (FLINK-4391)
>> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
>> > > Checkpoints
>> > > > > (FLINK-4484)
>> > > > > Added by Fabian:
>> > > > > - ONGOING [Pending PR] Clean up the packages of the Table API
>> > > > (FLINK-4704)
>> > > > >  Move Row to flink-core (
>> > > > > Added by Max:
>> > > > > - ONGOING [Pending PR] Change Akka configuration to allow
>> accessing
>> > > > actors
>> > > > > from different URLs (FLINK-2821)
>> > > > >
>> > > > >
>> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]>
>> > > wrote:
>> > > > >
>> > > > > > Hi Vijay!
>> > > > > >
>> > > > > > The workaround you suggest may be doable, but I am wondering how
>> > much
>> > > > > that
>> > > > > > helps, because the authorization feature would be incomplete
>> like
>> > > that
>> > > > > and
>> > > > > > thus of limited use.
>> > > > > >
>> > > > > > I would also assume that merging it properly and in full use
>> after
>> > > the
>> > > > > 1.2
>> > > > > > release would be a bit better - in general, we have often
>> avoided
>> > > last
>> > > > > > minute additions of sensitive and complex features.
>> > > > > >
>> > > > > > Do you think it is more urgent to have this in Flink?
>> > > > > >
>> > > > > > Best,
>> > > > > > Stephan
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
>> > <[hidden email]
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Max and Ufuk, I respect your concerns and fully understand the
>> > > > > importance
>> > > > > > > of the network layer stack in Flink code base. Will you be
>> > > > comfortable
>> > > > > to
>> > > > > > > merge the code if I remove the Netty layer changes and leave
>> the
>> > > rest
>> > > > > of
>> > > > > > > the code. We can address the Netty code changes post 1.2
>> release?
>> > > > > > >
>> > > > > > > Regards,
>> > > > > > > Vijay
>> > > > > > >
>> > > > > > > Sent from my iPhone
>> > > > > > >
>> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]>
>> > wrote:
>> > > > > > > >
>> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
>> > > > [hidden email])
>> > > > > > > wrote:
>> > > > > > > >>> It seems like we lack the resources for now to properly to
>> > take
>> > > > > > > >> care
>> > > > > > > >> of your pull request before the release. Unless someone
>> from
>> > > > > > > >> the
>> > > > > > > >> community is really eager to help out here, I would be in
>> > favor
>> > > > > > > >> of
>> > > > > > > >> merging the pull request to the master after the release
>> > branch
>> > > > > > > >> has
>> > > > > > > >> been forked off. We should make sure it gets the attention
>> it
>> > > > > deserves
>> > > > > > > >> then.
>> > > > > > > >
>> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to not
>> > include
>> > > > this
>> > > > > > in
>> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with you
>> > Vijay.
>> > > > > > > >
>> > > > > > > > - Ufuk
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

[Aljoscha Krettek-2]

I just merged the most important backwards compatibility changes, with
tests.

I think this one is still a blocker:
https://issues.apache.org/jira/browse/FLINK-5320 (WindowedStream.fold()
cannot be used). And this one is a potential blocker for some users:
https://issues.apache.org/jira/browse/FLINK-5363.

IMHO, we can cut the branch today and I'll get them in on master and both
the 1.2 branch. What do you think?

On Tue, 20 Dec 2016 at 15:24 Robert Metzger <[hidden email]> wrote:

> Quick update here: I talked to Aljoscha offline, and the backwards
> compatibility is still being tested (there were some bugs identified while
> writing the tests).
>
> Also, Stephan made some fixes to the build infrastructure (
> https://github.com/apache/flink/pull/3029) that would be good to be
> included into the release branch.
> To finally get the FLIP-6 branch merged to master, I'm considering
> branching off the 1.2 release later today. It will be a little bit more
> overhead for Stephan and Aljoscha, but it will unblock all features waiting
> for a Flink 1.3 master.
>
>
>
>
> On Fri, Dec 16, 2016 at 6:05 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > Thank you Aljoscha and Fabian for the updates.
> > I propose *Monday evening (6pm Berlin, 9am US west coast) for feature
> > freezing Flink 1.2 *then. This means that I'll create a release-1.2 fork
> > and create a 1.2 RC0 (non-voting) release candidate for testing.
> >
> > I don't think that I'll create the first (voting) RC until January
> because
> > of christmas and new years activities. Most of the committers I know are
> > out of office during these 1,5 weeks.
> >
> >
> >
> > On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]>
> wrote:
> >
> >> Hi,
> >>
> >> I merged the Table API refactoring changes:
> >>
> >> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
> >> - RESOLVED Move Row to flink-core (FLINK-5186)
> >>
> >> No blockers left from my side.
> >>
> >> Cheers, Fabian
> >>
> >> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
> >>
> >> > Yes, I'm confident that we can finish the tests until then and merge
> the
> >> > code.
> >> >
> >> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]>
> wrote:
> >> >
> >> > > Thank you for the update. Do you think you get it done until Monday
> >> > > evening?
> >> > >
> >> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <
> >> [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > Hi,
> >> > > > we're still working on making the backwards compatibility from 1.1
> >> > > > savepoints a reality. We have most of the code and some tests now
> >> but
> >> > it
> >> > > > still needs some work. This is the issue that tracks the progress
> on
> >> > the
> >> > > > operators that we would like to make backwards compatible:
> >> > > > https://issues.apache.org/jira/browse/FLINK-5292
> >> > > >
> >> > > > Cheers,
> >> > > > Aljoscha
> >> > > >
> >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]>
> >> wrote:
> >> > > >
> >> > > > > It will be pretty good if 1.2 branch could be forked off within
> >> this
> >> > > > week,
> >> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be
> merged
> >> > into
> >> > > > > master as soon as possible.
> >> > > > >
> >> > > > > Best Regards,
> >> > > > >
> >> > > > > Feng Wang
> >> > > > >
> >> > > > > Alibaba
> >> > > > >
> >> > > > > ________________________________
> >> > > > > From: Robert Metzger <[hidden email]>
> >> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
> >> > > > > To: [hidden email]
> >> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> >> > > > >
> >> > > > > Thank you all for figuring out a solution for the security pull
> >> > > request.
> >> > > > >
> >> > > > >
> >> > > > > Lets try to get 1.2 feature freezed as fast as possible so that
> we
> >> > can
> >> > > > > "unblock" waiting features like FLIP-6 and the remaining
> security
> >> > > > changes.
> >> > > > >
> >> > > > > *What do you think about Friday evening (6pm Berlin, 9am US west
> >> > coast)
> >> > > > for
> >> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
> >> > afterwards)
> >> > > > > I'll then fork-off a "release-1.2" branch and update the version
> >> in
> >> > > > > "master" to 1.3-SNAPSHOT.
> >> > > > > Please object if you have a bigger change or any other
> >> reservations
> >> > > > > regarding the feature freeze date!
> >> > > > >
> >> > > > > This is my current view of things on the release:
> >> > > > >
> >> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> >> > > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> >> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
> >> > > (FLINK-4797)
> >> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
> >> > > (FLINK-1984)
> >> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> >> > > > > - RESOLVED Queryable State (FLINK-3779)
> >> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> >> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> >> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691,
> >> FLIP-11)
> >> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> >> > > > > Added by Stephan:
> >> > > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
> >> > operations
> >> > > > > over streams (FLINK-4391)
> >> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
> >> > > Checkpoints
> >> > > > > (FLINK-4484)
> >> > > > > Added by Fabian:
> >> > > > > - ONGOING [Pending PR] Clean up the packages of the Table API
> >> > > > (FLINK-4704)
> >> > > > >  Move Row to flink-core (
> >> > > > > Added by Max:
> >> > > > > - ONGOING [Pending PR] Change Akka configuration to allow
> >> accessing
> >> > > > actors
> >> > > > > from different URLs (FLINK-2821)
> >> > > > >
> >> > > > >
> >> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <[hidden email]
> >
> >> > > wrote:
> >> > > > >
> >> > > > > > Hi Vijay!
> >> > > > > >
> >> > > > > > The workaround you suggest may be doable, but I am wondering
> how
> >> > much
> >> > > > > that
> >> > > > > > helps, because the authorization feature would be incomplete
> >> like
> >> > > that
> >> > > > > and
> >> > > > > > thus of limited use.
> >> > > > > >
> >> > > > > > I would also assume that merging it properly and in full use
> >> after
> >> > > the
> >> > > > > 1.2
> >> > > > > > release would be a bit better - in general, we have often
> >> avoided
> >> > > last
> >> > > > > > minute additions of sensitive and complex features.
> >> > > > > >
> >> > > > > > Do you think it is more urgent to have this in Flink?
> >> > > > > >
> >> > > > > > Best,
> >> > > > > > Stephan
> >> > > > > >
> >> > > > > >
> >> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
> >> > <[hidden email]
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Max and Ufuk, I respect your concerns and fully understand
> the
> >> > > > > importance
> >> > > > > > > of the network layer stack in Flink code base. Will you be
> >> > > > comfortable
> >> > > > > to
> >> > > > > > > merge the code if I remove the Netty layer changes and leave
> >> the
> >> > > rest
> >> > > > > of
> >> > > > > > > the code. We can address the Netty code changes post 1.2
> >> release?
> >> > > > > > >
> >> > > > > > > Regards,
> >> > > > > > > Vijay
> >> > > > > > >
> >> > > > > > > Sent from my iPhone
> >> > > > > > >
> >> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <[hidden email]>
> >> > wrote:
> >> > > > > > > >
> >> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> >> > > > [hidden email])
> >> > > > > > > wrote:
> >> > > > > > > >>> It seems like we lack the resources for now to properly
> to
> >> > take
> >> > > > > > > >> care
> >> > > > > > > >> of your pull request before the release. Unless someone
> >> from
> >> > > > > > > >> the
> >> > > > > > > >> community is really eager to help out here, I would be in
> >> > favor
> >> > > > > > > >> of
> >> > > > > > > >> merging the pull request to the master after the release
> >> > branch
> >> > > > > > > >> has
> >> > > > > > > >> been forked off. We should make sure it gets the
> attention
> >> it
> >> > > > > deserves
> >> > > > > > > >> then.
> >> > > > > > > >
> >> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to not
> >> > include
> >> > > > this
> >> > > > > > in
> >> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with you
> >> > Vijay.
> >> > > > > > > >
> >> > > > > > > > - Ufuk
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

[Robert Metzger]

Hi Aljoscha,
thanks a lot for resolving this last big release blocker!

I'll definitively fork off in the release branch in the next few hours.
Stephan asked me to wait for a few more minutes for his maven changes.

The JIRAs you've mentioned seem to be pretty isolated, so its fine to merge
them afterwards to both branches.

Regards,
Robert


On Tue, Dec 20, 2016 at 4:17 PM, Aljoscha Krettek <[hidden email]>
wrote:

> I just merged the most important backwards compatibility changes, with
> tests.
>
> I think this one is still a blocker:
> https://issues.apache.org/jira/browse/FLINK-5320 (WindowedStream.fold()
> cannot be used). And this one is a potential blocker for some users:
> https://issues.apache.org/jira/browse/FLINK-5363.
>
> IMHO, we can cut the branch today and I'll get them in on master and both
> the 1.2 branch. What do you think?
>
> On Tue, 20 Dec 2016 at 15:24 Robert Metzger <[hidden email]> wrote:
>
> > Quick update here: I talked to Aljoscha offline, and the backwards
> > compatibility is still being tested (there were some bugs identified
> while
> > writing the tests).
> >
> > Also, Stephan made some fixes to the build infrastructure (
> > https://github.com/apache/flink/pull/3029) that would be good to be
> > included into the release branch.
> > To finally get the FLIP-6 branch merged to master, I'm considering
> > branching off the 1.2 release later today. It will be a little bit more
> > overhead for Stephan and Aljoscha, but it will unblock all features
> waiting
> > for a Flink 1.3 master.
> >
> >
> >
> >
> > On Fri, Dec 16, 2016 at 6:05 PM, Robert Metzger <[hidden email]>
> > wrote:
> >
> > > Thank you Aljoscha and Fabian for the updates.
> > > I propose *Monday evening (6pm Berlin, 9am US west coast) for feature
> > > freezing Flink 1.2 *then. This means that I'll create a release-1.2
> fork
> > > and create a 1.2 RC0 (non-voting) release candidate for testing.
> > >
> > > I don't think that I'll create the first (voting) RC until January
> > because
> > > of christmas and new years activities. Most of the committers I know
> are
> > > out of office during these 1,5 weeks.
> > >
> > >
> > >
> > > On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]>
> > wrote:
> > >
> > >> Hi,
> > >>
> > >> I merged the Table API refactoring changes:
> > >>
> > >> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
> > >> - RESOLVED Move Row to flink-core (FLINK-5186)
> > >>
> > >> No blockers left from my side.
> > >>
> > >> Cheers, Fabian
> > >>
> > >> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
> > >>
> > >> > Yes, I'm confident that we can finish the tests until then and merge
> > the
> > >> > code.
> > >> >
> > >> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]>
> > wrote:
> > >> >
> > >> > > Thank you for the update. Do you think you get it done until
> Monday
> > >> > > evening?
> > >> > >
> > >> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <
> > >> [hidden email]>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi,
> > >> > > > we're still working on making the backwards compatibility from
> 1.1
> > >> > > > savepoints a reality. We have most of the code and some tests
> now
> > >> but
> > >> > it
> > >> > > > still needs some work. This is the issue that tracks the
> progress
> > on
> > >> > the
> > >> > > > operators that we would like to make backwards compatible:
> > >> > > > https://issues.apache.org/jira/browse/FLINK-5292
> > >> > > >
> > >> > > > Cheers,
> > >> > > > Aljoscha
> > >> > > >
> > >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]>
> > >> wrote:
> > >> > > >
> > >> > > > > It will be pretty good if 1.2 branch could be forked off
> within
> > >> this
> > >> > > > week,
> > >> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be
> > merged
> > >> > into
> > >> > > > > master as soon as possible.
> > >> > > > >
> > >> > > > > Best Regards,
> > >> > > > >
> > >> > > > > Feng Wang
> > >> > > > >
> > >> > > > > Alibaba
> > >> > > > >
> > >> > > > > ________________________________
> > >> > > > > From: Robert Metzger <[hidden email]>
> > >> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
> > >> > > > > To: [hidden email]
> > >> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> > >> > > > >
> > >> > > > > Thank you all for figuring out a solution for the security
> pull
> > >> > > request.
> > >> > > > >
> > >> > > > >
> > >> > > > > Lets try to get 1.2 feature freezed as fast as possible so
> that
> > we
> > >> > can
> > >> > > > > "unblock" waiting features like FLIP-6 and the remaining
> > security
> > >> > > > changes.
> > >> > > > >
> > >> > > > > *What do you think about Friday evening (6pm Berlin, 9am US
> west
> > >> > coast)
> > >> > > > for
> > >> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
> > >> > afterwards)
> > >> > > > > I'll then fork-off a "release-1.2" branch and update the
> version
> > >> in
> > >> > > > > "master" to 1.3-SNAPSHOT.
> > >> > > > > Please object if you have a bigger change or any other
> > >> reservations
> > >> > > > > regarding the feature freeze date!
> > >> > > > >
> > >> > > > > This is my current view of things on the release:
> > >> > > > >
> > >> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> > >> > > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
> > >> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
> > >> > > (FLINK-4797)
> > >> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
> > >> > > (FLINK-1984)
> > >> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> > >> > > > > - RESOLVED Queryable State (FLINK-3779)
> > >> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> > >> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> > >> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691,
> > >> FLIP-11)
> > >> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> > >> > > > > Added by Stephan:
> > >> > > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
> > >> > operations
> > >> > > > > over streams (FLINK-4391)
> > >> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
> > >> > > Checkpoints
> > >> > > > > (FLINK-4484)
> > >> > > > > Added by Fabian:
> > >> > > > > - ONGOING [Pending PR] Clean up the packages of the Table API
> > >> > > > (FLINK-4704)
> > >> > > > >  Move Row to flink-core (
> > >> > > > > Added by Max:
> > >> > > > > - ONGOING [Pending PR] Change Akka configuration to allow
> > >> accessing
> > >> > > > actors
> > >> > > > > from different URLs (FLINK-2821)
> > >> > > > >
> > >> > > > >
> > >> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <
> [hidden email]
> > >
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > Hi Vijay!
> > >> > > > > >
> > >> > > > > > The workaround you suggest may be doable, but I am wondering
> > how
> > >> > much
> > >> > > > > that
> > >> > > > > > helps, because the authorization feature would be incomplete
> > >> like
> > >> > > that
> > >> > > > > and
> > >> > > > > > thus of limited use.
> > >> > > > > >
> > >> > > > > > I would also assume that merging it properly and in full use
> > >> after
> > >> > > the
> > >> > > > > 1.2
> > >> > > > > > release would be a bit better - in general, we have often
> > >> avoided
> > >> > > last
> > >> > > > > > minute additions of sensitive and complex features.
> > >> > > > > >
> > >> > > > > > Do you think it is more urgent to have this in Flink?
> > >> > > > > >
> > >> > > > > > Best,
> > >> > > > > > Stephan
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
> > >> > <[hidden email]
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Max and Ufuk, I respect your concerns and fully understand
> > the
> > >> > > > > importance
> > >> > > > > > > of the network layer stack in Flink code base. Will you be
> > >> > > > comfortable
> > >> > > > > to
> > >> > > > > > > merge the code if I remove the Netty layer changes and
> leave
> > >> the
> > >> > > rest
> > >> > > > > of
> > >> > > > > > > the code. We can address the Netty code changes post 1.2
> > >> release?
> > >> > > > > > >
> > >> > > > > > > Regards,
> > >> > > > > > > Vijay
> > >> > > > > > >
> > >> > > > > > > Sent from my iPhone
> > >> > > > > > >
> > >> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <
> [hidden email]>
> > >> > wrote:
> > >> > > > > > > >
> > >> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> > >> > > > [hidden email])
> > >> > > > > > > wrote:
> > >> > > > > > > >>> It seems like we lack the resources for now to
> properly
> > to
> > >> > take
> > >> > > > > > > >> care
> > >> > > > > > > >> of your pull request before the release. Unless someone
> > >> from
> > >> > > > > > > >> the
> > >> > > > > > > >> community is really eager to help out here, I would be
> in
> > >> > favor
> > >> > > > > > > >> of
> > >> > > > > > > >> merging the pull request to the master after the
> release
> > >> > branch
> > >> > > > > > > >> has
> > >> > > > > > > >> been forked off. We should make sure it gets the
> > attention
> > >> it
> > >> > > > > deserves
> > >> > > > > > > >> then.
> > >> > > > > > > >
> > >> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to not
> > >> > include
> > >> > > > this
> > >> > > > > > in
> > >> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with
> you
> > >> > Vijay.
> > >> > > > > > > >
> > >> > > > > > > > - Ufuk
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

[Robert Metzger]

Okay, I forked off a release-1.2 branch and updated the version in master
to 1.3-SNAPSHOT.
I'll create the first RC (non voting) in the next few days.

On Tue, Dec 20, 2016 at 4:39 PM, Robert Metzger <[hidden email]> wrote:

> Hi Aljoscha,
> thanks a lot for resolving this last big release blocker!
>
> I'll definitively fork off in the release branch in the next few hours.
> Stephan asked me to wait for a few more minutes for his maven changes.
>
> The JIRAs you've mentioned seem to be pretty isolated, so its fine to
> merge them afterwards to both branches.
>
> Regards,
> Robert
>
>
> On Tue, Dec 20, 2016 at 4:17 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> I just merged the most important backwards compatibility changes, with
>> tests.
>>
>> I think this one is still a blocker:
>> https://issues.apache.org/jira/browse/FLINK-5320 (WindowedStream.fold()
>> cannot be used). And this one is a potential blocker for some users:
>> https://issues.apache.org/jira/browse/FLINK-5363.
>>
>> IMHO, we can cut the branch today and I'll get them in on master and both
>> the 1.2 branch. What do you think?
>>
>> On Tue, 20 Dec 2016 at 15:24 Robert Metzger <[hidden email]> wrote:
>>
>> > Quick update here: I talked to Aljoscha offline, and the backwards
>> > compatibility is still being tested (there were some bugs identified
>> while
>> > writing the tests).
>> >
>> > Also, Stephan made some fixes to the build infrastructure (
>> > https://github.com/apache/flink/pull/3029) that would be good to be
>> > included into the release branch.
>> > To finally get the FLIP-6 branch merged to master, I'm considering
>> > branching off the 1.2 release later today. It will be a little bit more
>> > overhead for Stephan and Aljoscha, but it will unblock all features
>> waiting
>> > for a Flink 1.3 master.
>> >
>> >
>> >
>> >
>> > On Fri, Dec 16, 2016 at 6:05 PM, Robert Metzger <[hidden email]>
>> > wrote:
>> >
>> > > Thank you Aljoscha and Fabian for the updates.
>> > > I propose *Monday evening (6pm Berlin, 9am US west coast) for feature
>> > > freezing Flink 1.2 *then. This means that I'll create a release-1.2
>> fork
>> > > and create a 1.2 RC0 (non-voting) release candidate for testing.
>> > >
>> > > I don't think that I'll create the first (voting) RC until January
>> > because
>> > > of christmas and new years activities. Most of the committers I know
>> are
>> > > out of office during these 1,5 weeks.
>> > >
>> > >
>> > >
>> > > On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]>
>> > wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> I merged the Table API refactoring changes:
>> > >>
>> > >> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
>> > >> - RESOLVED Move Row to flink-core (FLINK-5186)
>> > >>
>> > >> No blockers left from my side.
>> > >>
>> > >> Cheers, Fabian
>> > >>
>> > >> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
>> > >>
>> > >> > Yes, I'm confident that we can finish the tests until then and
>> merge
>> > the
>> > >> > code.
>> > >> >
>> > >> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]>
>> > wrote:
>> > >> >
>> > >> > > Thank you for the update. Do you think you get it done until
>> Monday
>> > >> > > evening?
>> > >> > >
>> > >> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <
>> > >> [hidden email]>
>> > >> > > wrote:
>> > >> > >
>> > >> > > > Hi,
>> > >> > > > we're still working on making the backwards compatibility from
>> 1.1
>> > >> > > > savepoints a reality. We have most of the code and some tests
>> now
>> > >> but
>> > >> > it
>> > >> > > > still needs some work. This is the issue that tracks the
>> progress
>> > on
>> > >> > the
>> > >> > > > operators that we would like to make backwards compatible:
>> > >> > > > https://issues.apache.org/jira/browse/FLINK-5292
>> > >> > > >
>> > >> > > > Cheers,
>> > >> > > > Aljoscha
>> > >> > > >
>> > >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <[hidden email]>
>> > >> wrote:
>> > >> > > >
>> > >> > > > > It will be pretty good if 1.2 branch could be forked off
>> within
>> > >> this
>> > >> > > > week,
>> > >> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be
>> > merged
>> > >> > into
>> > >> > > > > master as soon as possible.
>> > >> > > > >
>> > >> > > > > Best Regards,
>> > >> > > > >
>> > >> > > > > Feng Wang
>> > >> > > > >
>> > >> > > > > Alibaba
>> > >> > > > >
>> > >> > > > > ________________________________
>> > >> > > > > From: Robert Metzger <[hidden email]>
>> > >> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
>> > >> > > > > To: [hidden email]
>> > >> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
>> > >> > > > >
>> > >> > > > > Thank you all for figuring out a solution for the security
>> pull
>> > >> > > request.
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > Lets try to get 1.2 feature freezed as fast as possible so
>> that
>> > we
>> > >> > can
>> > >> > > > > "unblock" waiting features like FLIP-6 and the remaining
>> > security
>> > >> > > > changes.
>> > >> > > > >
>> > >> > > > > *What do you think about Friday evening (6pm Berlin, 9am US
>> west
>> > >> > coast)
>> > >> > > > for
>> > >> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
>> > >> > afterwards)
>> > >> > > > > I'll then fork-off a "release-1.2" branch and update the
>> version
>> > >> in
>> > >> > > > > "master" to 1.3-SNAPSHOT.
>> > >> > > > > Please object if you have a bigger change or any other
>> > >> reservations
>> > >> > > > > regarding the feature freeze date!
>> > >> > > > >
>> > >> > > > > This is my current view of things on the release:
>> > >> > > > >
>> > >> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
>> > >> > > > > - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379)
>> > >> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards compatability
>> > >> > > (FLINK-4797)
>> > >> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache Mesos
>> > >> > > (FLINK-1984)
>> > >> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
>> > >> > > > > - RESOLVED Queryable State (FLINK-3779)
>> > >> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
>> > >> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
>> > >> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691,
>> > >> FLIP-11)
>> > >> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
>> > >> > > > > Added by Stephan:
>> > >> > > > > - NON-BLOCKING [Pending PR] Provide support for asynchronous
>> > >> > operations
>> > >> > > > > over streams (FLINK-4391)
>> > >> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints and
>> > >> > > Checkpoints
>> > >> > > > > (FLINK-4484)
>> > >> > > > > Added by Fabian:
>> > >> > > > > - ONGOING [Pending PR] Clean up the packages of the Table API
>> > >> > > > (FLINK-4704)
>> > >> > > > >  Move Row to flink-core (
>> > >> > > > > Added by Max:
>> > >> > > > > - ONGOING [Pending PR] Change Akka configuration to allow
>> > >> accessing
>> > >> > > > actors
>> > >> > > > > from different URLs (FLINK-2821)
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <
>> [hidden email]
>> > >
>> > >> > > wrote:
>> > >> > > > >
>> > >> > > > > > Hi Vijay!
>> > >> > > > > >
>> > >> > > > > > The workaround you suggest may be doable, but I am
>> wondering
>> > how
>> > >> > much
>> > >> > > > > that
>> > >> > > > > > helps, because the authorization feature would be
>> incomplete
>> > >> like
>> > >> > > that
>> > >> > > > > and
>> > >> > > > > > thus of limited use.
>> > >> > > > > >
>> > >> > > > > > I would also assume that merging it properly and in full
>> use
>> > >> after
>> > >> > > the
>> > >> > > > > 1.2
>> > >> > > > > > release would be a bit better - in general, we have often
>> > >> avoided
>> > >> > > last
>> > >> > > > > > minute additions of sensitive and complex features.
>> > >> > > > > >
>> > >> > > > > > Do you think it is more urgent to have this in Flink?
>> > >> > > > > >
>> > >> > > > > > Best,
>> > >> > > > > > Stephan
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
>> > >> > <[hidden email]
>> > >> > > >
>> > >> > > > > > wrote:
>> > >> > > > > >
>> > >> > > > > > > Max and Ufuk, I respect your concerns and fully
>> understand
>> > the
>> > >> > > > > importance
>> > >> > > > > > > of the network layer stack in Flink code base. Will you
>> be
>> > >> > > > comfortable
>> > >> > > > > to
>> > >> > > > > > > merge the code if I remove the Netty layer changes and
>> leave
>> > >> the
>> > >> > > rest
>> > >> > > > > of
>> > >> > > > > > > the code. We can address the Netty code changes post 1.2
>> > >> release?
>> > >> > > > > > >
>> > >> > > > > > > Regards,
>> > >> > > > > > > Vijay
>> > >> > > > > > >
>> > >> > > > > > > Sent from my iPhone
>> > >> > > > > > >
>> > >> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <
>> [hidden email]>
>> > >> > wrote:
>> > >> > > > > > > >
>> > >> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
>> > >> > > > [hidden email])
>> > >> > > > > > > wrote:
>> > >> > > > > > > >>> It seems like we lack the resources for now to
>> properly
>> > to
>> > >> > take
>> > >> > > > > > > >> care
>> > >> > > > > > > >> of your pull request before the release. Unless
>> someone
>> > >> from
>> > >> > > > > > > >> the
>> > >> > > > > > > >> community is really eager to help out here, I would
>> be in
>> > >> > favor
>> > >> > > > > > > >> of
>> > >> > > > > > > >> merging the pull request to the master after the
>> release
>> > >> > branch
>> > >> > > > > > > >> has
>> > >> > > > > > > >> been forked off. We should make sure it gets the
>> > attention
>> > >> it
>> > >> > > > > deserves
>> > >> > > > > > > >> then.
>> > >> > > > > > > >
>> > >> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to
>> not
>> > >> > include
>> > >> > > > this
>> > >> > > > > > in
>> > >> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with
>> you
>> > >> > Vijay.
>> > >> > > > > > > >
>> > >> > > > > > > > - Ufuk
>> > >> > > > > > > >
>> > >> > > > > > > >
>> > >> > > > > > >
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

[Stephan Ewen]

Very nice, thanks Robert!

On Tue, Dec 20, 2016 at 6:02 PM, Robert Metzger <[hidden email]> wrote:

> Okay, I forked off a release-1.2 branch and updated the version in master
> to 1.3-SNAPSHOT.
> I'll create the first RC (non voting) in the next few days.
>
> On Tue, Dec 20, 2016 at 4:39 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > Hi Aljoscha,
> > thanks a lot for resolving this last big release blocker!
> >
> > I'll definitively fork off in the release branch in the next few hours.
> > Stephan asked me to wait for a few more minutes for his maven changes.
> >
> > The JIRAs you've mentioned seem to be pretty isolated, so its fine to
> > merge them afterwards to both branches.
> >
> > Regards,
> > Robert
> >
> >
> > On Tue, Dec 20, 2016 at 4:17 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> >> I just merged the most important backwards compatibility changes, with
> >> tests.
> >>
> >> I think this one is still a blocker:
> >> https://issues.apache.org/jira/browse/FLINK-5320 (WindowedStream.fold()
> >> cannot be used). And this one is a potential blocker for some users:
> >> https://issues.apache.org/jira/browse/FLINK-5363.
> >>
> >> IMHO, we can cut the branch today and I'll get them in on master and
> both
> >> the 1.2 branch. What do you think?
> >>
> >> On Tue, 20 Dec 2016 at 15:24 Robert Metzger <[hidden email]>
> wrote:
> >>
> >> > Quick update here: I talked to Aljoscha offline, and the backwards
> >> > compatibility is still being tested (there were some bugs identified
> >> while
> >> > writing the tests).
> >> >
> >> > Also, Stephan made some fixes to the build infrastructure (
> >> > https://github.com/apache/flink/pull/3029) that would be good to be
> >> > included into the release branch.
> >> > To finally get the FLIP-6 branch merged to master, I'm considering
> >> > branching off the 1.2 release later today. It will be a little bit
> more
> >> > overhead for Stephan and Aljoscha, but it will unblock all features
> >> waiting
> >> > for a Flink 1.3 master.
> >> >
> >> >
> >> >
> >> >
> >> > On Fri, Dec 16, 2016 at 6:05 PM, Robert Metzger <[hidden email]>
> >> > wrote:
> >> >
> >> > > Thank you Aljoscha and Fabian for the updates.
> >> > > I propose *Monday evening (6pm Berlin, 9am US west coast) for
> feature
> >> > > freezing Flink 1.2 *then. This means that I'll create a release-1.2
> >> fork
> >> > > and create a 1.2 RC0 (non-voting) release candidate for testing.
> >> > >
> >> > > I don't think that I'll create the first (voting) RC until January
> >> > because
> >> > > of christmas and new years activities. Most of the committers I know
> >> are
> >> > > out of office during these 1,5 weeks.
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Dec 16, 2016 at 5:55 PM, Fabian Hueske <[hidden email]>
> >> > wrote:
> >> > >
> >> > >> Hi,
> >> > >>
> >> > >> I merged the Table API refactoring changes:
> >> > >>
> >> > >> - RESOLVED Clean up the packages of the Table API (FLINK-4704)
> >> > >> - RESOLVED Move Row to flink-core (FLINK-5186)
> >> > >>
> >> > >> No blockers left from my side.
> >> > >>
> >> > >> Cheers, Fabian
> >> > >>
> >> > >> 2016-12-16 17:47 GMT+01:00 Aljoscha Krettek <[hidden email]>:
> >> > >>
> >> > >> > Yes, I'm confident that we can finish the tests until then and
> >> merge
> >> > the
> >> > >> > code.
> >> > >> >
> >> > >> > On Fri, Dec 16, 2016, 17:41 Robert Metzger <[hidden email]>
> >> > wrote:
> >> > >> >
> >> > >> > > Thank you for the update. Do you think you get it done until
> >> Monday
> >> > >> > > evening?
> >> > >> > >
> >> > >> > > On Fri, Dec 16, 2016 at 5:23 PM, Aljoscha Krettek <
> >> > >> [hidden email]>
> >> > >> > > wrote:
> >> > >> > >
> >> > >> > > > Hi,
> >> > >> > > > we're still working on making the backwards compatibility
> from
> >> 1.1
> >> > >> > > > savepoints a reality. We have most of the code and some tests
> >> now
> >> > >> but
> >> > >> > it
> >> > >> > > > still needs some work. This is the issue that tracks the
> >> progress
> >> > on
> >> > >> > the
> >> > >> > > > operators that we would like to make backwards compatible:
> >> > >> > > > https://issues.apache.org/jira/browse/FLINK-5292
> >> > >> > > >
> >> > >> > > > Cheers,
> >> > >> > > > Aljoscha
> >> > >> > > >
> >> > >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang <
> [hidden email]>
> >> > >> wrote:
> >> > >> > > >
> >> > >> > > > > It will be pretty good if 1.2 branch could be forked off
> >> within
> >> > >> this
> >> > >> > > > week,
> >> > >> > > > > and our guys working on FLIP-6  hope FLIP-6 branch could be
> >> > merged
> >> > >> > into
> >> > >> > > > > master as soon as possible.
> >> > >> > > > >
> >> > >> > > > > Best Regards,
> >> > >> > > > >
> >> > >> > > > > Feng Wang
> >> > >> > > > >
> >> > >> > > > > Alibaba
> >> > >> > > > >
> >> > >> > > > > ________________________________
> >> > >> > > > > From: Robert Metzger <[hidden email]>
> >> > >> > > > > Sent: Tuesday, December 13, 2016 4:58 AM
> >> > >> > > > > To: [hidden email]
> >> > >> > > > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2
> >> > >> > > > >
> >> > >> > > > > Thank you all for figuring out a solution for the security
> >> pull
> >> > >> > > request.
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > > Lets try to get 1.2 feature freezed as fast as possible so
> >> that
> >> > we
> >> > >> > can
> >> > >> > > > > "unblock" waiting features like FLIP-6 and the remaining
> >> > security
> >> > >> > > > changes.
> >> > >> > > > >
> >> > >> > > > > *What do you think about Friday evening (6pm Berlin, 9am US
> >> west
> >> > >> > coast)
> >> > >> > > > for
> >> > >> > > > > feature freezing Flink 1.2?* (only bugfixes are allowed in
> >> > >> > afterwards)
> >> > >> > > > > I'll then fork-off a "release-1.2" branch and update the
> >> version
> >> > >> in
> >> > >> > > > > "master" to 1.3-SNAPSHOT.
> >> > >> > > > > Please object if you have a bigger change or any other
> >> > >> reservations
> >> > >> > > > > regarding the feature freeze date!
> >> > >> > > > >
> >> > >> > > > > This is my current view of things on the release:
> >> > >> > > > >
> >> > >> > > > > - RESOLVED dynamic Scaling / Key Groups (FLINK-3755)
> >> > >> > > > > - RESOLVED Add Rescalable Non-Partitioned State
> (FLINK-4379)
> >> > >> > > > > - UNRESOLVED Add Flink 1.1 savepoint backwards
> compatability
> >> > >> > > (FLINK-4797)
> >> > >> > > > > - RESOLVED [Split for 1.3] Integrate Flink with Apache
> Mesos
> >> > >> > > (FLINK-1984)
> >> > >> > > > > - UNDER DISCUSSION Secure Data Access (FLINK-3930)
> >> > >> > > > > - RESOLVED Queryable State (FLINK-3779)
> >> > >> > > > > - RESOLVED Metrics in Webinterface (FLINK-4389)
> >> > >> > > > > - RESOLVED Kafka 0.10 support (FLINK-4035)
> >> > >> > > > > - RESOLVED Table API: Group Window Aggregates (FLINK-4691,
> >> > >> FLIP-11)
> >> > >> > > > > - RESOLVED Table API: Scalar Functions (FLINK-3097)
> >> > >> > > > > Added by Stephan:
> >> > >> > > > > - NON-BLOCKING [Pending PR] Provide support for
> asynchronous
> >> > >> > operations
> >> > >> > > > > over streams (FLINK-4391)
> >> > >> > > > > - NON-BLOCKING [beginning of next week] Unify Savepoints
> and
> >> > >> > > Checkpoints
> >> > >> > > > > (FLINK-4484)
> >> > >> > > > > Added by Fabian:
> >> > >> > > > > - ONGOING [Pending PR] Clean up the packages of the Table
> API
> >> > >> > > > (FLINK-4704)
> >> > >> > > > >  Move Row to flink-core (
> >> > >> > > > > Added by Max:
> >> > >> > > > > - ONGOING [Pending PR] Change Akka configuration to allow
> >> > >> accessing
> >> > >> > > > actors
> >> > >> > > > > from different URLs (FLINK-2821)
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > > On Mon, Dec 12, 2016 at 5:43 PM, Stephan Ewen <
> >> [hidden email]
> >> > >
> >> > >> > > wrote:
> >> > >> > > > >
> >> > >> > > > > > Hi Vijay!
> >> > >> > > > > >
> >> > >> > > > > > The workaround you suggest may be doable, but I am
> >> wondering
> >> > how
> >> > >> > much
> >> > >> > > > > that
> >> > >> > > > > > helps, because the authorization feature would be
> >> incomplete
> >> > >> like
> >> > >> > > that
> >> > >> > > > > and
> >> > >> > > > > > thus of limited use.
> >> > >> > > > > >
> >> > >> > > > > > I would also assume that merging it properly and in full
> >> use
> >> > >> after
> >> > >> > > the
> >> > >> > > > > 1.2
> >> > >> > > > > > release would be a bit better - in general, we have often
> >> > >> avoided
> >> > >> > > last
> >> > >> > > > > > minute additions of sensitive and complex features.
> >> > >> > > > > >
> >> > >> > > > > > Do you think it is more urgent to have this in Flink?
> >> > >> > > > > >
> >> > >> > > > > > Best,
> >> > >> > > > > > Stephan
> >> > >> > > > > >
> >> > >> > > > > >
> >> > >> > > > > > On Mon, Dec 12, 2016 at 2:49 PM, Vijay
> >> > >> > <[hidden email]
> >> > >> > > >
> >> > >> > > > > > wrote:
> >> > >> > > > > >
> >> > >> > > > > > > Max and Ufuk, I respect your concerns and fully
> >> understand
> >> > the
> >> > >> > > > > importance
> >> > >> > > > > > > of the network layer stack in Flink code base. Will you
> >> be
> >> > >> > > > comfortable
> >> > >> > > > > to
> >> > >> > > > > > > merge the code if I remove the Netty layer changes and
> >> leave
> >> > >> the
> >> > >> > > rest
> >> > >> > > > > of
> >> > >> > > > > > > the code. We can address the Netty code changes post
> 1.2
> >> > >> release?
> >> > >> > > > > > >
> >> > >> > > > > > > Regards,
> >> > >> > > > > > > Vijay
> >> > >> > > > > > >
> >> > >> > > > > > > Sent from my iPhone
> >> > >> > > > > > >
> >> > >> > > > > > > > On Dec 12, 2016, at 3:38 AM, Ufuk Celebi <
> >> [hidden email]>
> >> > >> > wrote:
> >> > >> > > > > > > >
> >> > >> > > > > > > > On 12 December 2016 at 12:30:31, Maximilian Michels (
> >> > >> > > > [hidden email])
> >> > >> > > > > > > wrote:
> >> > >> > > > > > > >>> It seems like we lack the resources for now to
> >> properly
> >> > to
> >> > >> > take
> >> > >> > > > > > > >> care
> >> > >> > > > > > > >> of your pull request before the release. Unless
> >> someone
> >> > >> from
> >> > >> > > > > > > >> the
> >> > >> > > > > > > >> community is really eager to help out here, I would
> >> be in
> >> > >> > favor
> >> > >> > > > > > > >> of
> >> > >> > > > > > > >> merging the pull request to the master after the
> >> release
> >> > >> > branch
> >> > >> > > > > > > >> has
> >> > >> > > > > > > >> been forked off. We should make sure it gets the
> >> > attention
> >> > >> it
> >> > >> > > > > deserves
> >> > >> > > > > > > >> then.
> >> > >> > > > > > > >
> >> > >> > > > > > > > Thanks Max! I fully agree with your reasoning. +1 to
> >> not
> >> > >> > include
> >> > >> > > > this
> >> > >> > > > > > in
> >> > >> > > > > > > 1.2 now, but look at it afterwards. I hope that OK with
> >> you
> >> > >> > Vijay.
> >> > >> > > > > > > >
> >> > >> > > > > > > > - Ufuk
> >> > >> > > > > > > >
> >> > >> > > > > > > >
> >> > >> > > > > > >
> >> > >> > > > > > >
> >> > >> > > > > >
> >> > >> > > > >
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >
> >
>