[DISCUSS] Support Local Aggregation in Flink

[Forward Xu]

>
> hi

+1 nice work

best

forwardxu

[mayo zhang]

 + 1 for  this feature which is great helpful in product situations and look forward to see it as soon as possible.

> 在 2019年6月6日，下午4:58，qianjin Xu <[hidden email]> 写道：
>
>>
>> hi
>
> +1 nice work
>
> best
>
> forwardxu

[vino yang]

Hi all,

Thanks for all the feedback and comments.

Since the thread of this feature has been presented about one week in the
dev mailing list and has got much support from the community, I have
created a new JIRA feature issue[1] to track it and I will split subtasks
soon.

We can move further discussion under related issues.

The suggestions and opinions are still welcome and appreciated.

Best,
Vino

[1]: https://issues.apache.org/jira/browse/FLINK-12786

mayo zhang <[hidden email]> 于2019年6月7日周五 下午5:25写道：

>  + 1 for  this feature which is great helpful in product situations and
> look forward to see it as soon as possible.
>
> > 在 2019年6月6日，下午4:58，qianjin Xu <[hidden email]> 写道：
> >
> >>
> >> hi
> >
> > +1 nice work
> >
> > best
> >
> > forwardxu
>
>
>

[Aljoscha Krettek-2]

Hi,

I think this proposed change is big enough to warrant a FLIP [1], which should have a voting process as described in that link before the FLIP is accepted. I’m writing this because such a bigger change has the possibility of languishing for a long time due to lack of PMC/committer bandwidth. A vote indicates interest by the PMC/committers to put work into this effort. I mentioned earlier that a lot of PMCs/committers are currently busy with other efforts around the new Table API Runner story.

I think there is not yet consensus on whether this is the right approach or whether keeping a local cache and flushing on receiving the checkpoint barrier are better for different approaches. I for one did not yet have time to think through the implications of adding such API.

That being said, I don’t think this is a bad proposal, just that I don’t know yet, and others might not know as well.

Best,
Aljoscha

[1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals <https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals>

> On 10. Jun 2019, at 04:30, vino yang <[hidden email]> wrote:
>
> Hi all,
>
> Thanks for all the feedback and comments.
>
> Since the thread of this feature has been presented about one week in the dev mailing list and has got much support from the community, I have created a new JIRA feature issue[1] to track it and I will split subtasks soon.
>
> We can move further discussion under related issues.
>
> The suggestions and opinions are still welcome and appreciated.
>
> Best,
> Vino
>
> [1]: https://issues.apache.org/jira/browse/FLINK-12786 <https://issues.apache.org/jira/browse/FLINK-12786>
> mayo zhang <[hidden email] <mailto:[hidden email]>> 于2019年6月7日周五 下午5:25写道：
>  + 1 for  this feature which is great helpful in product situations and look forward to see it as soon as possible.
>
> > 在 2019年6月6日，下午4:58，qianjin Xu <[hidden email] <mailto:[hidden email]>> 写道：
> >
> >>
> >> hi
> >
> > +1 nice work
> >
> > best
> >
> > forwardxu
>
>

[vino yang]

Hi Aljoscha,

I am happy to create a FLIP and have a voting process for this feature. I
have already sent a mail to apply for the wiki permissions.

Once I get the permission, will start the next step. When it is ready, I
will ping you again.

Best,
Vino

Aljoscha Krettek <[hidden email]> 于2019年6月11日周二 下午10:57写道：

> Hi,
>
> I think this proposed change is big enough to warrant a FLIP [1], which
> should have a voting process as described in that link before the FLIP is
> accepted. I’m writing this because such a bigger change has the possibility
> of languishing for a long time due to lack of PMC/committer bandwidth. A
> vote indicates interest by the PMC/committers to put work into this effort.
> I mentioned earlier that a lot of PMCs/committers are currently busy with
> other efforts around the new Table API Runner story.
>
> I think there is not yet consensus on whether this is the right approach
> or whether keeping a local cache and flushing on receiving the checkpoint
> barrier are better for different approaches. I for one did not yet have
> time to think through the implications of adding such API.
>
> That being said, I don’t think this is a bad proposal, just that I don’t
> know yet, and others might not know as well.
>
> Best,
> Aljoscha
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>
> On 10. Jun 2019, at 04:30, vino yang <[hidden email]> wrote:
>
> Hi all,
>
> Thanks for all the feedback and comments.
>
> Since the thread of this feature has been presented about one week in the
> dev mailing list and has got much support from the community, I have
> created a new JIRA feature issue[1] to track it and I will split subtasks
> soon.
>
> We can move further discussion under related issues.
>
> The suggestions and opinions are still welcome and appreciated.
>
> Best,
> Vino
>
> [1]: https://issues.apache.org/jira/browse/FLINK-12786
>
> mayo zhang <[hidden email]> 于2019年6月7日周五 下午5:25写道：
>
>>  + 1 for  this feature which is great helpful in product situations and
>> look forward to see it as soon as possible.
>>
>> > 在 2019年6月6日，下午4:58，qianjin Xu <[hidden email]> 写道：
>> >
>> >>
>> >> hi
>> >
>> > +1 nice work
>> >
>> > best
>> >
>> > forwardxu
>>
>>
>>
>

[vino yang]

Hi all,

Thanks for all the feedback and comments.

Based on @Aljoscha Krettek <[hidden email]> 's suggestion, I have
created a new FLIP: "FLIP-44: Support Local Aggregation in Flink" and
started a new mail thread for it in the dev mailing list.

So any further feedback and discussion can be moved to the new thread.

Best,
Vino



vino yang <[hidden email]> 于2019年6月12日周三 上午11:33写道：

> Hi Aljoscha,
>
> I am happy to create a FLIP and have a voting process for this feature. I
> have already sent a mail to apply for the wiki permissions.
>
> Once I get the permission, will start the next step. When it is ready, I
> will ping you again.
>
> Best,
> Vino
>
> Aljoscha Krettek <[hidden email]> 于2019年6月11日周二 下午10:57写道：
>
>> Hi,
>>
>> I think this proposed change is big enough to warrant a FLIP [1], which
>> should have a voting process as described in that link before the FLIP is
>> accepted. I’m writing this because such a bigger change has the possibility
>> of languishing for a long time due to lack of PMC/committer bandwidth. A
>> vote indicates interest by the PMC/committers to put work into this effort.
>> I mentioned earlier that a lot of PMCs/committers are currently busy with
>> other efforts around the new Table API Runner story.
>>
>> I think there is not yet consensus on whether this is the right approach
>> or whether keeping a local cache and flushing on receiving the checkpoint
>> barrier are better for different approaches. I for one did not yet have
>> time to think through the implications of adding such API.
>>
>> That being said, I don’t think this is a bad proposal, just that I don’t
>> know yet, and others might not know as well.
>>
>> Best,
>> Aljoscha
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>>
>> On 10. Jun 2019, at 04:30, vino yang <[hidden email]> wrote:
>>
>> Hi all,
>>
>> Thanks for all the feedback and comments.
>>
>> Since the thread of this feature has been presented about one week in the
>> dev mailing list and has got much support from the community, I have
>> created a new JIRA feature issue[1] to track it and I will split subtasks
>> soon.
>>
>> We can move further discussion under related issues.
>>
>> The suggestions and opinions are still welcome and appreciated.
>>
>> Best,
>> Vino
>>
>> [1]: https://issues.apache.org/jira/browse/FLINK-12786
>>
>> mayo zhang <[hidden email]> 于2019年6月7日周五 下午5:25写道：
>>
>>>  + 1 for  this feature which is great helpful in product situations and
>>> look forward to see it as soon as possible.
>>>
>>> > 在 2019年6月6日，下午4:58，qianjin Xu <[hidden email]> 写道：
>>> >
>>> >>
>>> >> hi
>>> >
>>> > +1 nice work
>>> >
>>> > best
>>> >
>>> > forwardxu
>>>
>>>
>>>
>>