(DEPRECATED) Apache Flink Mailing List archive.

[DISCUSS] Creating a new repo to host Flink benchmarks

Classic

List

Threaded

13 messages Options

Yun Tang

[DISCUSS] Creating a new repo to host Flink benchmarks

Hi Flink devs,

As Flink develops rapidly with more and more features added, how to ensure no performance regression existed has become more and more important. And we would like to create a new repo under apache project to host previous flink-benchmarks [1] repo, which is inspired when we discuss under FLINK-16850 [2]

Some background context on flink-benchmarks, for those who are not familiar with the project yet:

- Current flink-benchmarks does not align with the Flink release, which lead developers not easy to verify
performance at specific Flink version because current flink-benchmarks always depends on the latest interfaces.
- Above problem could be solved well if we could ensure flink-benchmarks also create release branch when we
releasing Flink. However, current flink-benchmarks repo is hosted under dataArtisans (the former name of
ververica) project, which is not involved in Flink release manual [3]. We propose to promote this repo under
apache project so that release manager could have the right to release on flink-benchmarks.
- The reason why we not involve flink-benchmarks into the apache/flink repo is because it heavily depends on
JMH [4], which is under GPLv2 license.

What do you think?

Best,
Yun Tang

[1] https://github.com/dataArtisans/flink-benchmarks
[2] https://issues.apache.org/jira/browse/FLINK-16850
[3] https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
[4] https://openjdk.java.net/projects/code-tools/jmh/

Piotr Nowojski-3

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Hi Yun Tang,

Thanks for proposing the idea. Since we can not include benchmarks in the Flink repository what you are proposing is the second best option.

+1 from my side for the proposal.

I think benchmarks have proven their value to justify this.

Piotrek

> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
>
> Hi Flink devs,
>
> As Flink develops rapidly with more and more features added, how to ensure no performance regression existed has become more and more important. And we would like to create a new repo under apache project to host previous flink-benchmarks [1] repo, which is inspired when we discuss under FLINK-16850 [2]
>
> Some background context on flink-benchmarks, for those who are not familiar with the project yet:
>
> - Current flink-benchmarks does not align with the Flink release, which lead developers not easy to verify
> performance at specific Flink version because current flink-benchmarks always depends on the latest interfaces.
> - Above problem could be solved well if we could ensure flink-benchmarks also create release branch when we
> releasing Flink. However, current flink-benchmarks repo is hosted under dataArtisans (the former name of
> ververica) project, which is not involved in Flink release manual [3]. We propose to promote this repo under
> apache project so that release manager could have the right to release on flink-benchmarks.
> - The reason why we not involve flink-benchmarks into the apache/flink repo is because it heavily depends on
> JMH [4], which is under GPLv2 license.
>
> What do you think?
>
> Best,
> Yun Tang
>
> [1] https://github.com/dataArtisans/flink-benchmarks
> [2] https://issues.apache.org/jira/browse/FLINK-16850
> [3] https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> [4] https://openjdk.java.net/projects/code-tools/jmh/
>
>

Till Rohrmann

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

I think it is a good idea to make the benchmarks available to the community
via a repo under the Apache project and to make updating it part of the
release process. Hence +1 for the proposal.

Cheers,
Till

On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]> wrote:

> Hi Yun Tang,
>
> Thanks for proposing the idea. Since we can not include benchmarks in the
> Flink repository what you are proposing is the second best option.
>
> +1 from my side for the proposal.
>
> I think benchmarks have proven their value to justify this.
>
> Piotrek
>
> > On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
> >
> > Hi Flink devs,
> >
> > As Flink develops rapidly with more and more features added, how to
> ensure no performance regression existed has become more and more
> important. And we would like to create a new repo under apache project to
> host previous flink-benchmarks [1] repo, which is inspired when we discuss
> under FLINK-16850 [2]
> >
> > Some background context on flink-benchmarks, for those who are not
> familiar with the project yet:
> >
> > - Current flink-benchmarks does not align with the Flink release, which
> lead developers not easy to verify
> > performance at specific Flink version because current flink-benchmarks
> always depends on the latest interfaces.
> > - Above problem could be solved well if we could ensure flink-benchmarks
> also create release branch when we
> > releasing Flink. However, current flink-benchmarks repo is hosted
> under dataArtisans (the former name of
> > ververica) project, which is not involved in Flink release manual [3].
> We propose to promote this repo under
> > apache project so that release manager could have the right to release
> on flink-benchmarks.
> > - The reason why we not involve flink-benchmarks into the apache/flink
> repo is because it heavily depends on
> > JMH [4], which is under GPLv2 license.
> >
> > What do you think?
> >
> > Best,
> > Yun Tang
> >
> > [1] https://github.com/dataArtisans/flink-benchmarks
> > [2] https://issues.apache.org/jira/browse/FLINK-16850
> > [3]
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> > [4] https://openjdk.java.net/projects/code-tools/jmh/
> >
> >
>
>

Robert Metzger

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 on creating the repo.

On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]> wrote:

> I think it is a good idea to make the benchmarks available to the community
> via a repo under the Apache project and to make updating it part of the
> release process. Hence +1 for the proposal.
>
> Cheers,
> Till
>
> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]> wrote:
>
> > Hi Yun Tang,
> >
> > Thanks for proposing the idea. Since we can not include benchmarks in the
> > Flink repository what you are proposing is the second best option.
> >
> > +1 from my side for the proposal.
> >
> > I think benchmarks have proven their value to justify this.
> >
> > Piotrek
> >
> > > On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
> > >
> > > Hi Flink devs,
> > >
> > > As Flink develops rapidly with more and more features added, how to
> > ensure no performance regression existed has become more and more
> > important. And we would like to create a new repo under apache project to
> > host previous flink-benchmarks [1] repo, which is inspired when we
> discuss
> > under FLINK-16850 [2]
> > >
> > > Some background context on flink-benchmarks, for those who are not
> > familiar with the project yet:
> > >
> > > - Current flink-benchmarks does not align with the Flink release, which
> > lead developers not easy to verify
> > > performance at specific Flink version because current
> flink-benchmarks
> > always depends on the latest interfaces.
> > > - Above problem could be solved well if we could ensure
> flink-benchmarks
> > also create release branch when we
> > > releasing Flink. However, current flink-benchmarks repo is hosted
> > under dataArtisans (the former name of
> > > ververica) project, which is not involved in Flink release manual
> [3].
> > We propose to promote this repo under
> > > apache project so that release manager could have the right to
> release
> > on flink-benchmarks.
> > > - The reason why we not involve flink-benchmarks into the apache/flink
> > repo is because it heavily depends on
> > > JMH [4], which is under GPLv2 license.
> > >
> > > What do you think?
> > >
> > > Best,
> > > Yun Tang
> > >
> > > [1] https://github.com/dataArtisans/flink-benchmarks
> > > [2] https://issues.apache.org/jira/browse/FLINK-16850
> > > [3]
> >
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> > > [4] https://openjdk.java.net/projects/code-tools/jmh/
> > >
> > >
> >
> >
>

Zhijiang(wangzhijiang999)

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 for the proposal.

Best,
Zhijiang

------------------------------------------------------------------
From:Robert Metzger <[hidden email]>
Send Time:2020 Apr. 10 (Fri.) 02:15
To:dev <[hidden email]>
Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 on creating the repo.

On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]> wrote:

Dian Fu-2

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 for this proposal.

> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]> 写道：
>
> +1 for the proposal.
>
> Best,
> Zhijiang
>
>
> ------------------------------------------------------------------
> From:Robert Metzger <[hidden email]>
> Send Time:2020 Apr. 10 (Fri.) 02:15
> To:dev <[hidden email]>
> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>
> +1 on creating the repo.
>
>
> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]> wrote:
>
>> I think it is a good idea to make the benchmarks available to the community
>> via a repo under the Apache project and to make updating it part of the
>> release process. Hence +1 for the proposal.
>>
>> Cheers,
>> Till
>>
>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]> wrote:
>>
>>> Hi Yun Tang,
>>>
>>> Thanks for proposing the idea. Since we can not include benchmarks in the
>>> Flink repository what you are proposing is the second best option.
>>>
>>> +1 from my side for the proposal.
>>>
>>> I think benchmarks have proven their value to justify this.
>>>
>>> Piotrek
>>>
>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
>>>>
>>>> Hi Flink devs,
>>>>
>>>> As Flink develops rapidly with more and more features added, how to
>>> ensure no performance regression existed has become more and more
>>> important. And we would like to create a new repo under apache project to
>>> host previous flink-benchmarks [1] repo, which is inspired when we
>> discuss
>>> under FLINK-16850 [2]
>>>>
>>>> Some background context on flink-benchmarks, for those who are not
>>> familiar with the project yet:
>>>>
>>>> - Current flink-benchmarks does not align with the Flink release, which
>>> lead developers not easy to verify
>>>> performance at specific Flink version because current
>> flink-benchmarks
>>> always depends on the latest interfaces.
>>>> - Above problem could be solved well if we could ensure
>> flink-benchmarks
>>> also create release branch when we
>>>> releasing Flink. However, current flink-benchmarks repo is hosted
>>> under dataArtisans (the former name of
>>>> ververica) project, which is not involved in Flink release manual
>> [3].
>>> We propose to promote this repo under
>>>> apache project so that release manager could have the right to
>> release
>>> on flink-benchmarks.
>>>> - The reason why we not involve flink-benchmarks into the apache/flink
>>> repo is because it heavily depends on
>>>> JMH [4], which is under GPLv2 license.
>>>>
>>>> What do you think?
>>>>
>>>> Best,
>>>> Yun Tang
>>>>
>>>> [1] https://github.com/dataArtisans/flink-benchmarks
>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
>>>> [3]
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
>>>>
>>>>
>>>
>>>
>>
>

Yu Li

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 for the proposal, thanks for driving this Yun.

Best Regards,
Yu

On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:

> +1 for this proposal.
>
> > 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]> 写道：
> >
> > +1 for the proposal.
> >
> > Best,
> > Zhijiang
> >
> >
> > ------------------------------------------------------------------
> > From:Robert Metzger <[hidden email]>
> > Send Time:2020 Apr. 10 (Fri.) 02:15
> > To:dev <[hidden email]>
> > Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
> >
> > +1 on creating the repo.
> >
> >
> > On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]>
> wrote:
> >
> >> I think it is a good idea to make the benchmarks available to the
> community
> >> via a repo under the Apache project and to make updating it part of the
> >> release process. Hence +1 for the proposal.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]>
> wrote:
> >>
> >>> Hi Yun Tang,
> >>>
> >>> Thanks for proposing the idea. Since we can not include benchmarks in
> the
> >>> Flink repository what you are proposing is the second best option.
> >>>
> >>> +1 from my side for the proposal.
> >>>
> >>> I think benchmarks have proven their value to justify this.
> >>>
> >>> Piotrek
> >>>
> >>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
> >>>>
> >>>> Hi Flink devs,
> >>>>
> >>>> As Flink develops rapidly with more and more features added, how to
> >>> ensure no performance regression existed has become more and more
> >>> important. And we would like to create a new repo under apache project
> to
> >>> host previous flink-benchmarks [1] repo, which is inspired when we
> >> discuss
> >>> under FLINK-16850 [2]
> >>>>
> >>>> Some background context on flink-benchmarks, for those who are not
> >>> familiar with the project yet:
> >>>>
> >>>> - Current flink-benchmarks does not align with the Flink release,
> which
> >>> lead developers not easy to verify
> >>>> performance at specific Flink version because current
> >> flink-benchmarks
> >>> always depends on the latest interfaces.
> >>>> - Above problem could be solved well if we could ensure
> >> flink-benchmarks
> >>> also create release branch when we
> >>>> releasing Flink. However, current flink-benchmarks repo is hosted
> >>> under dataArtisans (the former name of
> >>>> ververica) project, which is not involved in Flink release manual
> >> [3].
> >>> We propose to promote this repo under
> >>>> apache project so that release manager could have the right to
> >> release
> >>> on flink-benchmarks.
> >>>> - The reason why we not involve flink-benchmarks into the apache/flink
> >>> repo is because it heavily depends on
> >>>> JMH [4], which is under GPLv2 license.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Best,
> >>>> Yun Tang
> >>>>
> >>>> [1] https://github.com/dataArtisans/flink-benchmarks
> >>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
> >>>> [3]
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> >>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
> >>>>
> >>>>
> >>>
> >>>
> >>
> >
>
>

Yun Tang

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Hi Flink devs

Thanks for the feedbacks! Overall, everyone who replied is positive about creating such a separate repo to host the flink benchmarks.

Since such an action does not necessarily need a vote (as defined by the project bylaws), I'll proceed to ask PMC's help to create the repo under Apache project.

Best
Yun Tang

________________________________
From: Yu Li <[hidden email]>
Sent: Friday, April 10, 2020 21:03
To: dev <[hidden email]>
Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 for the proposal, thanks for driving this Yun.

Best Regards,
Yu

On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:

Chesnay Schepler-3

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Isn't this technically the adoption of a new codebase?

On 14/04/2020 10:06, Yun Tang wrote:

> Hi Flink devs
>
> Thanks for the feedbacks! Overall, everyone who replied is positive about creating such a separate repo to host the flink benchmarks.
>
> Since such an action does not necessarily need a vote (as defined by the project bylaws), I'll proceed to ask PMC's help to create the repo under Apache project.
>
> Best
> Yun Tang
>
> ________________________________
> From: Yu Li <[hidden email]>
> Sent: Friday, April 10, 2020 21:03
> To: dev <[hidden email]>
> Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>
> +1 for the proposal, thanks for driving this Yun.
>
> Best Regards,
> Yu
>
>
> On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:
>
>> +1 for this proposal.
>>
>>> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]> 写道：
>>>
>>> +1 for the proposal.
>>>
>>> Best,
>>> Zhijiang
>>>
>>>
>>> ------------------------------------------------------------------
>>> From:Robert Metzger <[hidden email]>
>>> Send Time:2020 Apr. 10 (Fri.) 02:15
>>> To:dev <[hidden email]>
>>> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>>>
>>> +1 on creating the repo.
>>>
>>>
>>> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]>
>> wrote:
>>>> I think it is a good idea to make the benchmarks available to the
>> community
>>>> via a repo under the Apache project and to make updating it part of the
>>>> release process. Hence +1 for the proposal.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]>
>> wrote:
>>>>> Hi Yun Tang,
>>>>>
>>>>> Thanks for proposing the idea. Since we can not include benchmarks in
>> the
>>>>> Flink repository what you are proposing is the second best option.
>>>>>
>>>>> +1 from my side for the proposal.
>>>>>
>>>>> I think benchmarks have proven their value to justify this.
>>>>>
>>>>> Piotrek
>>>>>
>>>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Flink devs,
>>>>>>
>>>>>> As Flink develops rapidly with more and more features added, how to
>>>>> ensure no performance regression existed has become more and more
>>>>> important. And we would like to create a new repo under apache project
>> to
>>>>> host previous flink-benchmarks [1] repo, which is inspired when we
>>>> discuss
>>>>> under FLINK-16850 [2]
>>>>>> Some background context on flink-benchmarks, for those who are not
>>>>> familiar with the project yet:
>>>>>> - Current flink-benchmarks does not align with the Flink release,
>> which
>>>>> lead developers not easy to verify
>>>>>> performance at specific Flink version because current
>>>> flink-benchmarks
>>>>> always depends on the latest interfaces.
>>>>>> - Above problem could be solved well if we could ensure
>>>> flink-benchmarks
>>>>> also create release branch when we
>>>>>> releasing Flink. However, current flink-benchmarks repo is hosted
>>>>> under dataArtisans (the former name of
>>>>>> ververica) project, which is not involved in Flink release manual
>>>> [3].
>>>>> We propose to promote this repo under
>>>>>> apache project so that release manager could have the right to
>>>> release
>>>>> on flink-benchmarks.
>>>>>> - The reason why we not involve flink-benchmarks into the apache/flink
>>>>> repo is because it heavily depends on
>>>>>> JMH [4], which is under GPLv2 license.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Best,
>>>>>> Yun Tang
>>>>>>
>>>>>> [1] https://github.com/dataArtisans/flink-benchmarks
>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
>>>>>> [3]
>> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
>>>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
>>>>>>
>>>>>>
>>>>>
>>

Tzu-Li (Gordon) Tai

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

+1 on creating a new repo, and migrating the flink-benchmarks there.

Regarding Chesnay's concern whether this requires a vote:
While the flink-benchmarks is indeed an existing external codebase, I'm not
entirely sure we do need a vote here.
The bylaws state that:
"Adoption of large existing external codebase. This refers to contributions
big enough that potentially change the shape and direction of the project
with massive restructuring and future maintenance commitment."

The Flink benchmarks should not be a significant change to the shape /
direction of the project.
What do others think?

On Tue, Apr 14, 2020 at 4:53 PM Chesnay Schepler <[hidden email]> wrote:

> Isn't this technically the adoption of a new codebase?
>
> On 14/04/2020 10:06, Yun Tang wrote:
> > Hi Flink devs
> >
> > Thanks for the feedbacks! Overall, everyone who replied is positive
> about creating such a separate repo to host the flink benchmarks.
> >
> > Since such an action does not necessarily need a vote (as defined by the
> project bylaws), I'll proceed to ask PMC's help to create the repo under
> Apache project.
> >
> > Best
> > Yun Tang
> >
> > ________________________________
> > From: Yu Li <[hidden email]>
> > Sent: Friday, April 10, 2020 21:03
> > To: dev <[hidden email]>
> > Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks
> >
> > +1 for the proposal, thanks for driving this Yun.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:
> >
> >> +1 for this proposal.
> >>
> >>> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]> 写道：
> >>>
> >>> +1 for the proposal.
> >>>
> >>> Best,
> >>> Zhijiang
> >>>
> >>>
> >>> ------------------------------------------------------------------
> >>> From:Robert Metzger <[hidden email]>
> >>> Send Time:2020 Apr. 10 (Fri.) 02:15
> >>> To:dev <[hidden email]>
> >>> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
> >>>
> >>> +1 on creating the repo.
> >>>
> >>>
> >>> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]>
> >> wrote:
> >>>> I think it is a good idea to make the benchmarks available to the
> >> community
> >>>> via a repo under the Apache project and to make updating it part of
> the
> >>>> release process. Hence +1 for the proposal.
> >>>>
> >>>> Cheers,
> >>>> Till
> >>>>
> >>>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]>
> >> wrote:
> >>>>> Hi Yun Tang,
> >>>>>
> >>>>> Thanks for proposing the idea. Since we can not include benchmarks in
> >> the
> >>>>> Flink repository what you are proposing is the second best option.
> >>>>>
> >>>>> +1 from my side for the proposal.
> >>>>>
> >>>>> I think benchmarks have proven their value to justify this.
> >>>>>
> >>>>> Piotrek
> >>>>>
> >>>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
> >>>>>>
> >>>>>> Hi Flink devs,
> >>>>>>
> >>>>>> As Flink develops rapidly with more and more features added, how to
> >>>>> ensure no performance regression existed has become more and more
> >>>>> important. And we would like to create a new repo under apache
> project
> >> to
> >>>>> host previous flink-benchmarks [1] repo, which is inspired when we
> >>>> discuss
> >>>>> under FLINK-16850 [2]
> >>>>>> Some background context on flink-benchmarks, for those who are not
> >>>>> familiar with the project yet:
> >>>>>> - Current flink-benchmarks does not align with the Flink release,
> >> which
> >>>>> lead developers not easy to verify
> >>>>>> performance at specific Flink version because current
> >>>> flink-benchmarks
> >>>>> always depends on the latest interfaces.
> >>>>>> - Above problem could be solved well if we could ensure
> >>>> flink-benchmarks
> >>>>> also create release branch when we
> >>>>>> releasing Flink. However, current flink-benchmarks repo is hosted
> >>>>> under dataArtisans (the former name of
> >>>>>> ververica) project, which is not involved in Flink release manual
> >>>> [3].
> >>>>> We propose to promote this repo under
> >>>>>> apache project so that release manager could have the right to
> >>>> release
> >>>>> on flink-benchmarks.
> >>>>>> - The reason why we not involve flink-benchmarks into the
> apache/flink
> >>>>> repo is because it heavily depends on
> >>>>>> JMH [4], which is under GPLv2 license.
> >>>>>>
> >>>>>> What do you think?
> >>>>>>
> >>>>>> Best,
> >>>>>> Yun Tang
> >>>>>>
> >>>>>> [1] https://github.com/dataArtisans/flink-benchmarks
> >>>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
> >>>>>> [3]
> >>
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> >>>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
> >>>>>>
> >>>>>>
> >>>>>
> >>
>
>

Tzu-Li (Gordon) Tai

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

There doesn't seem to be any objections in not considering the
flink-benchmarks migration as a significant change that requires a vote.

If no-one objects in the next days, I'll assume lazy consensus and create
the new repo the following Monday (April 20).

On Tue, Apr 14, 2020 at 5:13 PM Tzu-Li (Gordon) Tai <[hidden email]>
wrote:

> +1 on creating a new repo, and migrating the flink-benchmarks there.
>
> Regarding Chesnay's concern whether this requires a vote:
> While the flink-benchmarks is indeed an existing external codebase, I'm
> not entirely sure we do need a vote here.
> The bylaws state that:
> "Adoption of large existing external codebase. This refers to
> contributions big enough that potentially change the shape and direction of
> the project with massive restructuring and future maintenance commitment."
>
> The Flink benchmarks should not be a significant change to the shape /
> direction of the project.
> What do others think?
>
> On Tue, Apr 14, 2020 at 4:53 PM Chesnay Schepler <[hidden email]>
> wrote:
>
>> Isn't this technically the adoption of a new codebase?
>>
>> On 14/04/2020 10:06, Yun Tang wrote:
>> > Hi Flink devs
>> >
>> > Thanks for the feedbacks! Overall, everyone who replied is positive
>> about creating such a separate repo to host the flink benchmarks.
>> >
>> > Since such an action does not necessarily need a vote (as defined by
>> the project bylaws), I'll proceed to ask PMC's help to create the repo
>> under Apache project.
>> >
>> > Best
>> > Yun Tang
>> >
>> > ________________________________
>> > From: Yu Li <[hidden email]>
>> > Sent: Friday, April 10, 2020 21:03
>> > To: dev <[hidden email]>
>> > Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>> >
>> > +1 for the proposal, thanks for driving this Yun.
>> >
>> > Best Regards,
>> > Yu
>> >
>> >
>> > On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:
>> >
>> >> +1 for this proposal.
>> >>
>> >>> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]>
>> 写道：
>> >>>
>> >>> +1 for the proposal.
>> >>>
>> >>> Best,
>> >>> Zhijiang
>> >>>
>> >>>
>> >>> ------------------------------------------------------------------
>> >>> From:Robert Metzger <[hidden email]>
>> >>> Send Time:2020 Apr. 10 (Fri.) 02:15
>> >>> To:dev <[hidden email]>
>> >>> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>> >>>
>> >>> +1 on creating the repo.
>> >>>
>> >>>
>> >>> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]>
>> >> wrote:
>> >>>> I think it is a good idea to make the benchmarks available to the
>> >> community
>> >>>> via a repo under the Apache project and to make updating it part of
>> the
>> >>>> release process. Hence +1 for the proposal.
>> >>>>
>> >>>> Cheers,
>> >>>> Till
>> >>>>
>> >>>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]>
>> >> wrote:
>> >>>>> Hi Yun Tang,
>> >>>>>
>> >>>>> Thanks for proposing the idea. Since we can not include benchmarks
>> in
>> >> the
>> >>>>> Flink repository what you are proposing is the second best option.
>> >>>>>
>> >>>>> +1 from my side for the proposal.
>> >>>>>
>> >>>>> I think benchmarks have proven their value to justify this.
>> >>>>>
>> >>>>> Piotrek
>> >>>>>
>> >>>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
>> >>>>>>
>> >>>>>> Hi Flink devs,
>> >>>>>>
>> >>>>>> As Flink develops rapidly with more and more features added, how to
>> >>>>> ensure no performance regression existed has become more and more
>> >>>>> important. And we would like to create a new repo under apache
>> project
>> >> to
>> >>>>> host previous flink-benchmarks [1] repo, which is inspired when we
>> >>>> discuss
>> >>>>> under FLINK-16850 [2]
>> >>>>>> Some background context on flink-benchmarks, for those who are not
>> >>>>> familiar with the project yet:
>> >>>>>> - Current flink-benchmarks does not align with the Flink release,
>> >> which
>> >>>>> lead developers not easy to verify
>> >>>>>> performance at specific Flink version because current
>> >>>> flink-benchmarks
>> >>>>> always depends on the latest interfaces.
>> >>>>>> - Above problem could be solved well if we could ensure
>> >>>> flink-benchmarks
>> >>>>> also create release branch when we
>> >>>>>> releasing Flink. However, current flink-benchmarks repo is hosted
>> >>>>> under dataArtisans (the former name of
>> >>>>>> ververica) project, which is not involved in Flink release manual
>> >>>> [3].
>> >>>>> We propose to promote this repo under
>> >>>>>> apache project so that release manager could have the right to
>> >>>> release
>> >>>>> on flink-benchmarks.
>> >>>>>> - The reason why we not involve flink-benchmarks into the
>> apache/flink
>> >>>>> repo is because it heavily depends on
>> >>>>>> JMH [4], which is under GPLv2 license.
>> >>>>>>
>> >>>>>> What do you think?
>> >>>>>>
>> >>>>>> Best,
>> >>>>>> Yun Tang
>> >>>>>>
>> >>>>>> [1] https://github.com/dataArtisans/flink-benchmarks
>> >>>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
>> >>>>>> [3]
>> >>
>> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
>> >>>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>
>>
>>

Tzu-Li (Gordon) Tai

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Repository has been created:
https://github.com/apache/flink-benchmarks

On Sat, Apr 18, 2020 at 5:10 PM Tzu-Li (Gordon) Tai <[hidden email]>
wrote:

> There doesn't seem to be any objections in not considering the
> flink-benchmarks migration as a significant change that requires a vote.
>
> If no-one objects in the next days, I'll assume lazy consensus and create
> the new repo the following Monday (April 20).
>
> On Tue, Apr 14, 2020 at 5:13 PM Tzu-Li (Gordon) Tai <[hidden email]>
> wrote:
>
>> +1 on creating a new repo, and migrating the flink-benchmarks there.
>>
>> Regarding Chesnay's concern whether this requires a vote:
>> While the flink-benchmarks is indeed an existing external codebase, I'm
>> not entirely sure we do need a vote here.
>> The bylaws state that:
>> "Adoption of large existing external codebase. This refers to
>> contributions big enough that potentially change the shape and direction of
>> the project with massive restructuring and future maintenance commitment."
>>
>> The Flink benchmarks should not be a significant change to the shape /
>> direction of the project.
>> What do others think?
>>
>> On Tue, Apr 14, 2020 at 4:53 PM Chesnay Schepler <[hidden email]>
>> wrote:
>>
>>> Isn't this technically the adoption of a new codebase?
>>>
>>> On 14/04/2020 10:06, Yun Tang wrote:
>>> > Hi Flink devs
>>> >
>>> > Thanks for the feedbacks! Overall, everyone who replied is positive
>>> about creating such a separate repo to host the flink benchmarks.
>>> >
>>> > Since such an action does not necessarily need a vote (as defined by
>>> the project bylaws), I'll proceed to ask PMC's help to create the repo
>>> under Apache project.
>>> >
>>> > Best
>>> > Yun Tang
>>> >
>>> > ________________________________
>>> > From: Yu Li <[hidden email]>
>>> > Sent: Friday, April 10, 2020 21:03
>>> > To: dev <[hidden email]>
>>> > Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>>> >
>>> > +1 for the proposal, thanks for driving this Yun.
>>> >
>>> > Best Regards,
>>> > Yu
>>> >
>>> >
>>> > On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]> wrote:
>>> >
>>> >> +1 for this proposal.
>>> >>
>>> >>> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]>
>>> 写道：
>>> >>>
>>> >>> +1 for the proposal.
>>> >>>
>>> >>> Best,
>>> >>> Zhijiang
>>> >>>
>>> >>>
>>> >>> ------------------------------------------------------------------
>>> >>> From:Robert Metzger <[hidden email]>
>>> >>> Send Time:2020 Apr. 10 (Fri.) 02:15
>>> >>> To:dev <[hidden email]>
>>> >>> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>>> >>>
>>> >>> +1 on creating the repo.
>>> >>>
>>> >>>
>>> >>> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]>
>>> >> wrote:
>>> >>>> I think it is a good idea to make the benchmarks available to the
>>> >> community
>>> >>>> via a repo under the Apache project and to make updating it part of
>>> the
>>> >>>> release process. Hence +1 for the proposal.
>>> >>>>
>>> >>>> Cheers,
>>> >>>> Till
>>> >>>>
>>> >>>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]>
>>> >> wrote:
>>> >>>>> Hi Yun Tang,
>>> >>>>>
>>> >>>>> Thanks for proposing the idea. Since we can not include benchmarks
>>> in
>>> >> the
>>> >>>>> Flink repository what you are proposing is the second best option.
>>> >>>>>
>>> >>>>> +1 from my side for the proposal.
>>> >>>>>
>>> >>>>> I think benchmarks have proven their value to justify this.
>>> >>>>>
>>> >>>>> Piotrek
>>> >>>>>
>>> >>>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]> wrote:
>>> >>>>>>
>>> >>>>>> Hi Flink devs,
>>> >>>>>>
>>> >>>>>> As Flink develops rapidly with more and more features added, how
>>> to
>>> >>>>> ensure no performance regression existed has become more and more
>>> >>>>> important. And we would like to create a new repo under apache
>>> project
>>> >> to
>>> >>>>> host previous flink-benchmarks [1] repo, which is inspired when we
>>> >>>> discuss
>>> >>>>> under FLINK-16850 [2]
>>> >>>>>> Some background context on flink-benchmarks, for those who are not
>>> >>>>> familiar with the project yet:
>>> >>>>>> - Current flink-benchmarks does not align with the Flink release,
>>> >> which
>>> >>>>> lead developers not easy to verify
>>> >>>>>> performance at specific Flink version because current
>>> >>>> flink-benchmarks
>>> >>>>> always depends on the latest interfaces.
>>> >>>>>> - Above problem could be solved well if we could ensure
>>> >>>> flink-benchmarks
>>> >>>>> also create release branch when we
>>> >>>>>> releasing Flink. However, current flink-benchmarks repo is
>>> hosted
>>> >>>>> under dataArtisans (the former name of
>>> >>>>>> ververica) project, which is not involved in Flink release
>>> manual
>>> >>>> [3].
>>> >>>>> We propose to promote this repo under
>>> >>>>>> apache project so that release manager could have the right to
>>> >>>> release
>>> >>>>> on flink-benchmarks.
>>> >>>>>> - The reason why we not involve flink-benchmarks into the
>>> apache/flink
>>> >>>>> repo is because it heavily depends on
>>> >>>>>> JMH [4], which is under GPLv2 license.
>>> >>>>>>
>>> >>>>>> What do you think?
>>> >>>>>>
>>> >>>>>> Best,
>>> >>>>>> Yun Tang
>>> >>>>>>
>>> >>>>>> [1] https://github.com/dataArtisans/flink-benchmarks
>>> >>>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
>>> >>>>>> [3]
>>> >>
>>> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
>>> >>>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>
>>>
>>>

Yun Tang

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Thanks for Gordon's help!

We will move on and create FLINK-17280<https://issues.apache.org/jira/browse/FLINK-17280> [1] to track the progress.

[1] https://issues.apache.org/jira/browse/FLINK-17280

Best
Yun Tang
________________________________
From: Tzu-Li (Gordon) Tai <[hidden email]>
Sent: Monday, April 20, 2020 18:03
To: dev <[hidden email]>
Cc: Yun Tang <[hidden email]>
Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks

Repository has been created:
https://github.com/apache/flink-benchmarks

On Sat, Apr 18, 2020 at 5:10 PM Tzu-Li (Gordon) Tai <[hidden email]<mailto:[hidden email]>> wrote:
There doesn't seem to be any objections in not considering the flink-benchmarks migration as a significant change that requires a vote.

If no-one objects in the next days, I'll assume lazy consensus and create the new repo the following Monday (April 20).

On Tue, Apr 14, 2020 at 5:13 PM Tzu-Li (Gordon) Tai <[hidden email]<mailto:[hidden email]>> wrote:
+1 on creating a new repo, and migrating the flink-benchmarks there.

Regarding Chesnay's concern whether this requires a vote:
While the flink-benchmarks is indeed an existing external codebase, I'm not entirely sure we do need a vote here.
The bylaws state that:
"Adoption of large existing external codebase. This refers to contributions big enough that potentially change the shape and direction of the project with massive restructuring and future maintenance commitment."

The Flink benchmarks should not be a significant change to the shape / direction of the project.
What do others think?

On Tue, Apr 14, 2020 at 4:53 PM Chesnay Schepler <[hidden email]<mailto:[hidden email]>> wrote:
Isn't this technically the adoption of a new codebase?

On 14/04/2020 10:06, Yun Tang wrote:

> Hi Flink devs
>
> Thanks for the feedbacks! Overall, everyone who replied is positive about creating such a separate repo to host the flink benchmarks.
>
> Since such an action does not necessarily need a vote (as defined by the project bylaws), I'll proceed to ask PMC's help to create the repo under Apache project.
>
> Best
> Yun Tang
>
> ________________________________
> From: Yu Li <[hidden email]<mailto:[hidden email]>>
> Sent: Friday, April 10, 2020 21:03
> To: dev <[hidden email]<mailto:[hidden email]>>
> Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>
> +1 for the proposal, thanks for driving this Yun.
>
> Best Regards,
> Yu
>
>
> On Fri, 10 Apr 2020 at 11:25, Dian Fu <[hidden email]<mailto:[hidden email]>> wrote:
>
>> +1 for this proposal.
>>
>>> 在 2020年4月10日，上午10:58，Zhijiang <[hidden email]<mailto:[hidden email]>.INVALID> 写道：
>>>
>>> +1 for the proposal.
>>>
>>> Best,
>>> Zhijiang
>>>
>>>
>>> ------------------------------------------------------------------
>>> From:Robert Metzger <[hidden email]<mailto:[hidden email]>>
>>> Send Time:2020 Apr. 10 (Fri.) 02:15
>>> To:dev <[hidden email]<mailto:[hidden email]>>
>>> Subject:Re: [DISCUSS] Creating a new repo to host Flink benchmarks
>>>
>>> +1 on creating the repo.
>>>
>>>
>>> On Thu, Apr 9, 2020 at 5:54 PM Till Rohrmann <[hidden email]<mailto:[hidden email]>>
>> wrote:
>>>> I think it is a good idea to make the benchmarks available to the
>> community
>>>> via a repo under the Apache project and to make updating it part of the
>>>> release process. Hence +1 for the proposal.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <[hidden email]<mailto:[hidden email]>>
>> wrote:
>>>>> Hi Yun Tang,
>>>>>
>>>>> Thanks for proposing the idea. Since we can not include benchmarks in
>> the
>>>>> Flink repository what you are proposing is the second best option.
>>>>>
>>>>> +1 from my side for the proposal.
>>>>>
>>>>> I think benchmarks have proven their value to justify this.
>>>>>
>>>>> Piotrek
>>>>>
>>>>>> On 9 Apr 2020, at 08:56, Yun Tang <[hidden email]<mailto:[hidden email]>> wrote:
>>>>>>
>>>>>> Hi Flink devs,
>>>>>>
>>>>>> As Flink develops rapidly with more and more features added, how to
>>>>> ensure no performance regression existed has become more and more
>>>>> important. And we would like to create a new repo under apache project
>> to
>>>>> host previous flink-benchmarks [1] repo, which is inspired when we
>>>> discuss
>>>>> under FLINK-16850 [2]
>>>>>> Some background context on flink-benchmarks, for those who are not
>>>>> familiar with the project yet:
>>>>>> - Current flink-benchmarks does not align with the Flink release,
>> which
>>>>> lead developers not easy to verify
>>>>>> performance at specific Flink version because current
>>>> flink-benchmarks
>>>>> always depends on the latest interfaces.
>>>>>> - Above problem could be solved well if we could ensure
>>>> flink-benchmarks
>>>>> also create release branch when we
>>>>>> releasing Flink. However, current flink-benchmarks repo is hosted
>>>>> under dataArtisans (the former name of
>>>>>> ververica) project, which is not involved in Flink release manual
>>>> [3].
>>>>> We propose to promote this repo under
>>>>>> apache project so that release manager could have the right to
>>>> release
>>>>> on flink-benchmarks.
>>>>>> - The reason why we not involve flink-benchmarks into the apache/flink
>>>>> repo is because it heavily depends on
>>>>>> JMH [4], which is under GPLv2 license.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Best,
>>>>>> Yun Tang
>>>>>>
>>>>>> [1] https://github.com/dataArtisans/flink-benchmarks
>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-16850
>>>>>> [3]
>> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
>>>>>> [4] https://openjdk.java.net/projects/code-tools/jmh/
>>>>>>
>>>>>>
>>>>>
>>