(DEPRECATED) Apache Flink Mailing List archive.

[ANNOUNCE] Weekly Community Update 2019/25

Classic

List

Threaded

6 messages Options

Konstantin Knauf-3

[ANNOUNCE] Weekly Community Update 2019/25

Dear community,

this week's update contains a few pointers to upcoming changes in the
development process, two recent proposals around metrics as well as smaller
discussions on the dev mailing list. Still processing the backlog of topics
& discussions by including a few of them each week.

As always, please feel free to add additional updates and news to this
thread!

Flink Development
===============

* [development-process] Back in February the Flink community discussed
changes to its *code contribution process* (implemented in Jira) [1]. In
the future only committers will be able to assign contributors to tickets
hopefully moving discussions to an earlier stage in the contribution
process. The new process has now been fully documented and published on the
Flink Website [2]. As far as I know, the next step is changing the Jira
permissions accordingly.

* [development-process] Stephan has recently started a conversation about
adopting a c*ode style and quality guide* for contributions to Apache
Flink. It covers Flink-specific aspects like PR naming, but also general
Java best practices and lessons-learned in the context of a complex system
like Flink. The feedback was quite positive and Robert is now going to
split the guide into subsections and add it to the Flink website. [3]

* [releases] The last currently known blocker for* Flink 1.8.1* has been
resolved and Jincheng is in the process of creating a first release
candidate. [4]

* [releases] The feature freeze for *Flink 1.9.0 *has been set to the
originally proposed date, 28th of June [5,6].

* [metrics] In May Hwanju proposed to significantly enhance the *availability
metrics* of Flink and classify downtimes based on the root cause of the
failure (mainly between system and user code). [8] This will make it easier
for service providers to monitor the SLAs of a Flink-based service. The
discussion thread has by now been split into two design documents for
exception classification [9] and more fine-grained execution tracking [10].

* [metrics] Becket proposed to standardize the metrics provided by
connectors. It defines a set of *standard metrics*, which each source or
sink should expose (e.g. numBytesIn, pendingRecord). As far as I can tell,
the FLIP is still being voted on with positive feedback so far [11].

* Dyana brought up the possibility to include the *Kinesis Connector*
module in the official Apache Flink releases as these are now available
under Apache 2.0 license. The corresponding contribution is pending. [12]

* Xingcam started a discussion of whether to deprecate, fix or replace the
split/select methods of the DataStream API. They were marked deprecated in
Flink 1.8.0, but the current discussion has not come to a conclusion yet on
whether to really remove them going forward.

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html
[2] https://flink.apache.org/contributing/contribute-code.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Adopting-a-Code-Style-and-Quality-Guide-td29524.html#a29532
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Proposal-for-Flink-job-execution-availability-metrics-impovement-td28882.html
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
[9]
https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67Hmjgy0-hRDeuFnrMgT4/edit#heading=h.v3ssap5tvszp
[10]
https://docs.google.com/document/d/1oLF3w1wYyr8vqoFoQZhw1QxTofmAtlD8IF694oPLjNI/edit#heading=h.9zul23uwx8c6
[11]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-33-Standardize-connector-metrics-td29527.html
[12]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Updating-Kinesis-Connector-to-latest-Apache-licensed-libs-td29559.html#a29566
[13]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/About-Deprecating-split-select-for-DataStream-API-td29617.html

Notable Bugs
===========

* [FLINK-12871] [1.7.2] [ 1.8.0] The documentation how to setup SSL for
Flink's REST API and intra-cluster communication was erroneous. Fixed in
[1.7.3] [1.8.1]. [14]
* [FLINK-12912] [1.6.4] [1.7.2] [ 1.8.0] With the configuration parameter
"task.checkpoint.alignment.max-size" users can limit the data buffered
during the alignment phase of a checkpoint. Once reached, the checkpoint is
aborted. Currently, this check does not work correctly when one checkpoint
subsumes another one. Unresolved. [15]

[14] https://issues.apache.org/jira/browse/FLINK-12871
[15] https://issues.apache.org/jira/browse/FLINK-12912

Events, Blog Posts, Misc
====================

* This week there were a couple of Flink-related talks at Berlin Buzzwords
and Beam Summit Europe. Talks from Berlin Buzzwords are already available
online [16]. Not sure if there are recordings of Beam Summit, but
definitely saw a lot of slide sets being shared on Twitter.
* Liu Biao did an online training with about 400 participants on metrics in
Flink [16].
* There are a couple of meetups coming up in the next weeks:
* 2019/06/24: Cloud Native Meetup in *Aarhus* with a Flink talk by
Lasse Nedergard (TrackUnit) [18]
* 2019/06/26: *Bay Area* Apache Flink Meetup with talks by Zendesk,
Parag Kesar and Ben Liu (Pinterest) and Ken Krugler (Scale Unlimited) [19]
* 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by myself
(Ververica) [20]
* 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
Hausmann (AWS) and Michel David (Ryte) [21]

[16] https://berlinbuzzwords.de/19/sessions
[17] https://zh.ververica.com/developers/flink-training-course2
[18] https://flink.apache.org/2019/06/05/flink-network-stack.html
[19] https://www.meetup.com/Cloud-Native-Aarhus/events/261346897/
[20] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262216929
[21] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
[22] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/

Cheers,

Konstantin (@snntrable)

--

Konstantin Knauf | Solutions Architect

+49 160 91394525

--

Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Data Artisans GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen

Robert Metzger

Re: [ANNOUNCE] Weekly Community Update 2019/25

Thank you for the update! I always enjoy reading them.

Maybe we could use these threads as an input for a monthly "Community
Update" blog post series?
I've discussed this idea offline already my colleague Marta. Are you still
interested?

On Sun, Jun 23, 2019 at 10:18 PM Konstantin Knauf <[hidden email]>
wrote:

> Dear community,
>
> this week's update contains a few pointers to upcoming changes in the
> development process, two recent proposals around metrics as well as smaller
> discussions on the dev mailing list. Still processing the backlog of topics
> & discussions by including a few of them each week.
>
> As always, please feel free to add additional updates and news to this
> thread!
>
> Flink Development
> ===============
>
> * [development-process] Back in February the Flink community discussed
> changes to its *code contribution process* (implemented in Jira) [1]. In
> the future only committers will be able to assign contributors to tickets
> hopefully moving discussions to an earlier stage in the contribution
> process. The new process has now been fully documented and published on the
> Flink Website [2]. As far as I know, the next step is changing the Jira
> permissions accordingly.
>
> * [development-process] Stephan has recently started a conversation about
> adopting a c*ode style and quality guide* for contributions to Apache
> Flink. It covers Flink-specific aspects like PR naming, but also general
> Java best practices and lessons-learned in the context of a complex system
> like Flink. The feedback was quite positive and Robert is now going to
> split the guide into subsections and add it to the Flink website. [3]
>
> * [releases] The last currently known blocker for* Flink 1.8.1* has been
> resolved and Jincheng is in the process of creating a first release
> candidate. [4]
>
> * [releases] The feature freeze for *Flink 1.9.0 *has been set to the
> originally proposed date, 28th of June [5,6].
>
> * [metrics] In May Hwanju proposed to significantly enhance the *availability
> metrics* of Flink and classify downtimes based on the root cause of the
> failure (mainly between system and user code). [8] This will make it easier
> for service providers to monitor the SLAs of a Flink-based service. The
> discussion thread has by now been split into two design documents for
> exception classification [9] and more fine-grained execution tracking [10].
>
> * [metrics] Becket proposed to standardize the metrics provided by
> connectors. It defines a set of *standard metrics*, which each source or
> sink should expose (e.g. numBytesIn, pendingRecord). As far as I can
> tell, the FLIP is still being voted on with positive feedback so far [11].
>
> * Dyana brought up the possibility to include the *Kinesis Connector*
> module in the official Apache Flink releases as these are now available
> under Apache 2.0 license. The corresponding contribution is pending. [12]
>
> * Xingcam started a discussion of whether to deprecate, fix or replace the
> split/select methods of the DataStream API. They were marked deprecated in
> Flink 1.8.0, but the current discussion has not come to a conclusion yet on
> whether to really remove them going forward.
>
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html
> [2] https://flink.apache.org/contributing/contribute-code.html
> [3]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Adopting-a-Code-Style-and-Quality-Guide-td29524.html#a29532
> [4]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> [5]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html
> [6]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
> [7]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Proposal-for-Flink-job-execution-availability-metrics-impovement-td28882.html
> [8]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
> [9]
> https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67Hmjgy0-hRDeuFnrMgT4/edit#heading=h.v3ssap5tvszp
> [10]
> https://docs.google.com/document/d/1oLF3w1wYyr8vqoFoQZhw1QxTofmAtlD8IF694oPLjNI/edit#heading=h.9zul23uwx8c6
> [11]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-33-Standardize-connector-metrics-td29527.html
> [12]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Updating-Kinesis-Connector-to-latest-Apache-licensed-libs-td29559.html#a29566
> [13]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/About-Deprecating-split-select-for-DataStream-API-td29617.html
>
> Notable Bugs
> ===========
>
> * [FLINK-12871] [1.7.2] [ 1.8.0] The documentation how to setup SSL for
> Flink's REST API and intra-cluster communication was erroneous. Fixed in
> [1.7.3] [1.8.1]. [14]
> * [FLINK-12912] [1.6.4] [1.7.2] [ 1.8.0] With the configuration parameter
> "task.checkpoint.alignment.max-size" users can limit the data buffered
> during the alignment phase of a checkpoint. Once reached, the checkpoint is
> aborted. Currently, this check does not work correctly when one checkpoint
> subsumes another one. Unresolved. [15]
>
> [14] https://issues.apache.org/jira/browse/FLINK-12871
> [15] https://issues.apache.org/jira/browse/FLINK-12912
>
> Events, Blog Posts, Misc
> ====================
>
> * This week there were a couple of Flink-related talks at Berlin Buzzwords
> and Beam Summit Europe. Talks from Berlin Buzzwords are already available
> online [16]. Not sure if there are recordings of Beam Summit, but
> definitely saw a lot of slide sets being shared on Twitter.
> * Liu Biao did an online training with about 400 participants on metrics
> in Flink [16].
> * There are a couple of meetups coming up in the next weeks:
> * 2019/06/24: Cloud Native Meetup in *Aarhus* with a Flink talk by
> Lasse Nedergard (TrackUnit) [18]
> * 2019/06/26: *Bay Area* Apache Flink Meetup with talks by Zendesk,
> Parag Kesar and Ben Liu (Pinterest) and Ken Krugler (Scale Unlimited) [19]
> * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by myself
> (Ververica) [20]
> * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
> Hausmann (AWS) and Michel David (Ryte) [21]
>
> [16] https://berlinbuzzwords.de/19/sessions
> [17] https://zh.ververica.com/developers/flink-training-course2
> [18] https://flink.apache.org/2019/06/05/flink-network-stack.html
> [19] https://www.meetup.com/Cloud-Native-Aarhus/events/261346897/
> [20] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262216929
> [21] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
> [22] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/
>
> Cheers,
>
> Konstantin (@snntrable)
>
> --
>
> Konstantin Knauf | Solutions Architect
>
> +49 160 91394525
>
>
> --
>
> Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Data Artisans GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
>

vino yang

Re: [ANNOUNCE] Weekly Community Update 2019/25

Hi Robert,

I like this idea.
Having a Flink "monthly community update" could let the users and
developers know more about the community. So Big +1 from my side.

Robert Metzger <[hidden email]> 于2019年6月28日周五下午3:51写道：

> Thank you for the update! I always enjoy reading them.
>
> Maybe we could use these threads as an input for a monthly "Community
> Update" blog post series?
> I've discussed this idea offline already my colleague Marta. Are you still
> interested?
>
>
>
> On Sun, Jun 23, 2019 at 10:18 PM Konstantin Knauf <
> [hidden email]>
> wrote:
>
> > Dear community,
> >
> > this week's update contains a few pointers to upcoming changes in the
> > development process, two recent proposals around metrics as well as
> smaller
> > discussions on the dev mailing list. Still processing the backlog of
> topics
> > & discussions by including a few of them each week.
> >
> > As always, please feel free to add additional updates and news to this
> > thread!
> >
> > Flink Development
> > ===============
> >
> > * [development-process] Back in February the Flink community discussed
> > changes to its *code contribution process* (implemented in Jira) [1]. In
> > the future only committers will be able to assign contributors to tickets
> > hopefully moving discussions to an earlier stage in the contribution
> > process. The new process has now been fully documented and published on
> the
> > Flink Website [2]. As far as I know, the next step is changing the Jira
> > permissions accordingly.
> >
> > * [development-process] Stephan has recently started a conversation about
> > adopting a c*ode style and quality guide* for contributions to Apache
> > Flink. It covers Flink-specific aspects like PR naming, but also general
> > Java best practices and lessons-learned in the context of a complex
> system
> > like Flink. The feedback was quite positive and Robert is now going to
> > split the guide into subsections and add it to the Flink website. [3]
> >
> > * [releases] The last currently known blocker for* Flink 1.8.1* has been
> > resolved and Jincheng is in the process of creating a first release
> > candidate. [4]
> >
> > * [releases] The feature freeze for *Flink 1.9.0 *has been set to the
> > originally proposed date, 28th of June [5,6].
> >
> > * [metrics] In May Hwanju proposed to significantly enhance the
> *availability
> > metrics* of Flink and classify downtimes based on the root cause of the
> > failure (mainly between system and user code). [8] This will make it
> easier
> > for service providers to monitor the SLAs of a Flink-based service. The
> > discussion thread has by now been split into two design documents for
> > exception classification [9] and more fine-grained execution tracking
> [10].
> >
> > * [metrics] Becket proposed to standardize the metrics provided by
> > connectors. It defines a set of *standard metrics*, which each source or
> > sink should expose (e.g. numBytesIn, pendingRecord). As far as I can
> > tell, the FLIP is still being voted on with positive feedback so far
> [11].
> >
> > * Dyana brought up the possibility to include the *Kinesis Connector*
> > module in the official Apache Flink releases as these are now available
> > under Apache 2.0 license. The corresponding contribution is pending. [12]
> >
> > * Xingcam started a discussion of whether to deprecate, fix or replace
> the
> > split/select methods of the DataStream API. They were marked deprecated
> in
> > Flink 1.8.0, but the current discussion has not come to a conclusion yet
> on
> > whether to really remove them going forward.
> >
> > [1]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html
> > [2] https://flink.apache.org/contributing/contribute-code.html
> > [3]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Adopting-a-Code-Style-and-Quality-Guide-td29524.html#a29532
> > [4]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > [5]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html
> > [6]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
> > [7]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Proposal-for-Flink-job-execution-availability-metrics-impovement-td28882.html
> > [8]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
> > [9]
> >
> https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67Hmjgy0-hRDeuFnrMgT4/edit#heading=h.v3ssap5tvszp
> > [10]
> >
> https://docs.google.com/document/d/1oLF3w1wYyr8vqoFoQZhw1QxTofmAtlD8IF694oPLjNI/edit#heading=h.9zul23uwx8c6
> > [11]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-33-Standardize-connector-metrics-td29527.html
> > [12]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Updating-Kinesis-Connector-to-latest-Apache-licensed-libs-td29559.html#a29566
> > [13]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/About-Deprecating-split-select-for-DataStream-API-td29617.html
> >
> > Notable Bugs
> > ===========
> >
> > * [FLINK-12871] [1.7.2] [ 1.8.0] The documentation how to setup SSL for
> > Flink's REST API and intra-cluster communication was erroneous. Fixed in
> > [1.7.3] [1.8.1]. [14]
> > * [FLINK-12912] [1.6.4] [1.7.2] [ 1.8.0] With the configuration parameter
> > "task.checkpoint.alignment.max-size" users can limit the data buffered
> > during the alignment phase of a checkpoint. Once reached, the checkpoint
> is
> > aborted. Currently, this check does not work correctly when one
> checkpoint
> > subsumes another one. Unresolved. [15]
> >
> > [14] https://issues.apache.org/jira/browse/FLINK-12871
> > [15] https://issues.apache.org/jira/browse/FLINK-12912
> >
> > Events, Blog Posts, Misc
> > ====================
> >
> > * This week there were a couple of Flink-related talks at Berlin
> Buzzwords
> > and Beam Summit Europe. Talks from Berlin Buzzwords are already available
> > online [16]. Not sure if there are recordings of Beam Summit, but
> > definitely saw a lot of slide sets being shared on Twitter.
> > * Liu Biao did an online training with about 400 participants on metrics
> > in Flink [16].
> > * There are a couple of meetups coming up in the next weeks:
> > * 2019/06/24: Cloud Native Meetup in *Aarhus* with a Flink talk by
> > Lasse Nedergard (TrackUnit) [18]
> > * 2019/06/26: *Bay Area* Apache Flink Meetup with talks by Zendesk,
> > Parag Kesar and Ben Liu (Pinterest) and Ken Krugler (Scale Unlimited)
> [19]
> > * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by myself
> > (Ververica) [20]
> > * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
> > Hausmann (AWS) and Michel David (Ryte) [21]
> >
> > [16] https://berlinbuzzwords.de/19/sessions
> > [17] https://zh.ververica.com/developers/flink-training-course2
> > [18] https://flink.apache.org/2019/06/05/flink-network-stack.html
> > [19] https://www.meetup.com/Cloud-Native-Aarhus/events/261346897/
> > [20]
> https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262216929
> > [21] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
> > [22] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/
> >
> > Cheers,
> >
> > Konstantin (@snntrable)
> >
> > --
> >
> > Konstantin Knauf | Solutions Architect
> >
> > +49 160 91394525
> >
> >
> > --
> >
> > Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> >
> > --
> > Data Artisans GmbH
> > Registered at Amtsgericht Charlottenburg: HRB 158244 B
> > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
> >
>

Marta Paes Moreira

Re: [ANNOUNCE] Weekly Community Update 2019/25

In reply to this post by Robert Metzger

Hi, Robert.

Using these digests for that purpose is something I also discussed with
Konstantin, so I believe we are aligned on this. A monthly community update
could go a bit beyond and recap on events (e.g. meetups, conferences) and
other relevant work that is brewing in the community.

If everyone is on board with the idea, I can take the initiative to write
these up on the Flink blog.

Thanks!

On Fri, Jun 28, 2019 at 9:51 AM Robert Metzger <[hidden email]> wrote:

Robert Metzger

Re: [ANNOUNCE] Weekly Community Update 2019/25

Great! Thanks Marta. Looking forward to review the first pull request for a
blog post.

On Fri, Jun 28, 2019 at 10:09 AM Marta Paes Moreira <[hidden email]>
wrote:

> Hi, Robert.
>
> Using these digests for that purpose is something I also discussed with
> Konstantin, so I believe we are aligned on this. A monthly community update
> could go a bit beyond and recap on events (e.g. meetups, conferences) and
> other relevant work that is brewing in the community.
>
> If everyone is on board with the idea, I can take the initiative to write
> these up on the Flink blog.
>
> Thanks!
>
> On Fri, Jun 28, 2019 at 9:51 AM Robert Metzger <[hidden email]>
> wrote:
>
> > Thank you for the update! I always enjoy reading them.
> >
> > Maybe we could use these threads as an input for a monthly "Community
> > Update" blog post series?
> > I've discussed this idea offline already my colleague Marta. Are you
> still
> > interested?
> >
> >
> >
> > On Sun, Jun 23, 2019 at 10:18 PM Konstantin Knauf <
> > [hidden email]>
> > wrote:
> >
> > > Dear community,
> > >
> > > this week's update contains a few pointers to upcoming changes in the
> > > development process, two recent proposals around metrics as well as
> > smaller
> > > discussions on the dev mailing list. Still processing the backlog of
> > topics
> > > & discussions by including a few of them each week.
> > >
> > > As always, please feel free to add additional updates and news to this
> > > thread!
> > >
> > > Flink Development
> > > ===============
> > >
> > > * [development-process] Back in February the Flink community discussed
> > > changes to its *code contribution process* (implemented in Jira) [1].
> In
> > > the future only committers will be able to assign contributors to
> tickets
> > > hopefully moving discussions to an earlier stage in the contribution
> > > process. The new process has now been fully documented and published on
> > the
> > > Flink Website [2]. As far as I know, the next step is changing the Jira
> > > permissions accordingly.
> > >
> > > * [development-process] Stephan has recently started a conversation
> about
> > > adopting a c*ode style and quality guide* for contributions to Apache
> > > Flink. It covers Flink-specific aspects like PR naming, but also
> general
> > > Java best practices and lessons-learned in the context of a complex
> > system
> > > like Flink. The feedback was quite positive and Robert is now going to
> > > split the guide into subsections and add it to the Flink website. [3]
> > >
> > > * [releases] The last currently known blocker for* Flink 1.8.1* has
> been
> > > resolved and Jincheng is in the process of creating a first release
> > > candidate. [4]
> > >
> > > * [releases] The feature freeze for *Flink 1.9.0 *has been set to the
> > > originally proposed date, 28th of June [5,6].
> > >
> > > * [metrics] In May Hwanju proposed to significantly enhance the
> > *availability
> > > metrics* of Flink and classify downtimes based on the root cause of the
> > > failure (mainly between system and user code). [8] This will make it
> > easier
> > > for service providers to monitor the SLAs of a Flink-based service. The
> > > discussion thread has by now been split into two design documents for
> > > exception classification [9] and more fine-grained execution tracking
> > [10].
> > >
> > > * [metrics] Becket proposed to standardize the metrics provided by
> > > connectors. It defines a set of *standard metrics*, which each source
> or
> > > sink should expose (e.g. numBytesIn, pendingRecord). As far as I can
> > > tell, the FLIP is still being voted on with positive feedback so far
> > [11].
> > >
> > > * Dyana brought up the possibility to include the *Kinesis Connector*
> > > module in the official Apache Flink releases as these are now available
> > > under Apache 2.0 license. The corresponding contribution is pending.
> [12]
> > >
> > > * Xingcam started a discussion of whether to deprecate, fix or replace
> > the
> > > split/select methods of the DataStream API. They were marked deprecated
> > in
> > > Flink 1.8.0, but the current discussion has not come to a conclusion
> yet
> > on
> > > whether to really remove them going forward.
> > >
> > > [1]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html
> > > [2] https://flink.apache.org/contributing/contribute-code.html
> > > [3]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Adopting-a-Code-Style-and-Quality-Guide-td29524.html#a29532
> > > [4]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > [5]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html
> > > [6]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
> > > [7]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Proposal-for-Flink-job-execution-availability-metrics-impovement-td28882.html
> > > [8]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
> > > [9]
> > >
> >
> https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67Hmjgy0-hRDeuFnrMgT4/edit#heading=h.v3ssap5tvszp
> > > [10]
> > >
> >
> https://docs.google.com/document/d/1oLF3w1wYyr8vqoFoQZhw1QxTofmAtlD8IF694oPLjNI/edit#heading=h.9zul23uwx8c6
> > > [11]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-33-Standardize-connector-metrics-td29527.html
> > > [12]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Updating-Kinesis-Connector-to-latest-Apache-licensed-libs-td29559.html#a29566
> > > [13]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/About-Deprecating-split-select-for-DataStream-API-td29617.html
> > >
> > > Notable Bugs
> > > ===========
> > >
> > > * [FLINK-12871] [1.7.2] [ 1.8.0] The documentation how to setup SSL for
> > > Flink's REST API and intra-cluster communication was erroneous. Fixed
> in
> > > [1.7.3] [1.8.1]. [14]
> > > * [FLINK-12912] [1.6.4] [1.7.2] [ 1.8.0] With the configuration
> parameter
> > > "task.checkpoint.alignment.max-size" users can limit the data buffered
> > > during the alignment phase of a checkpoint. Once reached, the
> checkpoint
> > is
> > > aborted. Currently, this check does not work correctly when one
> > checkpoint
> > > subsumes another one. Unresolved. [15]
> > >
> > > [14] https://issues.apache.org/jira/browse/FLINK-12871
> > > [15] https://issues.apache.org/jira/browse/FLINK-12912
> > >
> > > Events, Blog Posts, Misc
> > > ====================
> > >
> > > * This week there were a couple of Flink-related talks at Berlin
> > Buzzwords
> > > and Beam Summit Europe. Talks from Berlin Buzzwords are already
> available
> > > online [16]. Not sure if there are recordings of Beam Summit, but
> > > definitely saw a lot of slide sets being shared on Twitter.
> > > * Liu Biao did an online training with about 400 participants on
> metrics
> > > in Flink [16].
> > > * There are a couple of meetups coming up in the next weeks:
> > > * 2019/06/24: Cloud Native Meetup in *Aarhus* with a Flink talk by
> > > Lasse Nedergard (TrackUnit) [18]
> > > * 2019/06/26: *Bay Area* Apache Flink Meetup with talks by Zendesk,
> > > Parag Kesar and Ben Liu (Pinterest) and Ken Krugler (Scale Unlimited)
> > [19]
> > > * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by
> myself
> > > (Ververica) [20]
> > > * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
> > > Hausmann (AWS) and Michel David (Ryte) [21]
> > >
> > > [16] https://berlinbuzzwords.de/19/sessions
> > > [17] https://zh.ververica.com/developers/flink-training-course2
> > > [18] https://flink.apache.org/2019/06/05/flink-network-stack.html
> > > [19] https://www.meetup.com/Cloud-Native-Aarhus/events/261346897/
> > > [20]
> > https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262216929
> > > [21] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
> > > [22]
> https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/
> > >
> > > Cheers,
> > >
> > > Konstantin (@snntrable)
> > >
> > > --
> > >
> > > Konstantin Knauf | Solutions Architect
> > >
> > > +49 160 91394525
> > >
> > >
> > > --
> > >
> > > Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> > >
> > > --
> > > Data Artisans GmbH
> > > Registered at Amtsgericht Charlottenburg: HRB 158244 B
> > > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
> > >
> >
>

Konstantin Knauf-3

Re: [ANNOUNCE] Weekly Community Update 2019/25

Hi Robert, Hi Marta,

sounds good to me, happy to review.

Cheers,

Konstantin

On Fri, Jun 28, 2019 at 3:11 PM Robert Metzger <[hidden email]> wrote:

> Great! Thanks Marta. Looking forward to review the first pull request for a
> blog post.
>
> On Fri, Jun 28, 2019 at 10:09 AM Marta Paes Moreira <[hidden email]>
> wrote:
>
> > Hi, Robert.
> >
> > Using these digests for that purpose is something I also discussed with
> > Konstantin, so I believe we are aligned on this. A monthly community
> update
> > could go a bit beyond and recap on events (e.g. meetups, conferences) and
> > other relevant work that is brewing in the community.
> >
> > If everyone is on board with the idea, I can take the initiative to write
> > these up on the Flink blog.
> >
> > Thanks!
> >
> > On Fri, Jun 28, 2019 at 9:51 AM Robert Metzger <[hidden email]>
> > wrote:
> >
> > > Thank you for the update! I always enjoy reading them.
> > >
> > > Maybe we could use these threads as an input for a monthly "Community
> > > Update" blog post series?
> > > I've discussed this idea offline already my colleague Marta. Are you
> > still
> > > interested?
> > >
> > >
> > >
> > > On Sun, Jun 23, 2019 at 10:18 PM Konstantin Knauf <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Dear community,
> > > >
> > > > this week's update contains a few pointers to upcoming changes in the
> > > > development process, two recent proposals around metrics as well as
> > > smaller
> > > > discussions on the dev mailing list. Still processing the backlog of
> > > topics
> > > > & discussions by including a few of them each week.
> > > >
> > > > As always, please feel free to add additional updates and news to
> this
> > > > thread!
> > > >
> > > > Flink Development
> > > > ===============
> > > >
> > > > * [development-process] Back in February the Flink community
> discussed
> > > > changes to its *code contribution process* (implemented in Jira) [1].
> > In
> > > > the future only committers will be able to assign contributors to
> > tickets
> > > > hopefully moving discussions to an earlier stage in the contribution
> > > > process. The new process has now been fully documented and published
> on
> > > the
> > > > Flink Website [2]. As far as I know, the next step is changing the
> Jira
> > > > permissions accordingly.
> > > >
> > > > * [development-process] Stephan has recently started a conversation
> > about
> > > > adopting a c*ode style and quality guide* for contributions to Apache
> > > > Flink. It covers Flink-specific aspects like PR naming, but also
> > general
> > > > Java best practices and lessons-learned in the context of a complex
> > > system
> > > > like Flink. The feedback was quite positive and Robert is now going
> to
> > > > split the guide into subsections and add it to the Flink website. [3]
> > > >
> > > > * [releases] The last currently known blocker for* Flink 1.8.1* has
> > been
> > > > resolved and Jincheng is in the process of creating a first release
> > > > candidate. [4]
> > > >
> > > > * [releases] The feature freeze for *Flink 1.9.0 *has been set to the
> > > > originally proposed date, 28th of June [5,6].
> > > >
> > > > * [metrics] In May Hwanju proposed to significantly enhance the
> > > *availability
> > > > metrics* of Flink and classify downtimes based on the root cause of
> the
> > > > failure (mainly between system and user code). [8] This will make it
> > > easier
> > > > for service providers to monitor the SLAs of a Flink-based service.
> The
> > > > discussion thread has by now been split into two design documents for
> > > > exception classification [9] and more fine-grained execution tracking
> > > [10].
> > > >
> > > > * [metrics] Becket proposed to standardize the metrics provided by
> > > > connectors. It defines a set of *standard metrics*, which each source
> > or
> > > > sink should expose (e.g. numBytesIn, pendingRecord). As far as I can
> > > > tell, the FLIP is still being voted on with positive feedback so far
> > > [11].
> > > >
> > > > * Dyana brought up the possibility to include the *Kinesis Connector*
> > > > module in the official Apache Flink releases as these are now
> available
> > > > under Apache 2.0 license. The corresponding contribution is pending.
> > [12]
> > > >
> > > > * Xingcam started a discussion of whether to deprecate, fix or
> replace
> > > the
> > > > split/select methods of the DataStream API. They were marked
> deprecated
> > > in
> > > > Flink 1.8.0, but the current discussion has not come to a conclusion
> > yet
> > > on
> > > > whether to really remove them going forward.
> > > >
> > > > [1]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html
> > > > [2] https://flink.apache.org/contributing/contribute-code.html
> > > > [3]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Adopting-a-Code-Style-and-Quality-Guide-td29524.html#a29532
> > > > [4]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > [5]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html
> > > > [6]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
> > > > [7]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Proposal-for-Flink-job-execution-availability-metrics-impovement-td28882.html
> > > > [8]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
> > > > [9]
> > > >
> > >
> >
> https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67Hmjgy0-hRDeuFnrMgT4/edit#heading=h.v3ssap5tvszp
> > > > [10]
> > > >
> > >
> >
> https://docs.google.com/document/d/1oLF3w1wYyr8vqoFoQZhw1QxTofmAtlD8IF694oPLjNI/edit#heading=h.9zul23uwx8c6
> > > > [11]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-33-Standardize-connector-metrics-td29527.html
> > > > [12]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Updating-Kinesis-Connector-to-latest-Apache-licensed-libs-td29559.html#a29566
> > > > [13]
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/About-Deprecating-split-select-for-DataStream-API-td29617.html
> > > >
> > > > Notable Bugs
> > > > ===========
> > > >
> > > > * [FLINK-12871] [1.7.2] [ 1.8.0] The documentation how to setup SSL
> for
> > > > Flink's REST API and intra-cluster communication was erroneous.
> Fixed
> > in
> > > > [1.7.3] [1.8.1]. [14]
> > > > * [FLINK-12912] [1.6.4] [1.7.2] [ 1.8.0] With the configuration
> > parameter
> > > > "task.checkpoint.alignment.max-size" users can limit the data
> buffered
> > > > during the alignment phase of a checkpoint. Once reached, the
> > checkpoint
> > > is
> > > > aborted. Currently, this check does not work correctly when one
> > > checkpoint
> > > > subsumes another one. Unresolved. [15]
> > > >
> > > > [14] https://issues.apache.org/jira/browse/FLINK-12871
> > > > [15] https://issues.apache.org/jira/browse/FLINK-12912
> > > >
> > > > Events, Blog Posts, Misc
> > > > ====================
> > > >
> > > > * This week there were a couple of Flink-related talks at Berlin
> > > Buzzwords
> > > > and Beam Summit Europe. Talks from Berlin Buzzwords are already
> > available
> > > > online [16]. Not sure if there are recordings of Beam Summit, but
> > > > definitely saw a lot of slide sets being shared on Twitter.
> > > > * Liu Biao did an online training with about 400 participants on
> > metrics
> > > > in Flink [16].
> > > > * There are a couple of meetups coming up in the next weeks:
> > > > * 2019/06/24: Cloud Native Meetup in *Aarhus* with a Flink talk
> by
> > > > Lasse Nedergard (TrackUnit) [18]
> > > > * 2019/06/26: *Bay Area* Apache Flink Meetup with talks by
> Zendesk,
> > > > Parag Kesar and Ben Liu (Pinterest) and Ken Krugler (Scale Unlimited)
> > > [19]
> > > > * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by
> > myself
> > > > (Ververica) [20]
> > > > * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
> > > > Hausmann (AWS) and Michel David (Ryte) [21]
> > > >
> > > > [16] https://berlinbuzzwords.de/19/sessions
> > > > [17] https://zh.ververica.com/developers/flink-training-course2
> > > > [18] https://flink.apache.org/2019/06/05/flink-network-stack.html
> > > > [19] https://www.meetup.com/Cloud-Native-Aarhus/events/261346897/
> > > > [20]
> > > https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262216929
> > > > [21]
> https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
> > > > [22]
> > https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/
> > > >
> > > > Cheers,
> > > >
> > > > Konstantin (@snntrable)
> > > >
> > > > --
> > > >
> > > > Konstantin Knauf | Solutions Architect
> > > >
> > > > +49 160 91394525
> > > >
> > > >
> > > > --
> > > >
> > > > Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
> > > >
> > > > --
> > > > Data Artisans GmbH
> > > > Registered at Amtsgericht Charlottenburg: HRB 158244 B
> > > > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
> > > >
> > >
> >
>

--

Konstantin Knauf | Solutions Architect

+49 160 91394525

--

Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Data Artisans GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen