Weekly Community Update 2019/26

[Konstantin Knauf-3]

Dear community,

this weeks community digest with updates on Flink 1.8.1 & Flink 1.9.0,
another thread on our development process and a few API related threads. I
still include older threads from time to time. Enjoy!

As always, please feel free to add additional updates and news to this
thread!

Flink Development
===============

* [development-process] Ajoscha started a discussion on our FLIP process.
In particular, he pointed out that we are currently not following our own
rules, which require a "lazy majority" (3 binding +1s) to accept a FLIP. In
reality, most FLIPs are "accepted" without a vote at all or with a "lazy
approval". There has generally been a lot of positive feedback to enforce a
"lazy majority" in the future. In addition, questions were raised on which
changes require a FLIP, how large a FLIP can be and if we can use the
voting process to scope FLIPs into releases. [1]

* [releases] Jincheng has prepared a first release candidate for *Flink
1.8.1* to be voted on. Additional testing and votes are needed to move this
forward.

* [releases] The feature freeze for *Flink 1.9.0 *has been pushed by one
week (5hth of July) to finish work on the Blink-based Table API Runner and
fine-grained recovery.

* [dataset-api] Vino Yang has proposed (FLIP-44) to support local
aggregations in Flink by introducing a new concept of "local keyed state".
There seems to be a need of further discussion. Specifically, the community
has not yet agreed on the main purpose of the FLIP (local aggregations to
mitigate effects of data skew vs introduction of local keyed state, which
can also be used for the former) and consequentially there are also open
questions regarding the API and implementation. [4]

* [table-api] When a DynamicTable is created by a `flatAggregate` function
it is currently not possible to connect it to an `UpsertStreamTableSink`,
because after `flatAggregate` it is not clear, which unique keys to use for
the upsert. Jincheng has started a discussion to mitigate this and provided
a design document laying out different options to add the missing
information on unique keys. [5]

* [client-api] Jeff started a conversation a couple of weeks ago on
improving Flink's Client API and better programmatic job and cluster
control. This would make it easier for downstream projects (e.g. Zeppelin,
Beam) to integrate with Flink. The initiative has genreally been
well-received, but will require some additional upfront design discussions
due to its complexity and the many Public APIs, which are involved.

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-META-FLIP-Sticking-or-not-to-a-strict-FLIP-voting-process-td29978.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-1-8-1-release-candidate-1-td29929.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Local-Aggregation-in-Flink-td29307.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Upsert-mode-for-Streaming-Non-window-FlatAggregate-tp29975.html
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-client-api-enhancement-for-downstream-project-td25844.html

Notable Bugs
===========

Nothing that came to my attention. So far focusing this on released
versions only.

Its worth mentioning that FLINK-11662 (covered in Weekly Community Update
2019/24 [7]) has been resolved.

[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Weekly-Community-Update-2019-24-tp29630.html

Events, Blog Posts, Misc
====================

* Jincheng is now part of the Apache Flink PMC. Congrats! [8]
* There are a couple of meetups coming up in the coming week
    * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by myself
(Ververica) [9]
    * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen
Hausmann (AWS) and Michel David (Ryte) [10]

[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jincheng-Sun-is-now-part-of-the-Flink-PMC-td29868i20.html#a29966
[9] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
[10] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/

Cheers,

Konstantin (@snntrable)

--

Konstantin Knauf | Solutions Architect

[phone]


--

Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Data Artisans GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen