[ANNOUNCE] Weekly Community Update 2020/36

[Konstantin Knauf-4]

Dear community,

happy to share another community update for the past week. This time with
the upcoming release of Flink 1.11.2, a proposal for more efficient
aggregation for batch processing with the DataStream API, and the
comeback of two FLIPs that have been abandoned for a bit.

Flink Development
==============

* [releases] Zhu Zhu proposes to release Flink 1.11.2 soon and has started
collecting blockers. Not too many open tickets, so I am anticipating a
first release candidate this week. [1]

* [releases] Robert has shared another update on blockers and instabilities
for the upcoming release. [2]

* [apis] FLIP-134 states the goal to replace the DataSet API by DataStream
API for bounded data processing. The implementation is handled in follow up
FLIPs. One of these was now published by Dawid as FLIP-140. It proposes to
replace hash-based aggregation by sort-based aggregations in keyed
operators if the input is bounded in order to improve the performance of
the DataStream API for bounded data processing. [3]

* [connectors] Becket has revived the discussion on FLIP-33, which aims to
standardize common metrics across all connectors of Apache Flink to
facilitate the integration with 3rd party systems. [4]

* [connectors] Timo has updated FLIP-107 which builds the foundation to
read/write table columns from/to different parts of source records. For
example, this will allow you to read/write the key and timestamp
information of a Kafka record. [4]

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-11-2-tp44323.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Release-1-12-Stale-blockers-and-build-instabilities-tp43477.html

[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Re-DISCUSS-FLIP-140-Introduce-bounded-style-execution-for-keyed-streams-tp44395.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-tp26869.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-107-Reading-table-columns-from-different-parts-of-source-records-tp38277.html

Notable Bugs
==========

* [FLINK-19109] [1.11.1] [1.10.2] Watermarks generated (in the same
operator chain) after ContinousFileReaderOperator (env.readXXX) are
swallowed. Fixed for 1.11.2. [6]

* [FLINK-19133] [1.11.1] A bug in the KafkaSerialzationSchemaWrapper can
lead to situations where only Kafka partition 0 ever receives data. Fixed
for 1.11.2 [7]

[6] https://issues.apache.org/jira/browse/FLINK-19109
[7] https://issues.apache.org/jira/browse/FLINK-19133

Events, Blog Posts, Misc
===================

* In this blog post Andrey describes the recent changes around memory
management and configuration of the JobManager process. Similar changes had
already been released for the Taskamangers in Flink 1.10 [8] , with Flink
1.11 the Jobmanager received a corresponding update. [9]

* Marta has published a Flink Community Update blog post for August 2020.
[10]

* The second video of my colleague Alexander's "Introduction to Flink"
series has been published on Youtube. If you are looking for a concise
refresher on the basics of Apache Flink , stop here. [11]

[8]
https://flink.apache.org/2020/09/01/flink-1.11-memory-management-improvements.html
[9]
https://flink.apache.org/news/2020/04/21/memory-management-improvements-flink-1.10.html
[10] https://flink.apache.org/news/2020/09/04/community-update.html
[11]
https://www.youtube.com/watch?v=_G-hQfT02BA&list=PLaDktj9CFcS9YAaJ4bKWMWpjptudLr782&index=2

Cheers,

Konstantin

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk