[ANNOUNCE] Weekly Community Update 2020/17

[Konstantin Knauf-4]

Dear community,

happy to share this week's community digest with an update on Flink 1.11,
Flink 1.10.1 and Flink 1.9.3, the revival of FLIP-36 to support interactive
programming, a new FLIP to unify (& separate) TimestampAssigners and a bit
more.


Flink Development
==============

Releases
^^^^^^^^^

* [releases] Apache Flink 1.9.3 was released. [1,2]

* [releases] Stephan has proposed 15th of May as the feature freeze date
for Flink 1.11 [3]. Subsequently, Piotr also published a status update on
the development progress for the upcoming release. Check it out to get an
overview of all the features, which are still or not anymore planned for
this release. [4]

* [releases] The first release candidate for Flink 1.10.1 is out. [5]

FLIPs
^^^^^^

* [table api] Xuannan has revived the discussion on FLIP-36 to support
interactive programming the Table API. In essence, the FLIP proposes to
support caching (intermediate) results of one (batch) job, so that they can
be used by following (batch) jobs in the same TableEnvironment. [6]

* [time] Aljoscha proposes to a) unify punctuated and periodic watermark
assigners and b) to separate watermarks assigners and timestamp extractors.
[7]

More
^^^^^

* [configuration] Yangze started a discussion to unify the way max/min are
used in the config options. Currently, there is a max of different patterns
(**.max, max-**, and more). [8]

* [connectors] Karim Mansour proposes a change to the current RabbitMQ
connector in apache Flink to make message deduplication more flexible. [9]

* [metrics] Jinhai would like to add additional labels to the metrics
reported by the PrometheusReporter. [10]

* [datastream api] Stephan proposes to remove a couple of
deprecated methods for state access in Flink 1.11. [11]


[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-3-released-tp40730.html
[2] https://flink.apache.org/news/2020/04/24/release-1.9.3.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Exact-feature-freeze-date-tp40624.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Development-progress-of-Apache-Flink-1-11-tp40718.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-1-10-1-release-candidate-1-tp40724.html
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-126-Unify-and-separate-Watermark-Assigners-tp40525p40565.html
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Should-max-min-be-part-of-the-hierarchy-of-config-option-tp40578.html
[9]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-flink-connector-rabbitmq-api-changes-tp40704.html
[10]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Add-custom-labels-on-AbstractPrometheusReporter-like-PrometheusPushGatewayReporter-s-groupiny-tp40708.html
[11]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Removing-deprecated-state-methods-in-1-11-tp40651.html

Notable Bugs
==========

* [FLINK-17350] [1.10.0] [1.9.3] [1.8.3]  Since Flink 1.5 users can choose
not to fail their Flink job on checkpoint errors. For the synchronous part
of a checkpoint the implementation of this feature was incorrect, leaving
operators in an inconsistent state following such a failure. Piotr proposes
to always fail tasks on failures in the synchronous part of a checkpoint
going forward. [12]

* [FLINK-17351] [1.10.0] [1.9.3] CheckpointFailureManager ignores
checkpoint timeouts when checking against the maximally tolerable number of
checkpoints failures. So, checkpoint failures are not discovered when they
only surface in the CheckpointFailureManager as a checkpoint timeout
instead of an exception. [13]

Background: both issues were discovered based on a bug report by Jun Qin
[14].

[12] https://issues.apache.org/jira/browse/FLINK-17351
[13] https://issues.apache.org/jira/browse/FLINK-17350
[14] https://issues.apache.org/jira/browse/FLINK-17327

Events, Blog Posts, Misc
===================

* Andrey recaps the changes and simplifications to Flink's memory
management (released in Flink 1.10) on the Apache Flink blog. [15] Closely
related, there is also a small tool to test different memory configurations
on flink-packages.org. [16]

[15]
https://flink.apache.org/news/2020/04/21/memory-management-improvements-flink-1.10.html
[16] https://flink-packages.org/packages/flink-memory-calculator

Cheers,

Konstantin

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk