[ANNOUNCE] Weekly Community Update 2019/39

[Konstantin Knauf-3]

Dear community,

happy to share this week's community update with news about Flink 1.10 &
1.9.1, two FLIPs for better programmatic job and cluster control,
improvements to the web user interface and a bit more.

Flink Development
==============

* [releases] Jark has started a discussion on releasing a first patch
release for Flink 1.9. Looking at the discussion I expect a first release
candidate soonish. [1]

* [releases] Yu Li has shared a small progress report on Flink 1.10, which
gives a very good overview of ongoing FLIPs including their status. [2]

* [ui] Yadong has started a discussion on contributing improvements to the
Flink Web UI. It includes a variety of pretty cool changes such as better
overview over the cluster resources, flame graphs, additional backpressure
monitoring and much more. Best you check it out yourself. [3]

* [python] FLIP-58 aims to support stateless Python UDFs in the Table API.
Wei has shared a design document to support the usage of Python
dependencies inside such UDFs and is looking for feedback. [4]

* [sql] There a currently three ongoing votes for previously covered FLIPs
in the SQL ecosystem (FLIP-66 (Time-Attribute in SQL DDL), FLIP-57
(Function Catalog), FLIP-68 (Modular Plugins)). [5,6,7]

* [client] A couple of weeks ago Zili had started a discussion on how to
improve Flink's API for job submission and job and cluster management. As a
result of this discussion Kostas has now shared a design document to
refactor the (Stream)ExecutionEnvironments. The basic idea is to move the
responsibility of job submission out of the ExecutionEnvironment into
Executors. This would result in only one ExecutionEnvironment per API in
the future, and one Executor per environment (YARN, Standalone, ...). The
ExecutionEnvironment would then choose the correct Executor based on its
configuration. [8]

* [client] The second FLIP that originated from this discussion was also
shared by Zili this week and proposes to expose JobClient, ClusterClient
and ClusterDescriptor to users to programmatically control Flink Jobs and
Clusters during their runtime. [9]

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-tp33343.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Progress-of-Apache-Flink-1-10-1-tp33570.html
[3]
https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit#
[4]
https://docs.google.com/document/d/1vq5J3TSyhscQXbpRhz-Yd3KCX62PBJeC_a_h3amUvJ4/edit#heading=h.lvy7nudjmhjd
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-66-Support-Time-Attribute-in-SQL-DDL-2-tp33513.html
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-57-Rework-FunctionCatalog-tp33373.html
[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-68-Extend-Core-Table-System-with-Modular-Plugins-tp33372.html
[8]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-73%3A+Introducing+Executors+for+job+submission
[9]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-74:+Flink+JobClient+API

Notable Bugs
==========

[FLINK-14145] [1.9.0] When preferring checkpoints (as opposed to
savepoints) for recovery, the wrong checkpoint could be chosen for recovery
in certain situations. Fixed for 1.9.1. [10]

[FLINK-13708] [1.9.0] When calling "execute" twice on the same table
environment, transformations from the first execution will also be part of
the second execution. PR pending, might get into 1.9.1. [11]

[10] https://issues.apache.org/jira/browse/FLINK-14145
[11] https://issues.apache.org/jira/browse/FLINK-13708

Events, Blog Posts, Misc
===================

* Flink Forward Europe is getting close, 7th - 9th of October. Keynotes by
AWS, Cloudera, Cloud Foundry Foundation, Google and Ververica. [12]
* Upcoming Meetups
    * Tomorrow * Dean Shaw* and *Max McKittrick* will talk about click
stream analysis at scale at Capital One at the next dataCouncil.ai NYC Data
Engineering meetup. [13]

[12] https://europe-2019.flink-forward.org/
[13]
https://www.meetup.com/DataCouncil-AI-NYC-Data-Engineering-Science/events/264748638/

Cheers,

Konstantin

--

Konstantin Knauf | Solutions Architect

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng