[ANNOUNCE] Weekly Community Update 2019/38

[Konstantin Knauf-3]

Dear community,

happy to share this week's community update with a FLIP for the Pulsar
Connector contribution, three FLIPs for the SQL Ecosystem (plugin system,
computed columns, extended support for views), and a bit more. Enjoy!

Flink Development
==============

* [connectors]  After some discussion on the mailing list over the last
weeks, Sijie has opened a FLIP to add an exactly-once Pulsar Connector
(DataStream API, Table API, Catalog API) to Flink. [1]

* [sql] The discussion on supporting Hive built-in function in Flink SQL
lead to FLIP-69 to extend the core table system with modular plugins. [2]
While focusing on function modules as a first step, the FLIP proposes a
more general plugin system also covering user defined types, operators,
rules, etc. As part of this FLIP the existing functions in Flink SQL would
also be migrated into a "CorePlugin". [3]

* [sql] Danny proposes to add support for computed columns in Flink SQL (as
FLIP-10). [4]

* [sql] Zhenghua has started a discussion on extending support for VIEWs in
Flink SQL (as FLIP-71).  He proposes to add support to store views in a
catalog and to add support for "SHOW VIEWS" and "DESCRIBE VIEW". [5]


[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-72%3A+Introduce+Pulsar+Connector
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Modular+Plugins
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-68-Extend-Core-Table-System-with-Modular-Plugins-tp33161.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-70-Support-Computed-Column-for-Flink-SQL-tp33126.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-71-E2E-View-support-in-Flink-SQL-tp33131.html

Notable Bugs
==========

* [FLINK-14010] [ 1.9.0] [1.8.2] [1.7.2] [yarn] When the Flink Yarn
Application Manager receives a shut down request by the YARN Resource
Manager, the Flink cluster can get into an inconsistent state, where
leaderhship for JobManager, ResourceManager and Dispatcher components is
split between two master processes. Tison is working on a fix. [6]

* [FLINK-14107] [ 1.9.0] [ 1.8.2] [kinesis] When using event time alignment
with the Kinsesis Consumer the consumer might deadlock in one corner case.
Fixed for 1.9.1 and 1.8.3. [7]

[6] https://issues.apache.org/jira/browse/FLINK-14010
[7] https://issues.apache.org/jira/browse/FLINK-14107

Events, Blog Posts, Misc
===================

* Upcoming Meetups
    * *Enrico Canzonieri* of Yelp and *David Massart* of Tentative will
share their Apache Flink user stories of Yelp and BNP Paribas at the next *Bay
Area Apache Flink Meetup* 24th of September.  [8]

* *Ana Esguerra* has published a blog post on how to run Flink on YARN with
Kerberos for Kafka & YARN. [9]

[8] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262680261/
[9]
https://medium.com/@minyodev/apache-flink-on-yarn-with-kerberos-authentication-adeb62ef47d2

Cheers,

Konstantin

--

Konstantin Knauf | Solutions Architect

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng