[ANNOUNCE] Weekly Community Update 2020/09

[Konstantin Knauf-3]

Dear community,

happy to share this week's community update. It was a relatively quiet week
on the dev@ mailing list (mostly votes on previously covered FLIPs), but
there is always something to share. Additionally, I have decided to also
feature *flink-packages.org <http://flink-packages.org> *in this newsletter
going forward. Depending on the level of activity, I will cover newly added
packages or introduce one of the existing packages.

Flink Development
==============

* [sql] Dawid has started a discussion to enable Tabla API/SQL sources to
read columns from different parts of source records. With this it would,
for example, be possible to read partition, timestamp or offset from a
Kafka source record. Similarly, it would be possible to specify override
partitioning when writing to Kafka or Kinesis. [1]

* [sql, python] FLIP-58 introduced Python UDFs in SQL and Table API.
FLIP-79 added a Function DDL in Flink SQL to register Java & Scala UDFs in
pure SQL. Based on these two FLIPs, Wei Zhon published FLIP-106 to also
support Python UDFs in the SQL Function DDL. [2]

* [development] Chesnay started a discussion on Eclipse support for Apache
Flink (framework) development. If you are using Eclipse as an Apache Flink
contributor, please get involved in the thread. [3]

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-107-Reading-table-columns-from-different-parts-of-source-records-tp38277.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-106-Support-Python-UDF-in-SQL-Function-DDL-tp38107.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Remove-Eclipse-specific-plugin-configurations-tp38255.html

Notable Bugs
==========

[FLINK-16262] [1.10.0] The FlinkKafkaProducer can not be used in
EXACTLY_ONCE mode when using the user code classloader. For application
cluster (per-job clusters) you can work around this issue by using the
system classloader (user jar in lib/ directory). Will be fixed in 1.10.1.
[4]

[4] https://issues.apache.org/jira/browse/FLINK-16262

flink-packages.org
=============

DTStack, a Chinese cloud technology company, has recently published FlinkX
[5] on flink-packages.org. The documentation is Chinese only, but it seems
to be a configuration-based integration framework based on Apache Flink
with an impressive set of connectors.

[5] https://flink-packages.org/packages/flinkx

Events, Blog Posts, Misc
===================

* This week I stumbled across this Azure tutorial to use Event Hubs with
Apache Flink. [6]
* Gökce Sürenkök has written a blog post on setting up a highly available
Flink cluster on Kubernetes based on Zookeeper for Flink Master failover
and HDFS as checkpoint storage. [7]

* Upcoming Meetups
    * On March 5th, Stephan Ewen will talk about Apache Flink Stateful
Function at the Utrecht Data Engineering Meetup. [8]
    * On March 12th, Prateep Kumar will host an online event comparing
Kafka Streams and Apache Flink [9].
    * On April 22, Ververica will host the next Apache Flink meetup in
Berlin. [10]
    * Cloudera is hosting a couple of "Future of Data" events on stream
processing with Apache Flink in
        * Vienna (March 4th, full-day workshop) [11]
        * Zurich (March 10th, full-day workshop) [12]
        * New Jersey (May 5th, meetup) [13]

[6]
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-flink-tutorial
[7]
https://medium.com/hepsiburadatech/high-available-flink-cluster-on-kubernetes-setup-73b2baf9200e
[8] https://www.meetup.com/Data-Engineering-NL/events/268424399/
[9]
https://www.meetup.com/apache-flink-aws-kinesis-hyd-india/events/268930388/
[10] https://www.meetup.com/Apache-Flink-Meetup/events/269005339/
[11] https://www.meetup.com/futureofdata-vienna/events/268418974/
[12] https://www.meetup.com/futureofdata-zurich/events/268423809/
[13] https://www.meetup.com/futureofdata-princeton/events/268830725/

Cheers,

Konstantin (@snntrable)

--

Konstantin Knauf | Head of Product

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng