[ANNOUNCE] Weekly Community Update 2020/10

[Konstantin Knauf-3]

Dear community,

happy to share this week's community update featuring a set of exciting
proposals from very different areas of Apache Flink, e.g.

* Initial GPU Support
* Flink Development Blog
* "LIKE" in "CREATE TABLE"

and a bit more. Enjoy.

Flink Development
==============

* [deployment] In FLIP-108, Yangze Guo is proposing to add support for *GPU*
resource management to Flink. In a first step, users would be able to
configure the number of GPUs a Taskmanager should request from the resource
manager (YARN, Kubernetes). The number of available GPUs would then be made
available to operators (e.g. via the RuntimeContext). [1]

* [docker] Since Flink 1.10.0 the library/fink Docker repository is
maintained by Apache Flink project. Now, Andrey is proposing to *consolidate
the different Dockerfiles* and Docker utilities spread over the
documentation. Basically, all Dockerfiles would be based on library/flink
and different utilities would be provided to customize/extend/configure
library/flink are proposed. [2]

* [state management] Arvid has posted an update on FLIP-76 (*Unaligned
Checkpoints*) and would like to proceed to a vote soon. [3]

* [datastream] Tison proposes to change the *dependency structure* around
flink-streaming-java [4]

* [development process] Arvid has proposed to start a *"Flink Development
Blog"*, which covers recent & ongoing development work in Apache Flink and
is primarily targeted to the community of Apache Flink contributors. The
proposal has received a lot of positive feedback from other committers and
contributors. It looks like we will start with a dedicated section in
Flink's Wiki [5]. From there posts could then also be promoted to the Flink
Blog Post, if there are suited for a larger audience. [6]

* [development process] Jark Wu started a discussion to disable
Github's *"Squash
and Merge" *for Apache Flink mostly as it results in lost author
information of the original commits and can mess up email addresses. The
discussion revealed that there are also good reasons to keep it, so in the
end only committer guide has been updated.  [7,8]

* [sql] Dawid proposes to support *"LIKE" in "CREATE TABLE"* statement in
Flink SQL to make it easier to slightly adjust/extend existing tables. This
becomes more relevant as Flink is adding support for more external catalogs
(SchemaRegistry, HiveCatalog, JDBCCatalog, ...). [9]

* [sql, hive] Jingsong proposes to provide "convenience *fat JARs*" for the*
Flink Hive* connector. These JARs called "flink-connector-x.x" would come
in different versions and contain all the required dependencies to
integrate Flink with Hive. [10]

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-108-Add-GPU-support-in-Flink-tp38286.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-111-Docker-image-unification-tp38444.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-76-Unaligned-checkpoints-tp33651.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PROPOSAL-Reverse-the-dependency-from-flink-streaming-java-to-flink-client-tp38469.html
[5] https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Flink-dev-blog-tp38316p38344.html
[7]
https://cwiki.apache.org/confluence/display/FLINK/General+Information+for+Committers
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Disable-Squash-and-merge-button-for-Flink-repository-on-GitHub-tp38497.html
[9]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-110-Support-LIKE-clause-in-CREATE-TABLE-tp38378p38421.html
[10]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Introduce-flink-connector-hive-xx-modules-tp38440.html

Notable Bugs
==========

* [FLINK-16414] [1.10.0] You can not use a UDTF or UDAF without overriding
getResultType(). Fixed for 1.10.1. [11]

[11] https://issues.apache.org/jira/browse/FLINK-16414

flink-packages.org
=============

In the last months, I have covered the contribution of the Pulsar connector
to Apache Flink a couple of times. Currently, it is paused waiting for the
implementation of FLIP-27, the new source interface. Nevertheless, there
has already been a Pulsar Flink Connector maintained by StreamNative for
quite some time. You can check it out on flink-packages.org. [12]

[12] https://flink-packages.org/packages/pulsar-flink-connector

Events, Blog Posts, Misc
===================

* Upcoming Meetups
    * On March 12th, Prateep Kumar will host an online event comparing
Kafka Streams and Apache Flink [13].
    * On April 22, Ververica will host the next Apache Flink meetup in
Berlin. [14]
    * Cloudera is hosting a couple of "Future of Data" events on stream
processing with Apache Flink in
        * Vienna (March 4th, full-day workshop) [15]
        * Zurich (March 10th, full-day workshop) [16]
        * New Jersey (May 5th, meetup) [17]

[13]
https://www.meetup.com/apache-flink-aws-kinesis-hyd-india/events/268930388/
[14] https://www.meetup.com/Apache-Flink-Meetup/events/269005339/
[15] https://www.meetup.com/futureofdata-vienna/events/268418974/
[16] https://www.meetup.com/futureofdata-zurich/events/268423809/
[17] https://www.meetup.com/futureofdata-princeton/events/268830725/

Cheers,

Konstantin (@snntrable)

--

Konstantin Knauf | Head of Product

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng