[DISCUSS] Releasing Flink 1.13.1

[Konstantin Knauf-4]

Hi everyone,

I would like to start discussing Flink 1.13.1. There are already quite a
few critical fixes merged, specifically:

* https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files in
flink-python jars)
* https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop streaming
job with savepoint which uses kinesis consumer)
* https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive Scheduler:
Can not cancel restarting job)

The following two tickets are in progress:

* https://issues.apache.org/jira/browse/FLINK-22502
(DefaultCompletedCheckpointStore
drops unrecoverable checkpoints silently)
* https://issues.apache.org/jira/browse/FLINK-22494 (Avoid discarding
checkpoints in case of failure)

Matthias, Roman: could you give an update on those?

Are there any other open tickets that we should wait for? Is there a PMC
member who would like to manage the release?

Best,

Konstantin

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

[Robert Metzger]

Thanks a lot for starting the discussion about the release.
I'd like to include https://issues.apache.org/jira/browse/FLINK-22266 as
well (I forgot to set the fixVersion accordingly). It's an important fix
for the stop with savepoint operation of the adaptive scheduler.

On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <[hidden email]> wrote:

> Hi everyone,
>
> I would like to start discussing Flink 1.13.1. There are already quite a
> few critical fixes merged, specifically:
>
> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files in
> flink-python jars)
> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop streaming
> job with savepoint which uses kinesis consumer)
> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive Scheduler:
> Can not cancel restarting job)
>
> The following two tickets are in progress:
>
> * https://issues.apache.org/jira/browse/FLINK-22502
> (DefaultCompletedCheckpointStore
> drops unrecoverable checkpoints silently)
> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid discarding
> checkpoints in case of failure)
>
> Matthias, Roman: could you give an update on those?
>
> Are there any other open tickets that we should wait for? Is there a PMC
> member who would like to manage the release?
>
> Best,
>
> Konstantin
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

[Timo Walther-2]

Hi Konstantin,

thanks for starting the discussion. From the Table API side, we also
fixed a couple of critical issues already that justify releasing a
1.13.1 asap.

Personally, I would like to include
https://issues.apache.org/jira/browse/FLINK-22666 that fixes some last
issues with the Scala Table API to DataStream conversion. It should be
fixed today or tomorrow.

Otherwise +1.

Regards,
Timo


On 17.05.21 10:11, Robert Metzger wrote:

> Thanks a lot for starting the discussion about the release.
> I'd like to include https://issues.apache.org/jira/browse/FLINK-22266 as
> well (I forgot to set the fixVersion accordingly). It's an important fix
> for the stop with savepoint operation of the adaptive scheduler.
>
> On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <[hidden email]> wrote:
>
>> Hi everyone,
>>
>> I would like to start discussing Flink 1.13.1. There are already quite a
>> few critical fixes merged, specifically:
>>
>> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files in
>> flink-python jars)
>> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop streaming
>> job with savepoint which uses kinesis consumer)
>> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive Scheduler:
>> Can not cancel restarting job)
>>
>> The following two tickets are in progress:
>>
>> * https://issues.apache.org/jira/browse/FLINK-22502
>> (DefaultCompletedCheckpointStore
>> drops unrecoverable checkpoints silently)
>> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid discarding
>> checkpoints in case of failure)
>>
>> Matthias, Roman: could you give an update on those?
>>
>> Are there any other open tickets that we should wait for? Is there a PMC
>> member who would like to manage the release?
>>
>> Best,
>>
>> Konstantin
>>
>> --
>>
>> Konstantin Knauf
>>
>> https://twitter.com/snntrable
>>
>> https://github.com/knaufk
>>
>

[Arvid Heise-4]

I'd also like to have a fix for
https://issues.apache.org/jira/browse/FLINK-22686 that was reported today.

On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]> wrote:

> Hi Konstantin,
>
> thanks for starting the discussion. From the Table API side, we also
> fixed a couple of critical issues already that justify releasing a
> 1.13.1 asap.
>
> Personally, I would like to include
> https://issues.apache.org/jira/browse/FLINK-22666 that fixes some last
> issues with the Scala Table API to DataStream conversion. It should be
> fixed today or tomorrow.
>
> Otherwise +1.
>
> Regards,
> Timo
>
>
> On 17.05.21 10:11, Robert Metzger wrote:
> > Thanks a lot for starting the discussion about the release.
> > I'd like to include https://issues.apache.org/jira/browse/FLINK-22266 as
> > well (I forgot to set the fixVersion accordingly). It's an important fix
> > for the stop with savepoint operation of the adaptive scheduler.
> >
> > On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <[hidden email]>
> wrote:
> >
> >> Hi everyone,
> >>
> >> I would like to start discussing Flink 1.13.1. There are already quite a
> >> few critical fixes merged, specifically:
> >>
> >> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files in
> >> flink-python jars)
> >> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
> streaming
> >> job with savepoint which uses kinesis consumer)
> >> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
> Scheduler:
> >> Can not cancel restarting job)
> >>
> >> The following two tickets are in progress:
> >>
> >> * https://issues.apache.org/jira/browse/FLINK-22502
> >> (DefaultCompletedCheckpointStore
> >> drops unrecoverable checkpoints silently)
> >> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid discarding
> >> checkpoints in case of failure)
> >>
> >> Matthias, Roman: could you give an update on those?
> >>
> >> Are there any other open tickets that we should wait for? Is there a PMC
> >> member who would like to manage the release?
> >>
> >> Best,
> >>
> >> Konstantin
> >>
> >> --
> >>
> >> Konstantin Knauf
> >>
> >> https://twitter.com/snntrable
> >>
> >> https://github.com/knaufk
> >>
> >
>
>

[Matthias]

Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
discarding checkpoints in case of failure) is reviewed and backported. I
have FLINK-22688 [2] (exception history not working properly for unassigned
tasks) that came up recently. I provided a fix for it [3] already and hope
to get it reviewed soon.

Matthias

[1] https://issues.apache.org/jira/browse/FLINK-22494
[2] https://issues.apache.org/jira/browse/FLINK-22688
[3] https://github.com/apache/flink/pull/15945

On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> wrote:

> I'd also like to have a fix for
> https://issues.apache.org/jira/browse/FLINK-22686 that was reported today.
>
> On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]> wrote:
>
> > Hi Konstantin,
> >
> > thanks for starting the discussion. From the Table API side, we also
> > fixed a couple of critical issues already that justify releasing a
> > 1.13.1 asap.
> >
> > Personally, I would like to include
> > https://issues.apache.org/jira/browse/FLINK-22666 that fixes some last
> > issues with the Scala Table API to DataStream conversion. It should be
> > fixed today or tomorrow.
> >
> > Otherwise +1.
> >
> > Regards,
> > Timo
> >
> >
> > On 17.05.21 10:11, Robert Metzger wrote:
> > > Thanks a lot for starting the discussion about the release.
> > > I'd like to include https://issues.apache.org/jira/browse/FLINK-22266
> as
> > > well (I forgot to set the fixVersion accordingly). It's an important
> fix
> > > for the stop with savepoint operation of the adaptive scheduler.
> > >
> > > On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <[hidden email]>
> > wrote:
> > >
> > >> Hi everyone,
> > >>
> > >> I would like to start discussing Flink 1.13.1. There are already
> quite a
> > >> few critical fixes merged, specifically:
> > >>
> > >> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files
> in
> > >> flink-python jars)
> > >> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
> > streaming
> > >> job with savepoint which uses kinesis consumer)
> > >> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
> > Scheduler:
> > >> Can not cancel restarting job)
> > >>
> > >> The following two tickets are in progress:
> > >>
> > >> * https://issues.apache.org/jira/browse/FLINK-22502
> > >> (DefaultCompletedCheckpointStore
> > >> drops unrecoverable checkpoints silently)
> > >> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid discarding
> > >> checkpoints in case of failure)
> > >>
> > >> Matthias, Roman: could you give an update on those?
> > >>
> > >> Are there any other open tickets that we should wait for? Is there a
> PMC
> > >> member who would like to manage the release?
> > >>
> > >> Best,
> > >>
> > >> Konstantin
> > >>
> > >> --
> > >>
> > >> Konstantin Knauf
> > >>
> > >> https://twitter.com/snntrable
> > >>
> > >> https://github.com/knaufk
> > >>
> > >
> >
> >

[David Morávek-2]

Hi Konstantin,

Would it be possible to add FLINK-22646 [1] into the release? This is a
regression, that we need to workaround in order to support 1.13.x in Apache
Beam [2].

Best,
D.

[1] https://issues.apache.org/jira/browse/FLINK-22646
[2] https://github.com/apache/beam/pull/14719

On Tue, May 18, 2021 at 1:16 PM Matthias Pohl <[hidden email]>
wrote:

> Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
> discarding checkpoints in case of failure) is reviewed and backported. I
> have FLINK-22688 [2] (exception history not working properly for unassigned
> tasks) that came up recently. I provided a fix for it [3] already and hope
> to get it reviewed soon.
>
> Matthias
>
> [1] https://issues.apache.org/jira/browse/FLINK-22494
> [2] https://issues.apache.org/jira/browse/FLINK-22688
> [3] https://github.com/apache/flink/pull/15945
>
> On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> wrote:
>
> > I'd also like to have a fix for
> > https://issues.apache.org/jira/browse/FLINK-22686 that was reported
> today.
> >
> > On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]>
> wrote:
> >
> > > Hi Konstantin,
> > >
> > > thanks for starting the discussion. From the Table API side, we also
> > > fixed a couple of critical issues already that justify releasing a
> > > 1.13.1 asap.
> > >
> > > Personally, I would like to include
> > > https://issues.apache.org/jira/browse/FLINK-22666 that fixes some last
> > > issues with the Scala Table API to DataStream conversion. It should be
> > > fixed today or tomorrow.
> > >
> > > Otherwise +1.
> > >
> > > Regards,
> > > Timo
> > >
> > >
> > > On 17.05.21 10:11, Robert Metzger wrote:
> > > > Thanks a lot for starting the discussion about the release.
> > > > I'd like to include
> https://issues.apache.org/jira/browse/FLINK-22266
> > as
> > > > well (I forgot to set the fixVersion accordingly). It's an important
> > fix
> > > > for the stop with savepoint operation of the adaptive scheduler.
> > > >
> > > > On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <[hidden email]
> >
> > > wrote:
> > > >
> > > >> Hi everyone,
> > > >>
> > > >> I would like to start discussing Flink 1.13.1. There are already
> > quite a
> > > >> few critical fixes merged, specifically:
> > > >>
> > > >> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1 files
> > in
> > > >> flink-python jars)
> > > >> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
> > > streaming
> > > >> job with savepoint which uses kinesis consumer)
> > > >> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
> > > Scheduler:
> > > >> Can not cancel restarting job)
> > > >>
> > > >> The following two tickets are in progress:
> > > >>
> > > >> * https://issues.apache.org/jira/browse/FLINK-22502
> > > >> (DefaultCompletedCheckpointStore
> > > >> drops unrecoverable checkpoints silently)
> > > >> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid
> discarding
> > > >> checkpoints in case of failure)
> > > >>
> > > >> Matthias, Roman: could you give an update on those?
> > > >>
> > > >> Are there any other open tickets that we should wait for? Is there a
> > PMC
> > > >> member who would like to manage the release?
> > > >>
> > > >> Best,
> > > >>
> > > >> Konstantin
> > > >>
> > > >> --
> > > >>
> > > >> Konstantin Knauf
> > > >>
> > > >> https://twitter.com/snntrable
> > > >>
> > > >> https://github.com/knaufk
> > > >>
> > > >
> > >
> > >
>

[Konstantin Knauf-4]

Hi everyone,

Let's see where we stand towards releasing Flink 1.13.1. I am aware of one
additional license-related blocker that popped up:
https://issues.apache.org/jira/browse/FLINK-22706.  Overall:

Done or close to done:

* https://issues.apache.org/jira/browse/FLINK-22666: Done
* https://issues.apache.org/jira/browse/FLINK-22494: Done
* https://issues.apache.org/jira/browse/FLINK-22688: Matthias/Chesnay: PR
available.
* https://issues.apache.org/jira/browse/FLINK-22706

Unclear:

* https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
this a blocker for this bug fix as it only affects unaligned checkpoints in
combination with broadcast state. What do you think? Can you already give
an estimated time to resolution?
* https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert. No
PR yet.
* https://issues.apache.org/jira/browse/FLINK-22646: Chesnay/David: Closed
as "Won't fix", but the PR discussion is not fully concluded.

Is there anything else?

Overall, I would propose to wait until Friday for fixes to come in and
create a release candidate early on Monday next week. I think we've already
accumulated quite a few fixes that justify a patch release. Of course, it'd
be nice if more fixes make it in, in particular FLINK-22686 would be nice
to have so that we don't need a quick follow up patch release soon.

Does any PMC member have time to manage the actual release early next week?

Cheers,

Konstantin


On Tue, May 18, 2021 at 5:17 PM David Morávek <[hidden email]>
wrote:

> Hi Konstantin,
>
> Would it be possible to add FLINK-22646 [1] into the release? This is a
> regression, that we need to workaround in order to support 1.13.x in Apache
> Beam [2].
>
> Best,
> D.
>
> [1] https://issues.apache.org/jira/browse/FLINK-22646
> [2] https://github.com/apache/beam/pull/14719
>
> On Tue, May 18, 2021 at 1:16 PM Matthias Pohl <[hidden email]>
> wrote:
>
> > Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
> > discarding checkpoints in case of failure) is reviewed and backported. I
> > have FLINK-22688 [2] (exception history not working properly for
> unassigned
> > tasks) that came up recently. I provided a fix for it [3] already and
> hope
> > to get it reviewed soon.
> >
> > Matthias
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-22494
> > [2] https://issues.apache.org/jira/browse/FLINK-22688
> > [3] https://github.com/apache/flink/pull/15945
> >
> > On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> wrote:
> >
> > > I'd also like to have a fix for
> > > https://issues.apache.org/jira/browse/FLINK-22686 that was reported
> > today.
> > >
> > > On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]>
> > wrote:
> > >
> > > > Hi Konstantin,
> > > >
> > > > thanks for starting the discussion. From the Table API side, we also
> > > > fixed a couple of critical issues already that justify releasing a
> > > > 1.13.1 asap.
> > > >
> > > > Personally, I would like to include
> > > > https://issues.apache.org/jira/browse/FLINK-22666 that fixes some
> last
> > > > issues with the Scala Table API to DataStream conversion. It should
> be
> > > > fixed today or tomorrow.
> > > >
> > > > Otherwise +1.
> > > >
> > > > Regards,
> > > > Timo
> > > >
> > > >
> > > > On 17.05.21 10:11, Robert Metzger wrote:
> > > > > Thanks a lot for starting the discussion about the release.
> > > > > I'd like to include
> > https://issues.apache.org/jira/browse/FLINK-22266
> > > as
> > > > > well (I forgot to set the fixVersion accordingly). It's an
> important
> > > fix
> > > > > for the stop with savepoint operation of the adaptive scheduler.
> > > > >
> > > > > On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <
> [hidden email]
> > >
> > > > wrote:
> > > > >
> > > > >> Hi everyone,
> > > > >>
> > > > >> I would like to start discussing Flink 1.13.1. There are already
> > > quite a
> > > > >> few critical fixes merged, specifically:
> > > > >>
> > > > >> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1
> files
> > > in
> > > > >> flink-python jars)
> > > > >> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
> > > > streaming
> > > > >> job with savepoint which uses kinesis consumer)
> > > > >> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
> > > > Scheduler:
> > > > >> Can not cancel restarting job)
> > > > >>
> > > > >> The following two tickets are in progress:
> > > > >>
> > > > >> * https://issues.apache.org/jira/browse/FLINK-22502
> > > > >> (DefaultCompletedCheckpointStore
> > > > >> drops unrecoverable checkpoints silently)
> > > > >> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid
> > discarding
> > > > >> checkpoints in case of failure)
> > > > >>
> > > > >> Matthias, Roman: could you give an update on those?
> > > > >>
> > > > >> Are there any other open tickets that we should wait for? Is
> there a
> > > PMC
> > > > >> member who would like to manage the release?
> > > > >>
> > > > >> Best,
> > > > >>
> > > > >> Konstantin
> > > > >>
> > > > >> --
> > > > >>
> > > > >> Konstantin Knauf
> > > > >>
> > > > >> https://twitter.com/snntrable
> > > > >>
> > > > >> https://github.com/knaufk
> > > > >>
> > > > >
> > > >
> > > >
> >
>

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

[dwysakowicz]

* https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
this a blocker for this bug fix as it only affects unaligned checkpoints in
combination with broadcast state. What do you think? Can you already give
an estimated time to resolution?

I forgot to assign myself. I am actually working on it and I am positive we can fix it this week.

If anyone would like to help preparing the RC I am happy to help with the steps that require PMC permissions. If no one volunteers until Monday I can also do it myself. However I'd appreciate if someone else could take care of tracking the progress of the issues we want to include in the release.

Best,
Dawid 

On 20/05/2021 09:59, Konstantin Knauf wrote:

Hi everyone,

Let's see where we stand towards releasing Flink 1.13.1. I am aware of one
additional license-related blocker that popped up:
https://issues.apache.org/jira/browse/FLINK-22706.  Overall:

Done or close to done:

* https://issues.apache.org/jira/browse/FLINK-22666: Done
* https://issues.apache.org/jira/browse/FLINK-22494: Done
* https://issues.apache.org/jira/browse/FLINK-22688: Matthias/Chesnay: PR
available.
* https://issues.apache.org/jira/browse/FLINK-22706

Unclear:

* https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
this a blocker for this bug fix as it only affects unaligned checkpoints in
combination with broadcast state. What do you think? Can you already give
an estimated time to resolution?
* https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert. No
PR yet.
* https://issues.apache.org/jira/browse/FLINK-22646: Chesnay/David: Closed
as "Won't fix", but the PR discussion is not fully concluded.

Is there anything else?

Overall, I would propose to wait until Friday for fixes to come in and
create a release candidate early on Monday next week. I think we've already
accumulated quite a few fixes that justify a patch release. Of course, it'd
be nice if more fixes make it in, in particular FLINK-22686 would be nice
to have so that we don't need a quick follow up patch release soon.

Does any PMC member have time to manage the actual release early next week?

Cheers,

Konstantin


On Tue, May 18, 2021 at 5:17 PM David Morávek [hidden email]
wrote:

Hi Konstantin,

Would it be possible to add FLINK-22646 [1] into the release? This is a
regression, that we need to workaround in order to support 1.13.x in Apache
Beam [2].

Best,
D.

[1] https://issues.apache.org/jira/browse/FLINK-22646
[2] https://github.com/apache/beam/pull/14719

On Tue, May 18, 2021 at 1:16 PM Matthias Pohl [hidden email]
wrote:

Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
discarding checkpoints in case of failure) is reviewed and backported. I
have FLINK-22688 [2] (exception history not working properly for

unassigned

tasks) that came up recently. I provided a fix for it [3] already and

hope

to get it reviewed soon.

Matthias

[1] https://issues.apache.org/jira/browse/FLINK-22494
[2] https://issues.apache.org/jira/browse/FLINK-22688
[3] https://github.com/apache/flink/pull/15945

On Mon, May 17, 2021 at 2:13 PM Arvid Heise [hidden email] wrote:

I'd also like to have a fix for
https://issues.apache.org/jira/browse/FLINK-22686 that was reported

today.

On Mon, May 17, 2021 at 11:52 AM Timo Walther [hidden email]

wrote:

Hi Konstantin,

thanks for starting the discussion. From the Table API side, we also
fixed a couple of critical issues already that justify releasing a
1.13.1 asap.

Personally, I would like to include
https://issues.apache.org/jira/browse/FLINK-22666 that fixes some

last

issues with the Scala Table API to DataStream conversion. It should

be

fixed today or tomorrow.

Otherwise +1.

Regards,
Timo


On 17.05.21 10:11, Robert Metzger wrote:

Thanks a lot for starting the discussion about the release.
I'd like to include

https://issues.apache.org/jira/browse/FLINK-22266

as

well (I forgot to set the fixVersion accordingly). It's an

important

fix

for the stop with savepoint operation of the adaptive scheduler.

On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <

[hidden email]

wrote:

Hi everyone,

I would like to start discussing Flink 1.13.1. There are already

quite a

few critical fixes merged, specifically:

* https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1

files

in

flink-python jars)
* https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop

streaming

job with savepoint which uses kinesis consumer)
* https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive

Scheduler:

Can not cancel restarting job)

The following two tickets are in progress:

* https://issues.apache.org/jira/browse/FLINK-22502
(DefaultCompletedCheckpointStore
drops unrecoverable checkpoints silently)
* https://issues.apache.org/jira/browse/FLINK-22494 (Avoid

discarding

checkpoints in case of failure)

Matthias, Roman: could you give an update on those?

Are there any other open tickets that we should wait for? Is

there a

PMC

member who would like to manage the release?

Best,

Konstantin

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

[Robert Metzger]

Thanks Dawid for helping with the release & Thanks Konstantin for the
summary!


> * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert.
No PR yet.

There's actually a PR (Jira is not showing all links by default, you need
to click "show more"), and it is somewhat close to completion.


On Thu, May 20, 2021 at 10:15 AM Dawid Wysakowicz <[hidden email]>
wrote:

> * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> this a blocker for this bug fix as it only affects unaligned checkpoints in
> combination with broadcast state. What do you think? Can you already give
> an estimated time to resolution?
>
> I forgot to assign myself. I am actually working on it and I am positive we can fix it this week.
>
> If anyone would like to help preparing the RC I am happy to help with the steps that require PMC permissions. If no one volunteers until Monday I can also do it myself. However I'd appreciate if someone else could take care of tracking the progress of the issues we want to include in the release.
>
> Best,
> Dawid
>
> On 20/05/2021 09:59, Konstantin Knauf wrote:
>
> Hi everyone,
>
> Let's see where we stand towards releasing Flink 1.13.1. I am aware of one
> additional license-related blocker that popped up:https://issues.apache.org/jira/browse/FLINK-22706.  Overall:
>
> Done or close to done:
>
> * https://issues.apache.org/jira/browse/FLINK-22666: Done
> * https://issues.apache.org/jira/browse/FLINK-22494: Done
> * https://issues.apache.org/jira/browse/FLINK-22688: Matthias/Chesnay: PR
> available.
> * https://issues.apache.org/jira/browse/FLINK-22706
>
> Unclear:
>
> * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> this a blocker for this bug fix as it only affects unaligned checkpoints in
> combination with broadcast state. What do you think? Can you already give
> an estimated time to resolution?
> * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert. No
> PR yet.
> * https://issues.apache.org/jira/browse/FLINK-22646: Chesnay/David: Closed
> as "Won't fix", but the PR discussion is not fully concluded.
>
> Is there anything else?
>
> Overall, I would propose to wait until Friday for fixes to come in and
> create a release candidate early on Monday next week. I think we've already
> accumulated quite a few fixes that justify a patch release. Of course, it'd
> be nice if more fixes make it in, in particular FLINK-22686 would be nice
> to have so that we don't need a quick follow up patch release soon.
>
> Does any PMC member have time to manage the actual release early next week?
>
> Cheers,
>
> Konstantin
>
>
> On Tue, May 18, 2021 at 5:17 PM David Morávek <[hidden email]> <[hidden email]>
> wrote:
>
>
> Hi Konstantin,
>
> Would it be possible to add FLINK-22646 [1] into the release? This is a
> regression, that we need to workaround in order to support 1.13.x in Apache
> Beam [2].
>
> Best,
> D.
>
> [1] https://issues.apache.org/jira/browse/FLINK-22646
> [2] https://github.com/apache/beam/pull/14719
>
> On Tue, May 18, 2021 at 1:16 PM Matthias Pohl <[hidden email]> <[hidden email]>
> wrote:
>
>
> Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
> discarding checkpoints in case of failure) is reviewed and backported. I
> have FLINK-22688 [2] (exception history not working properly for
>
> unassigned
>
> tasks) that came up recently. I provided a fix for it [3] already and
>
> hope
>
> to get it reviewed soon.
>
> Matthias
>
> [1] https://issues.apache.org/jira/browse/FLINK-22494
> [2] https://issues.apache.org/jira/browse/FLINK-22688
> [3] https://github.com/apache/flink/pull/15945
>
> On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> <[hidden email]> wrote:
>
>
> I'd also like to have a fix forhttps://issues.apache.org/jira/browse/FLINK-22686 that was reported
>
> today.
>
> On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]> <[hidden email]>
>
> wrote:
>
> Hi Konstantin,
>
> thanks for starting the discussion. From the Table API side, we also
> fixed a couple of critical issues already that justify releasing a
> 1.13.1 asap.
>
> Personally, I would like to includehttps://issues.apache.org/jira/browse/FLINK-22666 that fixes some
>
> last
>
> issues with the Scala Table API to DataStream conversion. It should
>
> be
>
> fixed today or tomorrow.
>
> Otherwise +1.
>
> Regards,
> Timo
>
>
> On 17.05.21 10:11, Robert Metzger wrote:
>
> Thanks a lot for starting the discussion about the release.
> I'd like to include
>
> https://issues.apache.org/jira/browse/FLINK-22266
>
> as
>
> well (I forgot to set the fixVersion accordingly). It's an
>
> important
>
> fix
>
> for the stop with savepoint operation of the adaptive scheduler.
>
> On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <
>
> [hidden email]
>
> wrote:
>
> Hi everyone,
>
> I would like to start discussing Flink 1.13.1. There are already
>
> quite a
>
> few critical fixes merged, specifically:
>
> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1
>
> files
>
> in
>
> flink-python jars)
> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
>
> streaming
>
> job with savepoint which uses kinesis consumer)
> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
>
> Scheduler:
>
> Can not cancel restarting job)
>
> The following two tickets are in progress:
>
> * https://issues.apache.org/jira/browse/FLINK-22502
> (DefaultCompletedCheckpointStore
> drops unrecoverable checkpoints silently)
> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid
>
> discarding
>
> checkpoints in case of failure)
>
> Matthias, Roman: could you give an update on those?
>
> Are there any other open tickets that we should wait for? Is
>
> there a
>
> PMC
>
> member who would like to manage the release?
>
> Best,
>
> Konstantin
>
> --
>
> Konstantin Knauf
> https://twitter.com/snntrable
> https://github.com/knaufk
>
>

[Konstantin Knauf-4]

Hi everyone,

Thank you for all the updates. Let me summarize:

Done or close to done:

* https://issues.apache.org/jira/browse/FLINK-22666: Done
* https://issues.apache.org/jira/browse/FLINK-22494: Done
* https://issues.apache.org/jira/browse/FLINK-22688: Approved. Needs Merge.
* https://issues.apache.org/jira/browse/FLINK-22706 Done
* https://issues.apache.org/jira/browse/FLINK-22266: Done

In-Progress:

* https://issues.apache.org/jira/browse/FLINK-22686: Dawid, can you give an
update?
* https://issues.apache.org/jira/browse/FLINK-22646: New PR by Chesnay. I
wouldn't block the release on that one.

I also had a look at other open tickets with fixVersion: 1.3.1.

* Dawid, is it correct to assume that
https://issues.apache.org/jira/browse/FLINK-18934 (Idle stream does not
advance watermark in connected stream) will not make it into this release?
* Stephan, do you want to take the opportunity to merge
https://issues.apache.org/jira/browse/FLINK-22358 before the release?
* Dian, do you want to merge
https://issues.apache.org/jira/browse/FLINK-22733?

Overall, there are 19 open tickets with fixVersion 1.13.1, see [1]. Could
component maintainers please have a look at those and move them to Flink
1.13.2, if you consider them a must have/should have for the next bug fix
release or remove the fixVersion if not.

Thank you,

Konstantin

[1]
https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.13.1


On Thu, May 20, 2021 at 11:38 AM Robert Metzger <[hidden email]> wrote:

> Thanks Dawid for helping with the release & Thanks Konstantin for the
> summary!
>
>
> > * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert.
> No PR yet.
>
> There's actually a PR (Jira is not showing all links by default, you need
> to click "show more"), and it is somewhat close to completion.
>
>
> On Thu, May 20, 2021 at 10:15 AM Dawid Wysakowicz <[hidden email]>
> wrote:
>
> > * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> > Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> > this a blocker for this bug fix as it only affects unaligned checkpoints
> in
> > combination with broadcast state. What do you think? Can you already give
> > an estimated time to resolution?
> >
> > I forgot to assign myself. I am actually working on it and I am positive
> we can fix it this week.
> >
> > If anyone would like to help preparing the RC I am happy to help with
> the steps that require PMC permissions. If no one volunteers until Monday I
> can also do it myself. However I'd appreciate if someone else could take
> care of tracking the progress of the issues we want to include in the
> release.
> >
> > Best,
> > Dawid
> >
> > On 20/05/2021 09:59, Konstantin Knauf wrote:
> >
> > Hi everyone,
> >
> > Let's see where we stand towards releasing Flink 1.13.1. I am aware of
> one
> > additional license-related blocker that popped up:
> https://issues.apache.org/jira/browse/FLINK-22706.  Overall:
> >
> > Done or close to done:
> >
> > * https://issues.apache.org/jira/browse/FLINK-22666: Done
> > * https://issues.apache.org/jira/browse/FLINK-22494: Done
> > * https://issues.apache.org/jira/browse/FLINK-22688: Matthias/Chesnay:
> PR
> > available.
> > * https://issues.apache.org/jira/browse/FLINK-22706
> >
> > Unclear:
> >
> > * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> > Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> > this a blocker for this bug fix as it only affects unaligned checkpoints
> in
> > combination with broadcast state. What do you think? Can you already give
> > an estimated time to resolution?
> > * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to
> Robert. No
> > PR yet.
> > * https://issues.apache.org/jira/browse/FLINK-22646: Chesnay/David:
> Closed
> > as "Won't fix", but the PR discussion is not fully concluded.
> >
> > Is there anything else?
> >
> > Overall, I would propose to wait until Friday for fixes to come in and
> > create a release candidate early on Monday next week. I think we've
> already
> > accumulated quite a few fixes that justify a patch release. Of course,
> it'd
> > be nice if more fixes make it in, in particular FLINK-22686 would be nice
> > to have so that we don't need a quick follow up patch release soon.
> >
> > Does any PMC member have time to manage the actual release early next
> week?
> >
> > Cheers,
> >
> > Konstantin
> >
> >
> > On Tue, May 18, 2021 at 5:17 PM David Morávek <[hidden email]>
> <[hidden email]>
> > wrote:
> >
> >
> > Hi Konstantin,
> >
> > Would it be possible to add FLINK-22646 [1] into the release? This is a
> > regression, that we need to workaround in order to support 1.13.x in
> Apache
> > Beam [2].
> >
> > Best,
> > D.
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-22646
> > [2] https://github.com/apache/beam/pull/14719
> >
> > On Tue, May 18, 2021 at 1:16 PM Matthias Pohl <[hidden email]> <
> [hidden email]>
> > wrote:
> >
> >
> > Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
> > discarding checkpoints in case of failure) is reviewed and backported. I
> > have FLINK-22688 [2] (exception history not working properly for
> >
> > unassigned
> >
> > tasks) that came up recently. I provided a fix for it [3] already and
> >
> > hope
> >
> > to get it reviewed soon.
> >
> > Matthias
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-22494
> > [2] https://issues.apache.org/jira/browse/FLINK-22688
> > [3] https://github.com/apache/flink/pull/15945
> >
> > On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> <
> [hidden email]> wrote:
> >
> >
> > I'd also like to have a fix forhttps://
> issues.apache.org/jira/browse/FLINK-22686 that was reported
> >
> > today.
> >
> > On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]> <
> [hidden email]>
> >
> > wrote:
> >
> > Hi Konstantin,
> >
> > thanks for starting the discussion. From the Table API side, we also
> > fixed a couple of critical issues already that justify releasing a
> > 1.13.1 asap.
> >
> > Personally, I would like to includehttps://
> issues.apache.org/jira/browse/FLINK-22666 that fixes some
> >
> > last
> >
> > issues with the Scala Table API to DataStream conversion. It should
> >
> > be
> >
> > fixed today or tomorrow.
> >
> > Otherwise +1.
> >
> > Regards,
> > Timo
> >
> >
> > On 17.05.21 10:11, Robert Metzger wrote:
> >
> > Thanks a lot for starting the discussion about the release.
> > I'd like to include
> >
> > https://issues.apache.org/jira/browse/FLINK-22266
> >
> > as
> >
> > well (I forgot to set the fixVersion accordingly). It's an
> >
> > important
> >
> > fix
> >
> > for the stop with savepoint operation of the adaptive scheduler.
> >
> > On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <
> >
> > [hidden email]
> >
> > wrote:
> >
> > Hi everyone,
> >
> > I would like to start discussing Flink 1.13.1. There are already
> >
> > quite a
> >
> > few critical fixes merged, specifically:
> >
> > * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1
> >
> > files
> >
> > in
> >
> > flink-python jars)
> > * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
> >
> > streaming
> >
> > job with savepoint which uses kinesis consumer)
> > * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
> >
> > Scheduler:
> >
> > Can not cancel restarting job)
> >
> > The following two tickets are in progress:
> >
> > * https://issues.apache.org/jira/browse/FLINK-22502
> > (DefaultCompletedCheckpointStore
> > drops unrecoverable checkpoints silently)
> > * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid
> >
> > discarding
> >
> > checkpoints in case of failure)
> >
> > Matthias, Roman: could you give an update on those?
> >
> > Are there any other open tickets that we should wait for? Is
> >
> > there a
> >
> > PMC
> >
> > member who would like to manage the release?
> >
> > Best,
> >
> > Konstantin
> >
> > --
> >
> > Konstantin Knauf
> > https://twitter.com/snntrable
> > https://github.com/knaufk
> >
> >
>

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

[dwysakowicz]

Hey,

As for FLINK-22686, we discovered that fixing it would uncover a more severe problem so we decided to not include it for now. I tried describing the problem in the JIRA ticket.

FLINK-18934 changes the contract of StreamStatus a tiny bit. Moreover it exposed the StreamStatus to somewhat public operator interfaces so I am inclined to do it for 1.14, instead of in a bugfix version. It is almost done, so it will certainly make it into 1.14. The remaining comments are just about code structure and not about the implemented solution.

I merged two other issues to 1.13.1:

* https://issues.apache.org/jira/browse/FLINK-22638

* https://issues.apache.org/jira/browse/FLINK-22708

That said, I don't have any other issues I'd like to include in the release.

Lastly, thanks for helping with keeping track of the progress!

Best,

Dawid

On 21/05/2021 09:48, Konstantin Knauf wrote:

Hi everyone, 

Thank you for all the updates. Let me summarize:

Done or close to done: 

* https://issues.apache.org/jira/browse/FLINK-22666: Done
* https://issues.apache.org/jira/browse/FLINK-22494: Done

* https://issues.apache.org/jira/browse/FLINK-22688: Approved. Needs Merge.

* https://issues.apache.org/jira/browse/FLINK-22706 Done

* https://issues.apache.org/jira/browse/FLINK-22266: Done

In-Progress:

* https://issues.apache.org/jira/browse/FLINK-22686: Dawid, can you give an update?

* https://issues.apache.org/jira/browse/FLINK-22646: New PR by Chesnay. I wouldn't block the release on that one. 

I also had a look at other open tickets with fixVersion: 1.3.1.

* Dawid, is it correct to assume that https://issues.apache.org/jira/browse/FLINK-18934 (Idle stream does not advance watermark in connected stream) will not make it into this release?
* Stephan, do you want to take the opportunity to merge https://issues.apache.org/jira/browse/FLINK-22358 before the release?

* Dian, do you want to merge https://issues.apache.org/jira/browse/FLINK-22733?

Overall, there are 19 open tickets with fixVersion 1.13.1, see [1]. Could component maintainers please have a look at those and move them to Flink 1.13.2, if you consider them a must have/should have for the next bug fix release or remove the fixVersion if not. 

Thank you, 

Konstantin

[1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.13.1

On Thu, May 20, 2021 at 11:38 AM Robert Metzger <[hidden email]> wrote:

Thanks Dawid for helping with the release & Thanks Konstantin for the
summary!


> * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert.
No PR yet.

There's actually a PR (Jira is not showing all links by default, you need
to click "show more"), and it is somewhat close to completion.


On Thu, May 20, 2021 at 10:15 AM Dawid Wysakowicz <[hidden email]>
wrote:

> * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> this a blocker for this bug fix as it only affects unaligned checkpoints in
> combination with broadcast state. What do you think? Can you already give
> an estimated time to resolution?
>
> I forgot to assign myself. I am actually working on it and I am positive we can fix it this week.
>
> If anyone would like to help preparing the RC I am happy to help with the steps that require PMC permissions. If no one volunteers until Monday I can also do it myself. However I'd appreciate if someone else could take care of tracking the progress of the issues we want to include in the release.
>
> Best,
> Dawid
>
> On 20/05/2021 09:59, Konstantin Knauf wrote:
>
> Hi everyone,
>
> Let's see where we stand towards releasing Flink 1.13.1. I am aware of one
> additional license-related blocker that popped up:https://issues.apache.org/jira/browse/FLINK-22706.  Overall:
>
> Done or close to done:
>
> * https://issues.apache.org/jira/browse/FLINK-22666: Done
> * https://issues.apache.org/jira/browse/FLINK-22494: Done
> * https://issues.apache.org/jira/browse/FLINK-22688: Matthias/Chesnay: PR
> available.
> * https://issues.apache.org/jira/browse/FLINK-22706
>
> Unclear:
>
> * https://issues.apache.org/jira/browse/FLINK-22686: This is marked as
> Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make
> this a blocker for this bug fix as it only affects unaligned checkpoints in
> combination with broadcast state. What do you think? Can you already give
> an estimated time to resolution?
> * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert. No
> PR yet.
> * https://issues.apache.org/jira/browse/FLINK-22646: Chesnay/David: Closed
> as "Won't fix", but the PR discussion is not fully concluded.
>
> Is there anything else?
>
> Overall, I would propose to wait until Friday for fixes to come in and
> create a release candidate early on Monday next week. I think we've already
> accumulated quite a few fixes that justify a patch release. Of course, it'd
> be nice if more fixes make it in, in particular FLINK-22686 would be nice
> to have so that we don't need a quick follow up patch release soon.
>
> Does any PMC member have time to manage the actual release early next week?
>
> Cheers,
>
> Konstantin
>
>
> On Tue, May 18, 2021 at 5:17 PM David Morávek <[hidden email]> <[hidden email]>
> wrote:
>
>
> Hi Konstantin,
>
> Would it be possible to add FLINK-22646 [1] into the release? This is a
> regression, that we need to workaround in order to support 1.13.x in Apache
> Beam [2].
>
> Best,
> D.
>
> [1] https://issues.apache.org/jira/browse/FLINK-22646
> [2] https://github.com/apache/beam/pull/14719
>
> On Tue, May 18, 2021 at 1:16 PM Matthias Pohl <[hidden email]> <[hidden email]>
> wrote:
>
>
> Thanks for initiating this discussion, Konstantin. FLINK-22494 [1] (Avoid
> discarding checkpoints in case of failure) is reviewed and backported. I
> have FLINK-22688 [2] (exception history not working properly for
>
> unassigned
>
> tasks) that came up recently. I provided a fix for it [3] already and
>
> hope
>
> to get it reviewed soon.
>
> Matthias
>
> [1] https://issues.apache.org/jira/browse/FLINK-22494
> [2] https://issues.apache.org/jira/browse/FLINK-22688
> [3] https://github.com/apache/flink/pull/15945
>
> On Mon, May 17, 2021 at 2:13 PM Arvid Heise <[hidden email]> <[hidden email]> wrote:
>
>
> I'd also like to have a fix forhttps://issues.apache.org/jira/browse/FLINK-22686 that was reported
>
> today.
>
> On Mon, May 17, 2021 at 11:52 AM Timo Walther <[hidden email]> <[hidden email]>
>
> wrote:
>
> Hi Konstantin,
>
> thanks for starting the discussion. From the Table API side, we also
> fixed a couple of critical issues already that justify releasing a
> 1.13.1 asap.
>
> Personally, I would like to includehttps://issues.apache.org/jira/browse/FLINK-22666 that fixes some
>
> last
>
> issues with the Scala Table API to DataStream conversion. It should
>
> be
>
> fixed today or tomorrow.
>
> Otherwise +1.
>
> Regards,
> Timo
>
>
> On 17.05.21 10:11, Robert Metzger wrote:
>
> Thanks a lot for starting the discussion about the release.
> I'd like to include
>
> https://issues.apache.org/jira/browse/FLINK-22266
>
> as
>
> well (I forgot to set the fixVersion accordingly). It's an
>
> important
>
> fix
>
> for the stop with savepoint operation of the adaptive scheduler.
>
> On Mon, May 17, 2021 at 10:03 AM Konstantin Knauf <
>
> [hidden email]
>
> wrote:
>
> Hi everyone,
>
> I would like to start discussing Flink 1.13.1. There are already
>
> quite a
>
> few critical fixes merged, specifically:
>
> * https://issues.apache.org/jira/browse/FLINK-22555 (LGPL-2.1
>
> files
>
> in
>
> flink-python jars)
> * https://issues.apache.org/jira/browse/FLINK-17170 (Cannot stop
>
> streaming
>
> job with savepoint which uses kinesis consumer)
> * https://issues.apache.org/jira/browse/FLINK-22574 (Adaptive
>
> Scheduler:
>
> Can not cancel restarting job)
>
> The following two tickets are in progress:
>
> * https://issues.apache.org/jira/browse/FLINK-22502
> (DefaultCompletedCheckpointStore
> drops unrecoverable checkpoints silently)
> * https://issues.apache.org/jira/browse/FLINK-22494 (Avoid
>
> discarding
>
> checkpoints in case of failure)
>
> Matthias, Roman: could you give an update on those?
>
> Are there any other open tickets that we should wait for? Is
>
> there a
>
> PMC
>
> member who would like to manage the release?
>
> Best,
>
> Konstantin
>
> --
>
> Konstantin Knauf
> https://twitter.com/snntrable
> https://github.com/knaufk
>
>

--