(DEPRECATED) Apache Flink Mailing List archive.

[DISCUSS] Releasing Flink 1.11.2

Classic

List

Threaded

18 messages Options

Zhu Zhu

[DISCUSS] Releasing Flink 1.11.2

Hi All,

It has been about 1 month since we released Flink 1.11.1. It's not too far
but
we already have more than 80 resolved improvements/bugs in the release-1.11
branch. Some of them are quite critical. Therefore, I propose to create the
next
bugfix release 1.11.2 for Flink 1.11.

Most noticeable fixes are:
- FLINK-18769 MiniBatch doesn't work with FLIP-95 source
- FLINK-18902 Cannot serve results of asynchronous REST operations in
per-job mode
- FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
- FLINK-18608 CustomizedConvertRule#convertCast drops nullability
- FLINK-18646 Managed memory released check can block RPC thread
- FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
JobManagerFlinkMemoryUtils.java
- FLINK-18663 RestServerEndpoint may prevent server shutdown
- FLINK-18595 Deadlock during job shutdown
- FLINK-18581 Cannot find GC cleaner with java version previous
jdk8u72(-b01)
- FLINK-17075 Add task status reconciliation between TM and JM

Furthermore, I think the following blocker issue should be merged before
1.11.2 release

- FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory

I would volunteer as the release manager and kick off the release process.
What do you think?

Please let me know if there are any concerns or any missing blocker issues
need to be fixed in 1.11.2.

Thanks,
Zhu Zhu

David Anderson-3

Re: [DISCUSS] Releasing Flink 1.11.2

I think it's worth considering whether we can get this bugfix included in
1.11.2:

- FLINK-19109 Split Reader eats chained periodic watermarks

There is a PR, but it's still a work in progress. Cc'ing Roman, who has
been working on this.

Regards,
David

On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:

> Hi All,
>
> It has been about 1 month since we released Flink 1.11.1. It's not too far
> but
> we already have more than 80 resolved improvements/bugs in the release-1.11
> branch. Some of them are quite critical. Therefore, I propose to create the
> next
> bugfix release 1.11.2 for Flink 1.11.
>
> Most noticeable fixes are:
> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> - FLINK-18902 Cannot serve results of asynchronous REST operations in
> per-job mode
> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> - FLINK-18646 Managed memory released check can block RPC thread
> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> JobManagerFlinkMemoryUtils.java
> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> - FLINK-18595 Deadlock during job shutdown
> - FLINK-18581 Cannot find GC cleaner with java version previous
> jdk8u72(-b01)
> - FLINK-17075 Add task status reconciliation between TM and JM
>
> Furthermore, I think the following blocker issue should be merged before
> 1.11.2 release
>
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
>
> I would volunteer as the release manager and kick off the release process.
> What do you think?
>
> Please let me know if there are any concerns or any missing blocker issues
> need to be fixed in 1.11.2.
>
> Thanks,
> Zhu Zhu
>

Till Rohrmann

Re: [DISCUSS] Releasing Flink 1.11.2

Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
the release manager. Big +1 for creating the next 1.11 bug fix release. I
think we already collected quite a bit of fixes which our users will
benefit from.

For the pending fixes, I would suggest setting a soft deadline (maybe until
beginning of next week) and then starting to cut the release (given that no
other blocker issues pop up). Maybe we are able to resolve the issues even
earlier which would allow us to cut the release also earlier.

From my side I would like to include FLINK-18959 in the release. But it is
not a strict release blocker.

Cheers,
Till

On Wed, Sep 2, 2020 at 2:40 PM David Anderson <[hidden email]> wrote:

> I think it's worth considering whether we can get this bugfix included in
> 1.11.2:
>
> - FLINK-19109 Split Reader eats chained periodic watermarks
>
> There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> been working on this.
>
> Regards,
> David
>
>
> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
>
> > Hi All,
> >
> > It has been about 1 month since we released Flink 1.11.1. It's not too
> far
> > but
> > we already have more than 80 resolved improvements/bugs in the
> release-1.11
> > branch. Some of them are quite critical. Therefore, I propose to create
> the
> > next
> > bugfix release 1.11.2 for Flink 1.11.
> >
> > Most noticeable fixes are:
> > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > per-job mode
> > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > - FLINK-18646 Managed memory released check can block RPC thread
> > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> > JobManagerFlinkMemoryUtils.java
> > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > - FLINK-18595 Deadlock during job shutdown
> > - FLINK-18581 Cannot find GC cleaner with java version previous
> > jdk8u72(-b01)
> > - FLINK-17075 Add task status reconciliation between TM and JM
> >
> > Furthermore, I think the following blocker issue should be merged before
> > 1.11.2 release
> >
> > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> >
> > I would volunteer as the release manager and kick off the release
> process.
> > What do you think?
> >
> > Please let me know if there are any concerns or any missing blocker
> issues
> > need to be fixed in 1.11.2.
> >
> > Thanks,
> > Zhu Zhu
> >
>

Konstantin Knauf-4

Re: [DISCUSS] Releasing Flink 1.11.2

I think it would be nice to include a fix for
https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
highly requested feature of Flink 1.11 quite severely.

On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]> wrote:

> Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
> the release manager. Big +1 for creating the next 1.11 bug fix release. I
> think we already collected quite a bit of fixes which our users will
> benefit from.
>
> For the pending fixes, I would suggest setting a soft deadline (maybe until
> beginning of next week) and then starting to cut the release (given that no
> other blocker issues pop up). Maybe we are able to resolve the issues even
> earlier which would allow us to cut the release also earlier.
>
> From my side I would like to include FLINK-18959 in the release. But it is
> not a strict release blocker.
>
> Cheers,
> Till
>
> On Wed, Sep 2, 2020 at 2:40 PM David Anderson <[hidden email]>
> wrote:
>
> > I think it's worth considering whether we can get this bugfix included in
> > 1.11.2:
> >
> > - FLINK-19109 Split Reader eats chained periodic watermarks
> >
> > There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> > been working on this.
> >
> > Regards,
> > David
> >
> >
> > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
> >
> > > Hi All,
> > >
> > > It has been about 1 month since we released Flink 1.11.1. It's not too
> > far
> > > but
> > > we already have more than 80 resolved improvements/bugs in the
> > release-1.11
> > > branch. Some of them are quite critical. Therefore, I propose to create
> > the
> > > next
> > > bugfix release 1.11.2 for Flink 1.11.
> > >
> > > Most noticeable fixes are:
> > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > > per-job mode
> > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > - FLINK-18646 Managed memory released check can block RPC thread
> > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> > > JobManagerFlinkMemoryUtils.java
> > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > - FLINK-18595 Deadlock during job shutdown
> > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > jdk8u72(-b01)
> > > - FLINK-17075 Add task status reconciliation between TM and JM
> > >
> > > Furthermore, I think the following blocker issue should be merged
> before
> > > 1.11.2 release
> > >
> > > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > >
> > > I would volunteer as the release manager and kick off the release
> > process.
> > > What do you think?
> > >
> > > Please let me know if there are any concerns or any missing blocker
> > issues
> > > need to be fixed in 1.11.2.
> > >
> > > Thanks,
> > > Zhu Zhu
> > >
> >
>

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

Thank you all for the inputs!

I agree with Till that we should set a soft deadline first.
I'd like to propose next Monday if no new blocker issue pops up.
But feel free to raise your concerns if you feel next Monday as a deadline
may not work
for fixes which should be a blocker for 1.11.2.

Here's a summary of the wanted/blocker but still open fixes:
- FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
- FLINK-19109 Split Reader eats chained periodic watermarks
- (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph because
job is not finished when dispatcher close
- FLINK-18934 Idle stream does not advance watermark in connected stream

Thanks,
Zhu

Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：

> I think it would be nice to include a fix for
> https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
> highly requested feature of Flink 1.11 quite severely.
>
> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]> wrote:
>
> > Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
> > the release manager. Big +1 for creating the next 1.11 bug fix release. I
> > think we already collected quite a bit of fixes which our users will
> > benefit from.
> >
> > For the pending fixes, I would suggest setting a soft deadline (maybe
> until
> > beginning of next week) and then starting to cut the release (given that
> no
> > other blocker issues pop up). Maybe we are able to resolve the issues
> even
> > earlier which would allow us to cut the release also earlier.
> >
> > From my side I would like to include FLINK-18959 in the release. But it
> is
> > not a strict release blocker.
> >
> > Cheers,
> > Till
> >
> > On Wed, Sep 2, 2020 at 2:40 PM David Anderson <[hidden email]>
> > wrote:
> >
> > > I think it's worth considering whether we can get this bugfix included
> in
> > > 1.11.2:
> > >
> > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > >
> > > There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> > > been working on this.
> > >
> > > Regards,
> > > David
> > >
> > >
> > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
> > >
> > > > Hi All,
> > > >
> > > > It has been about 1 month since we released Flink 1.11.1. It's not
> too
> > > far
> > > > but
> > > > we already have more than 80 resolved improvements/bugs in the
> > > release-1.11
> > > > branch. Some of them are quite critical. Therefore, I propose to
> create
> > > the
> > > > next
> > > > bugfix release 1.11.2 for Flink 1.11.
> > > >
> > > > Most noticeable fixes are:
> > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > > > per-job mode
> > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > > - FLINK-18646 Managed memory released check can block RPC thread
> > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly
> in
> > > > JobManagerFlinkMemoryUtils.java
> > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > - FLINK-18595 Deadlock during job shutdown
> > > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > > jdk8u72(-b01)
> > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > >
> > > > Furthermore, I think the following blocker issue should be merged
> > before
> > > > 1.11.2 release
> > > >
> > > > - FLINK-19121 Avoid accessing HDFS frequently in
> HiveBulkWriterFactory
> > > >
> > > > I would volunteer as the release manager and kick off the release
> > > process.
> > > > What do you think?
> > > >
> > > > Please let me know if there are any concerns or any missing blocker
> > > issues
> > > > need to be fixed in 1.11.2.
> > > >
> > > > Thanks,
> > > > Zhu Zhu
> > > >
> > >
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

Zhijiang(wangzhijiang999)

Re: [DISCUSS] Releasing Flink 1.11.2

In reply to this post by Konstantin Knauf-4

Thanks for launching this discussion and volunteering as the release manager.
+1 on my side and I am willing to provide any help during the release procedure, :)

Best,
Zhijiang

------------------------------------------------------------------
From:Konstantin Knauf <[hidden email]>
Send Time:2020年9月2日(星期三) 23:44
To:dev <[hidden email]>
Cc:khachatryan.roman <[hidden email]>
Subject:Re: [DISCUSS] Releasing Flink 1.11.2

I think it would be nice to include a fix for
https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
highly requested feature of Flink 1.11 quite severely.

On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]> wrote:

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

Becket Qin

Re: [DISCUSS] Releasing Flink 1.11.2

In reply to this post by Zhu Zhu

Hi Zhuzhu,

Thanks for starting the discussion.

I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
from previous versions and is currently blocking the development of Pravega
source on top of FLIP-27.

Thanks,

Jiangjie (Becket) Qin

On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:

> Thank you all for the inputs!
>
> I agree with Till that we should set a soft deadline first.
> I'd like to propose next Monday if no new blocker issue pops up.
> But feel free to raise your concerns if you feel next Monday as a deadline
> may not work
> for fixes which should be a blocker for 1.11.2.
>
> Here's a summary of the wanted/blocker but still open fixes:
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> - FLINK-19109 Split Reader eats chained periodic watermarks
> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph because
> job is not finished when dispatcher close
> - FLINK-18934 Idle stream does not advance watermark in connected stream
>
> Thanks,
> Zhu
>
> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>
> > I think it would be nice to include a fix for
> > https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
> > highly requested feature of Flink 1.11 quite severely.
> >
> > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
> wrote:
> >
> > > Thanks a lot for starting this discussion Zhu Zhu and for volunteering
> as
> > > the release manager. Big +1 for creating the next 1.11 bug fix
> release. I
> > > think we already collected quite a bit of fixes which our users will
> > > benefit from.
> > >
> > > For the pending fixes, I would suggest setting a soft deadline (maybe
> > until
> > > beginning of next week) and then starting to cut the release (given
> that
> > no
> > > other blocker issues pop up). Maybe we are able to resolve the issues
> > even
> > > earlier which would allow us to cut the release also earlier.
> > >
> > > From my side I would like to include FLINK-18959 in the release. But it
> > is
> > > not a strict release blocker.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson <[hidden email]>
> > > wrote:
> > >
> > > > I think it's worth considering whether we can get this bugfix
> included
> > in
> > > > 1.11.2:
> > > >
> > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > >
> > > > There is a PR, but it's still a work in progress. Cc'ing Roman, who
> has
> > > > been working on this.
> > > >
> > > > Regards,
> > > > David
> > > >
> > > >
> > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > It has been about 1 month since we released Flink 1.11.1. It's not
> > too
> > > > far
> > > > > but
> > > > > we already have more than 80 resolved improvements/bugs in the
> > > > release-1.11
> > > > > branch. Some of them are quite critical. Therefore, I propose to
> > create
> > > > the
> > > > > next
> > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > >
> > > > > Most noticeable fixes are:
> > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > - FLINK-18902 Cannot serve results of asynchronous REST operations
> in
> > > > > per-job mode
> > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > > > - FLINK-18646 Managed memory released check can block RPC thread
> > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly
> > in
> > > > > JobManagerFlinkMemoryUtils.java
> > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > > > jdk8u72(-b01)
> > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > >
> > > > > Furthermore, I think the following blocker issue should be merged
> > > before
> > > > > 1.11.2 release
> > > > >
> > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > HiveBulkWriterFactory
> > > > >
> > > > > I would volunteer as the release manager and kick off the release
> > > > process.
> > > > > What do you think?
> > > > >
> > > > > Please let me know if there are any concerns or any missing blocker
> > > > issues
> > > > > need to be fixed in 1.11.2.
> > > > >
> > > > > Thanks,
> > > > > Zhu Zhu
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>

Zhou, Brian

RE: [DISCUSS] Releasing Flink 1.11.2

Hi,

Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for Pravega connector integration, hope we can have it included in 1.11.2 release.

Best Regards,
Brian

-----Original Message-----
From: Becket Qin <[hidden email]>
Sent: Thursday, September 3, 2020 11:18
To: dev
Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
Subject: Re: [DISCUSS] Releasing Flink 1.11.2

[EXTERNAL EMAIL]

Hi Zhuzhu,

Thanks for starting the discussion.

I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression from previous versions and is currently blocking the development of Pravega source on top of FLIP-27.

Thanks,

Jiangjie (Becket) Qin

On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:

> Thank you all for the inputs!
>
> I agree with Till that we should set a soft deadline first.
> I'd like to propose next Monday if no new blocker issue pops up.
> But feel free to raise your concerns if you feel next Monday as a
> deadline may not work for fixes which should be a blocker for 1.11.2.
>
> Here's a summary of the wanted/blocker but still open fixes:
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> - FLINK-19109 Split Reader eats chained periodic watermarks
> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> because job is not finished when dispatcher close
> - FLINK-18934 Idle stream does not advance watermark in connected
> stream
>
> Thanks,
> Zhu
>
> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>
> > I think it would be nice to include a fix for
> > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > affects a highly requested feature of Flink 1.11 quite severely.
> >
> > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
> wrote:
> >
> > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > volunteering
> as
> > > the release manager. Big +1 for creating the next 1.11 bug fix
> release. I
> > > think we already collected quite a bit of fixes which our users
> > > will benefit from.
> > >
> > > For the pending fixes, I would suggest setting a soft deadline
> > > (maybe
> > until
> > > beginning of next week) and then starting to cut the release
> > > (given
> that
> > no
> > > other blocker issues pop up). Maybe we are able to resolve the
> > > issues
> > even
> > > earlier which would allow us to cut the release also earlier.
> > >
> > > From my side I would like to include FLINK-18959 in the release.
> > > But it
> > is
> > > not a strict release blocker.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > <[hidden email]>
> > > wrote:
> > >
> > > > I think it's worth considering whether we can get this bugfix
> included
> > in
> > > > 1.11.2:
> > > >
> > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > >
> > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > who
> has
> > > > been working on this.
> > > >
> > > > Regards,
> > > > David
> > > >
> > > >
> > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > not
> > too
> > > > far
> > > > > but
> > > > > we already have more than 80 resolved improvements/bugs in the
> > > > release-1.11
> > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > to
> > create
> > > > the
> > > > > next
> > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > >
> > > > > Most noticeable fixes are:
> > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > operations
> in
> > > > > per-job mode
> > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > nullability
> > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > thread
> > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > incorrectly
> > in
> > > > > JobManagerFlinkMemoryUtils.java
> > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > previous
> > > > > jdk8u72(-b01)
> > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > >
> > > > > Furthermore, I think the following blocker issue should be
> > > > > merged
> > > before
> > > > > 1.11.2 release
> > > > >
> > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > HiveBulkWriterFactory
> > > > >
> > > > > I would volunteer as the release manager and kick off the
> > > > > release
> > > > process.
> > > > > What do you think?
> > > > >
> > > > > Please let me know if there are any concerns or any missing
> > > > > blocker
> > > > issues
> > > > > need to be fixed in 1.11.2.
> > > > >
> > > > > Thanks,
> > > > > Zhu Zhu
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>

Congxian Qiu

Re: [DISCUSS] Releasing Flink 1.11.2

Hi
I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
bug introduce in 1.11.0), there is a pr for it already.
Best,
Congxian

Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：

> Hi,
>
> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
> Pravega connector integration, hope we can have it included in 1.11.2
> release.
>
> Best Regards,
> Brian
>
> -----Original Message-----
> From: Becket Qin <[hidden email]>
> Sent: Thursday, September 3, 2020 11:18
> To: dev
> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>
>
> [EXTERNAL EMAIL]
>
> Hi Zhuzhu,
>
> Thanks for starting the discussion.
>
> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> from previous versions and is currently blocking the development of Pravega
> source on top of FLIP-27.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
>
> > Thank you all for the inputs!
> >
> > I agree with Till that we should set a soft deadline first.
> > I'd like to propose next Monday if no new blocker issue pops up.
> > But feel free to raise your concerns if you feel next Monday as a
> > deadline may not work for fixes which should be a blocker for 1.11.2.
> >
> > Here's a summary of the wanted/blocker but still open fixes:
> > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > - FLINK-19109 Split Reader eats chained periodic watermarks
> > - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> > because job is not finished when dispatcher close
> > - FLINK-18934 Idle stream does not advance watermark in connected
> > stream
> >
> > Thanks,
> > Zhu
> >
> > Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
> >
> > > I think it would be nice to include a fix for
> > > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > > affects a highly requested feature of Flink 1.11 quite severely.
> > >
> > > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
> > wrote:
> > >
> > > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > > volunteering
> > as
> > > > the release manager. Big +1 for creating the next 1.11 bug fix
> > release. I
> > > > think we already collected quite a bit of fixes which our users
> > > > will benefit from.
> > > >
> > > > For the pending fixes, I would suggest setting a soft deadline
> > > > (maybe
> > > until
> > > > beginning of next week) and then starting to cut the release
> > > > (given
> > that
> > > no
> > > > other blocker issues pop up). Maybe we are able to resolve the
> > > > issues
> > > even
> > > > earlier which would allow us to cut the release also earlier.
> > > >
> > > > From my side I would like to include FLINK-18959 in the release.
> > > > But it
> > > is
> > > > not a strict release blocker.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > > <[hidden email]>
> > > > wrote:
> > > >
> > > > > I think it's worth considering whether we can get this bugfix
> > included
> > > in
> > > > > 1.11.2:
> > > > >
> > > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > > >
> > > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > > who
> > has
> > > > > been working on this.
> > > > >
> > > > > Regards,
> > > > > David
> > > > >
> > > > >
> > > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]> wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > > not
> > > too
> > > > > far
> > > > > > but
> > > > > > we already have more than 80 resolved improvements/bugs in the
> > > > > release-1.11
> > > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > > to
> > > create
> > > > > the
> > > > > > next
> > > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > > >
> > > > > > Most noticeable fixes are:
> > > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > > operations
> > in
> > > > > > per-job mode
> > > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > > nullability
> > > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > > thread
> > > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > > incorrectly
> > > in
> > > > > > JobManagerFlinkMemoryUtils.java
> > > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > > previous
> > > > > > jdk8u72(-b01)
> > > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > > >
> > > > > > Furthermore, I think the following blocker issue should be
> > > > > > merged
> > > > before
> > > > > > 1.11.2 release
> > > > > >
> > > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > > HiveBulkWriterFactory
> > > > > >
> > > > > > I would volunteer as the release manager and kick off the
> > > > > > release
> > > > > process.
> > > > > > What do you think?
> > > > > >
> > > > > > Please let me know if there are any concerns or any missing
> > > > > > blocker
> > > > > issues
> > > > > > need to be fixed in 1.11.2.
> > > > > >
> > > > > > Thanks,
> > > > > > Zhu Zhu
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

Thanks for the inputs!
I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.

And thanks a lot for offering help, zhijiang!

Thanks,
Zhu

Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：

> Hi
> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
> bug introduce in 1.11.0), there is a pr for it already.
> Best,
> Congxian
>
>
> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>
> > Hi,
> >
> > Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
> > Pravega connector integration, hope we can have it included in 1.11.2
> > release.
> >
> > Best Regards,
> > Brian
> >
> > -----Original Message-----
> > From: Becket Qin <[hidden email]>
> > Sent: Thursday, September 3, 2020 11:18
> > To: dev
> > Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
> > Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >
> >
> > [EXTERNAL EMAIL]
> >
> > Hi Zhuzhu,
> >
> > Thanks for starting the discussion.
> >
> > I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> > from previous versions and is currently blocking the development of
> Pravega
> > source on top of FLIP-27.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
> >
> > > Thank you all for the inputs!
> > >
> > > I agree with Till that we should set a soft deadline first.
> > > I'd like to propose next Monday if no new blocker issue pops up.
> > > But feel free to raise your concerns if you feel next Monday as a
> > > deadline may not work for fixes which should be a blocker for 1.11.2.
> > >
> > > Here's a summary of the wanted/blocker but still open fixes:
> > > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> > > because job is not finished when dispatcher close
> > > - FLINK-18934 Idle stream does not advance watermark in connected
> > > stream
> > >
> > > Thanks,
> > > Zhu
> > >
> > > Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
> > >
> > > > I think it would be nice to include a fix for
> > > > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > > > affects a highly requested feature of Flink 1.11 quite severely.
> > > >
> > > > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
> > > wrote:
> > > >
> > > > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > > > volunteering
> > > as
> > > > > the release manager. Big +1 for creating the next 1.11 bug fix
> > > release. I
> > > > > think we already collected quite a bit of fixes which our users
> > > > > will benefit from.
> > > > >
> > > > > For the pending fixes, I would suggest setting a soft deadline
> > > > > (maybe
> > > > until
> > > > > beginning of next week) and then starting to cut the release
> > > > > (given
> > > that
> > > > no
> > > > > other blocker issues pop up). Maybe we are able to resolve the
> > > > > issues
> > > > even
> > > > > earlier which would allow us to cut the release also earlier.
> > > > >
> > > > > From my side I would like to include FLINK-18959 in the release.
> > > > > But it
> > > > is
> > > > > not a strict release blocker.
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > > > <[hidden email]>
> > > > > wrote:
> > > > >
> > > > > > I think it's worth considering whether we can get this bugfix
> > > included
> > > > in
> > > > > > 1.11.2:
> > > > > >
> > > > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > > > >
> > > > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > > > who
> > > has
> > > > > > been working on this.
> > > > > >
> > > > > > Regards,
> > > > > > David
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
> wrote:
> > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > > > not
> > > > too
> > > > > > far
> > > > > > > but
> > > > > > > we already have more than 80 resolved improvements/bugs in the
> > > > > > release-1.11
> > > > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > > > to
> > > > create
> > > > > > the
> > > > > > > next
> > > > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > > > >
> > > > > > > Most noticeable fixes are:
> > > > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > > > operations
> > > in
> > > > > > > per-job mode
> > > > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > > > nullability
> > > > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > > > thread
> > > > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > > > incorrectly
> > > > in
> > > > > > > JobManagerFlinkMemoryUtils.java
> > > > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > > > previous
> > > > > > > jdk8u72(-b01)
> > > > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > > > >
> > > > > > > Furthermore, I think the following blocker issue should be
> > > > > > > merged
> > > > > before
> > > > > > > 1.11.2 release
> > > > > > >
> > > > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > > > HiveBulkWriterFactory
> > > > > > >
> > > > > > > I would volunteer as the release manager and kick off the
> > > > > > > release
> > > > > > process.
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Please let me know if there are any concerns or any missing
> > > > > > > blocker
> > > > > > issues
> > > > > > > need to be fixed in 1.11.2.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Zhu Zhu
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Konstantin Knauf
> > > >
> > > > https://twitter.com/snntrable
> > > >
> > > > https://github.com/knaufk
> > > >
> > >
> >
>

dwysakowicz

Re: [DISCUSS] Releasing Flink 1.11.2

User has just reported another issue FLINK-19133 which I think should be
a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
possible.

On 03/09/2020 09:36, Zhu Zhu wrote:

> Thanks for the inputs!
> I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>
> And thanks a lot for offering help, zhijiang!
>
> Thanks,
> Zhu
>
> Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
>
>> Hi
>> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
>> bug introduce in 1.11.0), there is a pr for it already.
>> Best,
>> Congxian
>>
>>
>> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>>
>>> Hi,
>>>
>>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
>>> Pravega connector integration, hope we can have it included in 1.11.2
>>> release.
>>>
>>> Best Regards,
>>> Brian
>>>
>>> -----Original Message-----
>>> From: Becket Qin <[hidden email]>
>>> Sent: Thursday, September 3, 2020 11:18
>>> To: dev
>>> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
>>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>>>
>>>
>>> [EXTERNAL EMAIL]
>>>
>>> Hi Zhuzhu,
>>>
>>> Thanks for starting the discussion.
>>>
>>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
>>> from previous versions and is currently blocking the development of
>> Pravega
>>> source on top of FLIP-27.
>>>
>>> Thanks,
>>>
>>> Jiangjie (Becket) Qin
>>>
>>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
>>>
>>>> Thank you all for the inputs!
>>>>
>>>> I agree with Till that we should set a soft deadline first.
>>>> I'd like to propose next Monday if no new blocker issue pops up.
>>>> But feel free to raise your concerns if you feel next Monday as a
>>>> deadline may not work for fixes which should be a blocker for 1.11.2.
>>>>
>>>> Here's a summary of the wanted/blocker but still open fixes:
>>>> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>>>> because job is not finished when dispatcher close
>>>> - FLINK-18934 Idle stream does not advance watermark in connected
>>>> stream
>>>>
>>>> Thanks,
>>>> Zhu
>>>>
>>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>>>>
>>>>> I think it would be nice to include a fix for
>>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>>>>> affects a highly requested feature of Flink 1.11 quite severely.
>>>>>
>>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
>>>> wrote:
>>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>>>>>> volunteering
>>>> as
>>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>>>> release. I
>>>>>> think we already collected quite a bit of fixes which our users
>>>>>> will benefit from.
>>>>>>
>>>>>> For the pending fixes, I would suggest setting a soft deadline
>>>>>> (maybe
>>>>> until
>>>>>> beginning of next week) and then starting to cut the release
>>>>>> (given
>>>> that
>>>>> no
>>>>>> other blocker issues pop up). Maybe we are able to resolve the
>>>>>> issues
>>>>> even
>>>>>> earlier which would allow us to cut the release also earlier.
>>>>>>
>>>>>> From my side I would like to include FLINK-18959 in the release.
>>>>>> But it
>>>>> is
>>>>>> not a strict release blocker.
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>>>>>> <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> I think it's worth considering whether we can get this bugfix
>>>> included
>>>>> in
>>>>>>> 1.11.2:
>>>>>>>
>>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>>>>>>
>>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>>>>>>> who
>>>> has
>>>>>>> been working on this.
>>>>>>>
>>>>>>> Regards,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
>> wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>>>>>>>> not
>>>>> too
>>>>>>> far
>>>>>>>> but
>>>>>>>> we already have more than 80 resolved improvements/bugs in the
>>>>>>> release-1.11
>>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>>>>>>>> to
>>>>> create
>>>>>>> the
>>>>>>>> next
>>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>>>>>>>>
>>>>>>>> Most noticeable fixes are:
>>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>>>>>>>> operations
>>>> in
>>>>>>>> per-job mode
>>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>>>>>>>> nullability
>>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>>>>>>>> thread
>>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>>>>>>>> incorrectly
>>>>> in
>>>>>>>> JobManagerFlinkMemoryUtils.java
>>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>>>>>>>> - FLINK-18595 Deadlock during job shutdown
>>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>>>>>>>> previous
>>>>>>>> jdk8u72(-b01)
>>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>>>>>>>>
>>>>>>>> Furthermore, I think the following blocker issue should be
>>>>>>>> merged
>>>>>> before
>>>>>>>> 1.11.2 release
>>>>>>>>
>>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>>>> HiveBulkWriterFactory
>>>>>>>> I would volunteer as the release manager and kick off the
>>>>>>>> release
>>>>>>> process.
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Please let me know if there are any concerns or any missing
>>>>>>>> blocker
>>>>>>> issues
>>>>>>>> need to be fixed in 1.11.2.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Zhu Zhu
>>>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Konstantin Knauf
>>>>>
>>>>> https://twitter.com/snntrable
>>>>>
>>>>> https://github.com/knaufk
>>>>>

signature.asc (849 bytes) Download Attachment

Till Rohrmann

Re: [DISCUSS] Releasing Flink 1.11.2

Fyi, FLINK-18959 has been merged into the release-1.10 branch.

Cheers,
Till

On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <[hidden email]>
wrote:

> User has just reported another issue FLINK-19133 which I think should be
> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
> possible.
>
> On 03/09/2020 09:36, Zhu Zhu wrote:
> > Thanks for the inputs!
> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >
> > And thanks a lot for offering help, zhijiang!
> >
> > Thanks,
> > Zhu
> >
> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
> >
> >> Hi
> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
> fixes a
> >> bug introduce in 1.11.0), there is a pr for it already.
> >> Best,
> >> Congxian
> >>
> >>
> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
> >>
> >>> Hi,
> >>>
> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
> for
> >>> Pravega connector integration, hope we can have it included in 1.11.2
> >>> release.
> >>>
> >>> Best Regards,
> >>> Brian
> >>>
> >>> -----Original Message-----
> >>> From: Becket Qin <[hidden email]>
> >>> Sent: Thursday, September 3, 2020 11:18
> >>> To: dev
> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >>>
> >>>
> >>> [EXTERNAL EMAIL]
> >>>
> >>> Hi Zhuzhu,
> >>>
> >>> Thanks for starting the discussion.
> >>>
> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> >>> from previous versions and is currently blocking the development of
> >> Pravega
> >>> source on top of FLIP-27.
> >>>
> >>> Thanks,
> >>>
> >>> Jiangjie (Becket) Qin
> >>>
> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
> >>>
> >>>> Thank you all for the inputs!
> >>>>
> >>>> I agree with Till that we should set a soft deadline first.
> >>>> I'd like to propose next Monday if no new blocker issue pops up.
> >>>> But feel free to raise your concerns if you feel next Monday as a
> >>>> deadline may not work for fixes which should be a blocker for 1.11.2.
> >>>>
> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >>>> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> >>>> because job is not finished when dispatcher close
> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
> >>>> stream
> >>>>
> >>>> Thanks,
> >>>> Zhu
> >>>>
> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
> >>>>
> >>>>> I think it would be nice to include a fix for
> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
> >>>>>
> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
> >>>> wrote:
> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >>>>>> volunteering
> >>>> as
> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
> >>>> release. I
> >>>>>> think we already collected quite a bit of fixes which our users
> >>>>>> will benefit from.
> >>>>>>
> >>>>>> For the pending fixes, I would suggest setting a soft deadline
> >>>>>> (maybe
> >>>>> until
> >>>>>> beginning of next week) and then starting to cut the release
> >>>>>> (given
> >>>> that
> >>>>> no
> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
> >>>>>> issues
> >>>>> even
> >>>>>> earlier which would allow us to cut the release also earlier.
> >>>>>>
> >>>>>> From my side I would like to include FLINK-18959 in the release.
> >>>>>> But it
> >>>>> is
> >>>>>> not a strict release blocker.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Till
> >>>>>>
> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >>>>>> <[hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I think it's worth considering whether we can get this bugfix
> >>>> included
> >>>>> in
> >>>>>>> 1.11.2:
> >>>>>>>
> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>>>>>>
> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
> >>>>>>> who
> >>>> has
> >>>>>>> been working on this.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> David
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
> >> wrote:
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
> >>>>>>>> not
> >>>>> too
> >>>>>>> far
> >>>>>>>> but
> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
> >>>>>>> release-1.11
> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
> >>>>>>>> to
> >>>>> create
> >>>>>>> the
> >>>>>>>> next
> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >>>>>>>>
> >>>>>>>> Most noticeable fixes are:
> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >>>>>>>> operations
> >>>> in
> >>>>>>>> per-job mode
> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >>>>>>>> nullability
> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >>>>>>>> thread
> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >>>>>>>> incorrectly
> >>>>> in
> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >>>>>>>> previous
> >>>>>>>> jdk8u72(-b01)
> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
> >>>>>>>>
> >>>>>>>> Furthermore, I think the following blocker issue should be
> >>>>>>>> merged
> >>>>>> before
> >>>>>>>> 1.11.2 release
> >>>>>>>>
> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>>>> HiveBulkWriterFactory
> >>>>>>>> I would volunteer as the release manager and kick off the
> >>>>>>>> release
> >>>>>>> process.
> >>>>>>>> What do you think?
> >>>>>>>>
> >>>>>>>> Please let me know if there are any concerns or any missing
> >>>>>>>> blocker
> >>>>>>> issues
> >>>>>>>> need to be fixed in 1.11.2.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Zhu Zhu
> >>>>>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Konstantin Knauf
> >>>>>
> >>>>> https://twitter.com/snntrable
> >>>>>
> >>>>> https://github.com/knaufk
> >>>>>
>
>

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

@Dawid
Thanks for the information and preparing a fix for FLINK-19133!
I have made it a blocker for 1.11.2 and will keep tracking its status.

@Till
Thanks for the updates and efforts for FLINK-18959!

Thanks,
Zhu

Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：

> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>
> Cheers,
> Till
>
> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <[hidden email]>
> wrote:
>
>> User has just reported another issue FLINK-19133 which I think should be
>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>> possible.
>>
>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> > Thanks for the inputs!
>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >
>> > And thanks a lot for offering help, zhijiang!
>> >
>> > Thanks,
>> > Zhu
>> >
>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
>> >
>> >> Hi
>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>> fixes a
>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >> Best,
>> >> Congxian
>> >>
>> >>
>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>> >>
>> >>> Hi,
>> >>>
>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
>> for
>> >>> Pravega connector integration, hope we can have it included in 1.11.2
>> >>> release.
>> >>>
>> >>> Best Regards,
>> >>> Brian
>> >>>
>> >>> -----Original Message-----
>> >>> From: Becket Qin <[hidden email]>
>> >>> Sent: Thursday, September 3, 2020 11:18
>> >>> To: dev
>> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >>>
>> >>>
>> >>> [EXTERNAL EMAIL]
>> >>>
>> >>> Hi Zhuzhu,
>> >>>
>> >>> Thanks for starting the discussion.
>> >>>
>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> regression
>> >>> from previous versions and is currently blocking the development of
>> >> Pravega
>> >>> source on top of FLIP-27.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Jiangjie (Becket) Qin
>> >>>
>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
>> >>>
>> >>>> Thank you all for the inputs!
>> >>>>
>> >>>> I agree with Till that we should set a soft deadline first.
>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>> >>>> But feel free to raise your concerns if you feel next Monday as a
>> >>>> deadline may not work for fixes which should be a blocker for 1.11.2.
>> >>>>
>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> HiveBulkWriterFactory
>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>> >>>> because job is not finished when dispatcher close
>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>> >>>> stream
>> >>>>
>> >>>> Thanks,
>> >>>> Zhu
>> >>>>
>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>> >>>>
>> >>>>> I think it would be nice to include a fix for
>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>> >>>>>
>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]>
>> >>>> wrote:
>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >>>>>> volunteering
>> >>>> as
>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>> >>>> release. I
>> >>>>>> think we already collected quite a bit of fixes which our users
>> >>>>>> will benefit from.
>> >>>>>>
>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>> >>>>>> (maybe
>> >>>>> until
>> >>>>>> beginning of next week) and then starting to cut the release
>> >>>>>> (given
>> >>>> that
>> >>>>> no
>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>> >>>>>> issues
>> >>>>> even
>> >>>>>> earlier which would allow us to cut the release also earlier.
>> >>>>>>
>> >>>>>> From my side I would like to include FLINK-18959 in the release.
>> >>>>>> But it
>> >>>>> is
>> >>>>>> not a strict release blocker.
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Till
>> >>>>>>
>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >>>>>> <[hidden email]>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> I think it's worth considering whether we can get this bugfix
>> >>>> included
>> >>>>> in
>> >>>>>>> 1.11.2:
>> >>>>>>>
>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>>>>>>
>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>> >>>>>>> who
>> >>>> has
>> >>>>>>> been working on this.
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> David
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
>> >> wrote:
>> >>>>>>>> Hi All,
>> >>>>>>>>
>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>> >>>>>>>> not
>> >>>>> too
>> >>>>>>> far
>> >>>>>>>> but
>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
>> >>>>>>> release-1.11
>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>> >>>>>>>> to
>> >>>>> create
>> >>>>>>> the
>> >>>>>>>> next
>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >>>>>>>>
>> >>>>>>>> Most noticeable fixes are:
>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >>>>>>>> operations
>> >>>> in
>> >>>>>>>> per-job mode
>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >>>>>>>> nullability
>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >>>>>>>> thread
>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >>>>>>>> incorrectly
>> >>>>> in
>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >>>>>>>> previous
>> >>>>>>>> jdk8u72(-b01)
>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>> >>>>>>>>
>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >>>>>>>> merged
>> >>>>>> before
>> >>>>>>>> 1.11.2 release
>> >>>>>>>>
>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>>>> HiveBulkWriterFactory
>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >>>>>>>> release
>> >>>>>>> process.
>> >>>>>>>> What do you think?
>> >>>>>>>>
>> >>>>>>>> Please let me know if there are any concerns or any missing
>> >>>>>>>> blocker
>> >>>>>>> issues
>> >>>>>>>> need to be fixed in 1.11.2.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> Zhu Zhu
>> >>>>>>>>
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> Konstantin Knauf
>> >>>>>
>> >>>>> https://twitter.com/snntrable
>> >>>>>
>> >>>>> https://github.com/knaufk
>> >>>>>
>>
>>

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

Hi All,

Since there are still two 1.11.2 blockers items in progress, RC1 creation
will be postponed to tomorrow.

Thanks,
Zhu

Zhu Zhu <[hidden email]> 于2020年9月4日周五下午3:50写道：

> @Dawid
> Thanks for the information and preparing a fix for FLINK-19133!
> I have made it a blocker for 1.11.2 and will keep tracking its status.
>
> @Till
> Thanks for the updates and efforts for FLINK-18959!
>
> Thanks,
> Zhu
>
> Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：
>
>> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>>
>> Cheers,
>> Till
>>
>> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <[hidden email]>
>> wrote:
>>
>>> User has just reported another issue FLINK-19133 which I think should be
>>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>>> possible.
>>>
>>> On 03/09/2020 09:36, Zhu Zhu wrote:
>>> > Thanks for the inputs!
>>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>>> >
>>> > And thanks a lot for offering help, zhijiang!
>>> >
>>> > Thanks,
>>> > Zhu
>>> >
>>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
>>> >
>>> >> Hi
>>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>>> fixes a
>>> >> bug introduce in 1.11.0), there is a pr for it already.
>>> >> Best,
>>> >> Congxian
>>> >>
>>> >>
>>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
>>> for
>>> >>> Pravega connector integration, hope we can have it included in 1.11.2
>>> >>> release.
>>> >>>
>>> >>> Best Regards,
>>> >>> Brian
>>> >>>
>>> >>> -----Original Message-----
>>> >>> From: Becket Qin <[hidden email]>
>>> >>> Sent: Thursday, September 3, 2020 11:18
>>> >>> To: dev
>>> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
>>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>>> >>>
>>> >>>
>>> >>> [EXTERNAL EMAIL]
>>> >>>
>>> >>> Hi Zhuzhu,
>>> >>>
>>> >>> Thanks for starting the discussion.
>>> >>>
>>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>>> regression
>>> >>> from previous versions and is currently blocking the development of
>>> >> Pravega
>>> >>> source on top of FLIP-27.
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Jiangjie (Becket) Qin
>>> >>>
>>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
>>> >>>
>>> >>>> Thank you all for the inputs!
>>> >>>>
>>> >>>> I agree with Till that we should set a soft deadline first.
>>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>>> >>>> But feel free to raise your concerns if you feel next Monday as a
>>> >>>> deadline may not work for fixes which should be a blocker for
>>> 1.11.2.
>>> >>>>
>>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>> HiveBulkWriterFactory
>>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>>> >>>> because job is not finished when dispatcher close
>>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>>> >>>> stream
>>> >>>>
>>> >>>> Thanks,
>>> >>>> Zhu
>>> >>>>
>>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>>> >>>>
>>> >>>>> I think it would be nice to include a fix for
>>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>>> >>>>>
>>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <[hidden email]
>>> >
>>> >>>> wrote:
>>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>>> >>>>>> volunteering
>>> >>>> as
>>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>>> >>>> release. I
>>> >>>>>> think we already collected quite a bit of fixes which our users
>>> >>>>>> will benefit from.
>>> >>>>>>
>>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>>> >>>>>> (maybe
>>> >>>>> until
>>> >>>>>> beginning of next week) and then starting to cut the release
>>> >>>>>> (given
>>> >>>> that
>>> >>>>> no
>>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>>> >>>>>> issues
>>> >>>>> even
>>> >>>>>> earlier which would allow us to cut the release also earlier.
>>> >>>>>>
>>> >>>>>> From my side I would like to include FLINK-18959 in the release.
>>> >>>>>> But it
>>> >>>>> is
>>> >>>>>> not a strict release blocker.
>>> >>>>>>
>>> >>>>>> Cheers,
>>> >>>>>> Till
>>> >>>>>>
>>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>>> >>>>>> <[hidden email]>
>>> >>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> I think it's worth considering whether we can get this bugfix
>>> >>>> included
>>> >>>>> in
>>> >>>>>>> 1.11.2:
>>> >>>>>>>
>>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>> >>>>>>>
>>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>>> >>>>>>> who
>>> >>>> has
>>> >>>>>>> been working on this.
>>> >>>>>>>
>>> >>>>>>> Regards,
>>> >>>>>>> David
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
>>> >> wrote:
>>> >>>>>>>> Hi All,
>>> >>>>>>>>
>>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>>> >>>>>>>> not
>>> >>>>> too
>>> >>>>>>> far
>>> >>>>>>>> but
>>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
>>> >>>>>>> release-1.11
>>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>>> >>>>>>>> to
>>> >>>>> create
>>> >>>>>>> the
>>> >>>>>>>> next
>>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>>> >>>>>>>>
>>> >>>>>>>> Most noticeable fixes are:
>>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>>> >>>>>>>> operations
>>> >>>> in
>>> >>>>>>>> per-job mode
>>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>>> >>>>>>>> nullability
>>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>>> >>>>>>>> thread
>>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>>> >>>>>>>> incorrectly
>>> >>>>> in
>>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>>> >>>>>>>> previous
>>> >>>>>>>> jdk8u72(-b01)
>>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>>> >>>>>>>>
>>> >>>>>>>> Furthermore, I think the following blocker issue should be
>>> >>>>>>>> merged
>>> >>>>>> before
>>> >>>>>>>> 1.11.2 release
>>> >>>>>>>>
>>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>> >>>>> HiveBulkWriterFactory
>>> >>>>>>>> I would volunteer as the release manager and kick off the
>>> >>>>>>>> release
>>> >>>>>>> process.
>>> >>>>>>>> What do you think?
>>> >>>>>>>>
>>> >>>>>>>> Please let me know if there are any concerns or any missing
>>> >>>>>>>> blocker
>>> >>>>>>> issues
>>> >>>>>>>> need to be fixed in 1.11.2.
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> Zhu Zhu
>>> >>>>>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>>
>>> >>>>> Konstantin Knauf
>>> >>>>>
>>> >>>>> https://twitter.com/snntrable
>>> >>>>>
>>> >>>>> https://github.com/knaufk
>>> >>>>>
>>>
>>>

Jingsong Li

Re: [DISCUSS] Releasing Flink 1.11.2

Hi Zhu Zhu,

Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166

Will fix it soon.

Best,
Jingsong

On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <[hidden email]> wrote:

> Hi All,
>
> Since there are still two 1.11.2 blockers items in progress, RC1 creation
> will be postponed to tomorrow.
>
> Thanks,
> Zhu
>
> Zhu Zhu <[hidden email]> 于2020年9月4日周五下午3:50写道：
>
> > @Dawid
> > Thanks for the information and preparing a fix for FLINK-19133!
> > I have made it a blocker for 1.11.2 and will keep tracking its status.
> >
> > @Till
> > Thanks for the updates and efforts for FLINK-18959!
> >
> > Thanks,
> > Zhu
> >
> > Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：
> >
> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <[hidden email]
> >
> >> wrote:
> >>
> >>> User has just reported another issue FLINK-19133 which I think should
> be
> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
> >>> possible.
> >>>
> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
> >>> > Thanks for the inputs!
> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >>> >
> >>> > And thanks a lot for offering help, zhijiang!
> >>> >
> >>> > Thanks,
> >>> > Zhu
> >>> >
> >>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
> >>> >
> >>> >> Hi
> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
> >>> fixes a
> >>> >> bug introduce in 1.11.0), there is a pr for it already.
> >>> >> Best,
> >>> >> Congxian
> >>> >>
> >>> >>
> >>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
> >>> >>
> >>> >>> Hi,
> >>> >>>
> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
> blocker
> >>> for
> >>> >>> Pravega connector integration, hope we can have it included in
> 1.11.2
> >>> >>> release.
> >>> >>>
> >>> >>> Best Regards,
> >>> >>> Brian
> >>> >>>
> >>> >>> -----Original Message-----
> >>> >>> From: Becket Qin <[hidden email]>
> >>> >>> Sent: Thursday, September 3, 2020 11:18
> >>> >>> To: dev
> >>> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong Li
> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >>> >>>
> >>> >>>
> >>> >>> [EXTERNAL EMAIL]
> >>> >>>
> >>> >>> Hi Zhuzhu,
> >>> >>>
> >>> >>> Thanks for starting the discussion.
> >>> >>>
> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
> >>> regression
> >>> >>> from previous versions and is currently blocking the development of
> >>> >> Pravega
> >>> >>> source on top of FLIP-27.
> >>> >>>
> >>> >>> Thanks,
> >>> >>>
> >>> >>> Jiangjie (Becket) Qin
> >>> >>>
> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]> wrote:
> >>> >>>
> >>> >>>> Thank you all for the inputs!
> >>> >>>>
> >>> >>>> I agree with Till that we should set a soft deadline first.
> >>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
> >>> >>>> But feel free to raise your concerns if you feel next Monday as a
> >>> >>>> deadline may not work for fixes which should be a blocker for
> >>> 1.11.2.
> >>> >>>>
> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>> HiveBulkWriterFactory
> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> >>> >>>> because job is not finished when dispatcher close
> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
> >>> >>>> stream
> >>> >>>>
> >>> >>>> Thanks,
> >>> >>>> Zhu
> >>> >>>>
> >>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
> >>> >>>>
> >>> >>>>> I think it would be nice to include a fix for
> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
> >>> >>>>>
> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
> [hidden email]
> >>> >
> >>> >>>> wrote:
> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >>> >>>>>> volunteering
> >>> >>>> as
> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
> >>> >>>> release. I
> >>> >>>>>> think we already collected quite a bit of fixes which our users
> >>> >>>>>> will benefit from.
> >>> >>>>>>
> >>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
> >>> >>>>>> (maybe
> >>> >>>>> until
> >>> >>>>>> beginning of next week) and then starting to cut the release
> >>> >>>>>> (given
> >>> >>>> that
> >>> >>>>> no
> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
> >>> >>>>>> issues
> >>> >>>>> even
> >>> >>>>>> earlier which would allow us to cut the release also earlier.
> >>> >>>>>>
> >>> >>>>>> From my side I would like to include FLINK-18959 in the release.
> >>> >>>>>> But it
> >>> >>>>> is
> >>> >>>>>> not a strict release blocker.
> >>> >>>>>>
> >>> >>>>>> Cheers,
> >>> >>>>>> Till
> >>> >>>>>>
> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >>> >>>>>> <[hidden email]>
> >>> >>>>>> wrote:
> >>> >>>>>>
> >>> >>>>>>> I think it's worth considering whether we can get this bugfix
> >>> >>>> included
> >>> >>>>> in
> >>> >>>>>>> 1.11.2:
> >>> >>>>>>>
> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>> >>>>>>>
> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
> >>> >>>>>>> who
> >>> >>>> has
> >>> >>>>>>> been working on this.
> >>> >>>>>>>
> >>> >>>>>>> Regards,
> >>> >>>>>>> David
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
> >>> >> wrote:
> >>> >>>>>>>> Hi All,
> >>> >>>>>>>>
> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
> >>> >>>>>>>> not
> >>> >>>>> too
> >>> >>>>>>> far
> >>> >>>>>>>> but
> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
> >>> >>>>>>> release-1.11
> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
> >>> >>>>>>>> to
> >>> >>>>> create
> >>> >>>>>>> the
> >>> >>>>>>>> next
> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >>> >>>>>>>>
> >>> >>>>>>>> Most noticeable fixes are:
> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >>> >>>>>>>> operations
> >>> >>>> in
> >>> >>>>>>>> per-job mode
> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >>> >>>>>>>> nullability
> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >>> >>>>>>>> thread
> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >>> >>>>>>>> incorrectly
> >>> >>>>> in
> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >>> >>>>>>>> previous
> >>> >>>>>>>> jdk8u72(-b01)
> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
> >>> >>>>>>>>
> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
> >>> >>>>>>>> merged
> >>> >>>>>> before
> >>> >>>>>>>> 1.11.2 release
> >>> >>>>>>>>
> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>> >>>>> HiveBulkWriterFactory
> >>> >>>>>>>> I would volunteer as the release manager and kick off the
> >>> >>>>>>>> release
> >>> >>>>>>> process.
> >>> >>>>>>>> What do you think?
> >>> >>>>>>>>
> >>> >>>>>>>> Please let me know if there are any concerns or any missing
> >>> >>>>>>>> blocker
> >>> >>>>>>> issues
> >>> >>>>>>>> need to be fixed in 1.11.2.
> >>> >>>>>>>>
> >>> >>>>>>>> Thanks,
> >>> >>>>>>>> Zhu Zhu
> >>> >>>>>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>>
> >>> >>>>> Konstantin Knauf
> >>> >>>>>
> >>> >>>>> https://twitter.com/snntrable
> >>> >>>>>
> >>> >>>>> https://github.com/knaufk
> >>> >>>>>
> >>>
> >>>
>

--
Best, Jingsong Lee

Jingsong Li

Re: [DISCUSS] Releasing Flink 1.11.2

Hi Zhu Zhu,

Replenish its[1] influence:
For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
partition visible for downstream Hive/Spark engines.
But due to FLINK-19166, will lose some partitions to commit after Job
failover in some cases, especially for short partitions.
In the user's opinion, the data is lost, which does not conform to
exactly-once.

[1]https://issues.apache.org/jira/browse/FLINK-19166
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit

Best,
Jingsong

On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <[hidden email]> wrote:

> Hi Zhu Zhu,
>
> Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
>
> Will fix it soon.
>
> Best,
> Jingsong
>
> On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <[hidden email]> wrote:
>
>> Hi All,
>>
>> Since there are still two 1.11.2 blockers items in progress, RC1 creation
>> will be postponed to tomorrow.
>>
>> Thanks,
>> Zhu
>>
>> Zhu Zhu <[hidden email]> 于2020年9月4日周五下午3:50写道：
>>
>> > @Dawid
>> > Thanks for the information and preparing a fix for FLINK-19133!
>> > I have made it a blocker for 1.11.2 and will keep tracking its status.
>> >
>> > @Till
>> > Thanks for the updates and efforts for FLINK-18959!
>> >
>> > Thanks,
>> > Zhu
>> >
>> > Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：
>> >
>> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
>> [hidden email]>
>> >> wrote:
>> >>
>> >>> User has just reported another issue FLINK-19133 which I think should
>> be
>> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>> >>> possible.
>> >>>
>> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> >>> > Thanks for the inputs!
>> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >>> >
>> >>> > And thanks a lot for offering help, zhijiang!
>> >>> >
>> >>> > Thanks,
>> >>> > Zhu
>> >>> >
>> >>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
>> >>> >
>> >>> >> Hi
>> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>> >>> fixes a
>> >>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >>> >> Best,
>> >>> >> Congxian
>> >>> >>
>> >>> >>
>> >>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>> >>> >>
>> >>> >>> Hi,
>> >>> >>>
>> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
>> blocker
>> >>> for
>> >>> >>> Pravega connector integration, hope we can have it included in
>> 1.11.2
>> >>> >>> release.
>> >>> >>>
>> >>> >>> Best Regards,
>> >>> >>> Brian
>> >>> >>>
>> >>> >>> -----Original Message-----
>> >>> >>> From: Becket Qin <[hidden email]>
>> >>> >>> Sent: Thursday, September 3, 2020 11:18
>> >>> >>> To: dev
>> >>> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong
>> Li
>> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >>> >>>
>> >>> >>>
>> >>> >>> [EXTERNAL EMAIL]
>> >>> >>>
>> >>> >>> Hi Zhuzhu,
>> >>> >>>
>> >>> >>> Thanks for starting the discussion.
>> >>> >>>
>> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> >>> regression
>> >>> >>> from previous versions and is currently blocking the development
>> of
>> >>> >> Pravega
>> >>> >>> source on top of FLIP-27.
>> >>> >>>
>> >>> >>> Thanks,
>> >>> >>>
>> >>> >>> Jiangjie (Becket) Qin
>> >>> >>>
>> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]>
>> wrote:
>> >>> >>>
>> >>> >>>> Thank you all for the inputs!
>> >>> >>>>
>> >>> >>>> I agree with Till that we should set a soft deadline first.
>> >>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>> >>> >>>> But feel free to raise your concerns if you feel next Monday as a
>> >>> >>>> deadline may not work for fixes which should be a blocker for
>> >>> 1.11.2.
>> >>> >>>>
>> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>> HiveBulkWriterFactory
>> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
>> archiveExecutionGraph
>> >>> >>>> because job is not finished when dispatcher close
>> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>> >>> >>>> stream
>> >>> >>>>
>> >>> >>>> Thanks,
>> >>> >>>> Zhu
>> >>> >>>>
>> >>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>> >>> >>>>
>> >>> >>>>> I think it would be nice to include a fix for
>> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>> >>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>> >>> >>>>>
>> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
>> [hidden email]
>> >>> >
>> >>> >>>> wrote:
>> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >>> >>>>>> volunteering
>> >>> >>>> as
>> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>> >>> >>>> release. I
>> >>> >>>>>> think we already collected quite a bit of fixes which our users
>> >>> >>>>>> will benefit from.
>> >>> >>>>>>
>> >>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>> >>> >>>>>> (maybe
>> >>> >>>>> until
>> >>> >>>>>> beginning of next week) and then starting to cut the release
>> >>> >>>>>> (given
>> >>> >>>> that
>> >>> >>>>> no
>> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>> >>> >>>>>> issues
>> >>> >>>>> even
>> >>> >>>>>> earlier which would allow us to cut the release also earlier.
>> >>> >>>>>>
>> >>> >>>>>> From my side I would like to include FLINK-18959 in the
>> release.
>> >>> >>>>>> But it
>> >>> >>>>> is
>> >>> >>>>>> not a strict release blocker.
>> >>> >>>>>>
>> >>> >>>>>> Cheers,
>> >>> >>>>>> Till
>> >>> >>>>>>
>> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >>> >>>>>> <[hidden email]>
>> >>> >>>>>> wrote:
>> >>> >>>>>>
>> >>> >>>>>>> I think it's worth considering whether we can get this bugfix
>> >>> >>>> included
>> >>> >>>>> in
>> >>> >>>>>>> 1.11.2:
>> >>> >>>>>>>
>> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>> >>>>>>>
>> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
>> Roman,
>> >>> >>>>>>> who
>> >>> >>>> has
>> >>> >>>>>>> been working on this.
>> >>> >>>>>>>
>> >>> >>>>>>> Regards,
>> >>> >>>>>>> David
>> >>> >>>>>>>
>> >>> >>>>>>>
>> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
>> >>> >> wrote:
>> >>> >>>>>>>> Hi All,
>> >>> >>>>>>>>
>> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
>> It's
>> >>> >>>>>>>> not
>> >>> >>>>> too
>> >>> >>>>>>> far
>> >>> >>>>>>>> but
>> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
>> the
>> >>> >>>>>>> release-1.11
>> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>> >>> >>>>>>>> to
>> >>> >>>>> create
>> >>> >>>>>>> the
>> >>> >>>>>>>> next
>> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >>> >>>>>>>>
>> >>> >>>>>>>> Most noticeable fixes are:
>> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >>> >>>>>>>> operations
>> >>> >>>> in
>> >>> >>>>>>>> per-job mode
>> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >>> >>>>>>>> nullability
>> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >>> >>>>>>>> thread
>> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >>> >>>>>>>> incorrectly
>> >>> >>>>> in
>> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >>> >>>>>>>> previous
>> >>> >>>>>>>> jdk8u72(-b01)
>> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and
>> JM
>> >>> >>>>>>>>
>> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >>> >>>>>>>> merged
>> >>> >>>>>> before
>> >>> >>>>>>>> 1.11.2 release
>> >>> >>>>>>>>
>> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>> >>>>> HiveBulkWriterFactory
>> >>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >>> >>>>>>>> release
>> >>> >>>>>>> process.
>> >>> >>>>>>>> What do you think?
>> >>> >>>>>>>>
>> >>> >>>>>>>> Please let me know if there are any concerns or any missing
>> >>> >>>>>>>> blocker
>> >>> >>>>>>> issues
>> >>> >>>>>>>> need to be fixed in 1.11.2.
>> >>> >>>>>>>>
>> >>> >>>>>>>> Thanks,
>> >>> >>>>>>>> Zhu Zhu
>> >>> >>>>>>>>
>> >>> >>>>>
>> >>> >>>>> --
>> >>> >>>>>
>> >>> >>>>> Konstantin Knauf
>> >>> >>>>>
>> >>> >>>>> https://twitter.com/snntrable
>> >>> >>>>>
>> >>> >>>>> https://github.com/knaufk
>> >>> >>>>>
>> >>>
>> >>>
>>
>
>
> --
> Best, Jingsong Lee
>

--
Best, Jingsong Lee

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

Thanks for reporting this issue and offering to fix it @Jingsong Li
<[hidden email]>
Agreed it is a reasonable blocker. I will postpone 1.11.2 RC1 creation
until it is fixed.

Thanks,
Zhu

Jingsong Li <[hidden email]> 于2020年9月9日周三上午11:27写道：

> Hi Zhu Zhu,
>
> Replenish its[1] influence:
> For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
> partition visible for downstream Hive/Spark engines.
> But due to FLINK-19166, will lose some partitions to commit after Job
> failover in some cases, especially for short partitions.
> In the user's opinion, the data is lost, which does not conform to
> exactly-once.
>
> [1]https://issues.apache.org/jira/browse/FLINK-19166
> [2]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit
>
> Best,
> Jingsong
>
> On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <[hidden email]>
> wrote:
>
> > Hi Zhu Zhu,
> >
> > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
> >
> > Will fix it soon.
> >
> > Best,
> > Jingsong
> >
> > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <[hidden email]> wrote:
> >
> >> Hi All,
> >>
> >> Since there are still two 1.11.2 blockers items in progress, RC1
> creation
> >> will be postponed to tomorrow.
> >>
> >> Thanks,
> >> Zhu
> >>
> >> Zhu Zhu <[hidden email]> 于2020年9月4日周五下午3:50写道：
> >>
> >> > @Dawid
> >> > Thanks for the information and preparing a fix for FLINK-19133!
> >> > I have made it a blocker for 1.11.2 and will keep tracking its status.
> >> >
> >> > @Till
> >> > Thanks for the updates and efforts for FLINK-18959!
> >> >
> >> > Thanks,
> >> > Zhu
> >> >
> >> > Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：
> >> >
> >> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
> >> >>
> >> >> Cheers,
> >> >> Till
> >> >>
> >> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
> >> [hidden email]>
> >> >> wrote:
> >> >>
> >> >>> User has just reported another issue FLINK-19133 which I think
> should
> >> be
> >> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon
> as
> >> >>> possible.
> >> >>>
> >> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
> >> >>> > Thanks for the inputs!
> >> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >> >>> >
> >> >>> > And thanks a lot for offering help, zhijiang!
> >> >>> >
> >> >>> > Thanks,
> >> >>> > Zhu
> >> >>> >
> >> >>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
> >> >>> >
> >> >>> >> Hi
> >> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942
> (this
> >> >>> fixes a
> >> >>> >> bug introduce in 1.11.0), there is a pr for it already.
> >> >>> >> Best,
> >> >>> >> Congxian
> >> >>> >>
> >> >>> >>
> >> >>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
> >> >>> >>
> >> >>> >>> Hi,
> >> >>> >>>
> >> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
> >> blocker
> >> >>> for
> >> >>> >>> Pravega connector integration, hope we can have it included in
> >> 1.11.2
> >> >>> >>> release.
> >> >>> >>>
> >> >>> >>> Best Regards,
> >> >>> >>> Brian
> >> >>> >>>
> >> >>> >>> -----Original Message-----
> >> >>> >>> From: Becket Qin <[hidden email]>
> >> >>> >>> Sent: Thursday, September 3, 2020 11:18
> >> >>> >>> To: dev
> >> >>> >>> Cc: [hidden email]; Till Rohrmann; david; Jingsong
> >> Li
> >> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >> >>> >>>
> >> >>> >>>
> >> >>> >>> [EXTERNAL EMAIL]
> >> >>> >>>
> >> >>> >>> Hi Zhuzhu,
> >> >>> >>>
> >> >>> >>> Thanks for starting the discussion.
> >> >>> >>>
> >> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
> >> >>> regression
> >> >>> >>> from previous versions and is currently blocking the development
> >> of
> >> >>> >> Pravega
> >> >>> >>> source on top of FLIP-27.
> >> >>> >>>
> >> >>> >>> Thanks,
> >> >>> >>>
> >> >>> >>> Jiangjie (Becket) Qin
> >> >>> >>>
> >> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]>
> >> wrote:
> >> >>> >>>
> >> >>> >>>> Thank you all for the inputs!
> >> >>> >>>>
> >> >>> >>>> I agree with Till that we should set a soft deadline first.
> >> >>> >>>> I'd like to propose next Monday if no new blocker issue pops
> up.
> >> >>> >>>> But feel free to raise your concerns if you feel next Monday
> as a
> >> >>> >>>> deadline may not work for fixes which should be a blocker for
> >> >>> 1.11.2.
> >> >>> >>>>
> >> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >> >>> HiveBulkWriterFactory
> >> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
> >> archiveExecutionGraph
> >> >>> >>>> because job is not finished when dispatcher close
> >> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in
> connected
> >> >>> >>>> stream
> >> >>> >>>>
> >> >>> >>>> Thanks,
> >> >>> >>>> Zhu
> >> >>> >>>>
> >> >>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
> >> >>> >>>>
> >> >>> >>>>> I think it would be nice to include a fix for
> >> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >> >>> >>>>> affects a highly requested feature of Flink 1.11 quite
> severely.
> >> >>> >>>>>
> >> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
> >> [hidden email]
> >> >>> >
> >> >>> >>>> wrote:
> >> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >> >>> >>>>>> volunteering
> >> >>> >>>> as
> >> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug
> fix
> >> >>> >>>> release. I
> >> >>> >>>>>> think we already collected quite a bit of fixes which our
> users
> >> >>> >>>>>> will benefit from.
> >> >>> >>>>>>
> >> >>> >>>>>> For the pending fixes, I would suggest setting a soft
> deadline
> >> >>> >>>>>> (maybe
> >> >>> >>>>> until
> >> >>> >>>>>> beginning of next week) and then starting to cut the release
> >> >>> >>>>>> (given
> >> >>> >>>> that
> >> >>> >>>>> no
> >> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve
> the
> >> >>> >>>>>> issues
> >> >>> >>>>> even
> >> >>> >>>>>> earlier which would allow us to cut the release also earlier.
> >> >>> >>>>>>
> >> >>> >>>>>> From my side I would like to include FLINK-18959 in the
> >> release.
> >> >>> >>>>>> But it
> >> >>> >>>>> is
> >> >>> >>>>>> not a strict release blocker.
> >> >>> >>>>>>
> >> >>> >>>>>> Cheers,
> >> >>> >>>>>> Till
> >> >>> >>>>>>
> >> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >> >>> >>>>>> <[hidden email]>
> >> >>> >>>>>> wrote:
> >> >>> >>>>>>
> >> >>> >>>>>>> I think it's worth considering whether we can get this
> bugfix
> >> >>> >>>> included
> >> >>> >>>>> in
> >> >>> >>>>>>> 1.11.2:
> >> >>> >>>>>>>
> >> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >> >>> >>>>>>>
> >> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
> >> Roman,
> >> >>> >>>>>>> who
> >> >>> >>>> has
> >> >>> >>>>>>> been working on this.
> >> >>> >>>>>>>
> >> >>> >>>>>>> Regards,
> >> >>> >>>>>>> David
> >> >>> >>>>>>>
> >> >>> >>>>>>>
> >> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
> >> >>> >> wrote:
> >> >>> >>>>>>>> Hi All,
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
> >> It's
> >> >>> >>>>>>>> not
> >> >>> >>>>> too
> >> >>> >>>>>>> far
> >> >>> >>>>>>>> but
> >> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
> >> the
> >> >>> >>>>>>> release-1.11
> >> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I
> propose
> >> >>> >>>>>>>> to
> >> >>> >>>>> create
> >> >>> >>>>>>> the
> >> >>> >>>>>>>> next
> >> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Most noticeable fixes are:
> >> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >> >>> >>>>>>>> operations
> >> >>> >>>> in
> >> >>> >>>>>>>> per-job mode
> >> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0
> table
> >> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >> >>> >>>>>>>> nullability
> >> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >> >>> >>>>>>>> thread
> >> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >> >>> >>>>>>>> incorrectly
> >> >>> >>>>> in
> >> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server
> shutdown
> >> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >> >>> >>>>>>>> previous
> >> >>> >>>>>>>> jdk8u72(-b01)
> >> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and
> >> JM
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
> >> >>> >>>>>>>> merged
> >> >>> >>>>>> before
> >> >>> >>>>>>>> 1.11.2 release
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >> >>> >>>>> HiveBulkWriterFactory
> >> >>> >>>>>>>> I would volunteer as the release manager and kick off the
> >> >>> >>>>>>>> release
> >> >>> >>>>>>> process.
> >> >>> >>>>>>>> What do you think?
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Please let me know if there are any concerns or any missing
> >> >>> >>>>>>>> blocker
> >> >>> >>>>>>> issues
> >> >>> >>>>>>>> need to be fixed in 1.11.2.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Thanks,
> >> >>> >>>>>>>> Zhu Zhu
> >> >>> >>>>>>>>
> >> >>> >>>>>
> >> >>> >>>>> --
> >> >>> >>>>>
> >> >>> >>>>> Konstantin Knauf
> >> >>> >>>>>
> >> >>> >>>>> https://twitter.com/snntrable
> >> >>> >>>>>
> >> >>> >>>>> https://github.com/knaufk
> >> >>> >>>>>
> >> >>>
> >> >>>
> >>
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
>
> --
> Best, Jingsong Lee
>

Zhu Zhu

Re: [DISCUSS] Releasing Flink 1.11.2

Hi All,

Just an update.
All known blockers are resolved and I'm starting to create RC1 for release
1.11.2.

Thanks,
Zhu

Zhu Zhu <[hidden email]> 于2020年9月9日周三上午11:36写道：

> Thanks for reporting this issue and offering to fix it @Jingsong Li
> <[hidden email]>
> Agreed it is a reasonable blocker. I will postpone 1.11.2 RC1 creation
> until it is fixed.
>
> Thanks,
> Zhu
>
> Jingsong Li <[hidden email]> 于2020年9月9日周三上午11:27写道：
>
>> Hi Zhu Zhu,
>>
>> Replenish its[1] influence:
>> For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
>> partition visible for downstream Hive/Spark engines.
>> But due to FLINK-19166, will lose some partitions to commit after Job
>> failover in some cases, especially for short partitions.
>> In the user's opinion, the data is lost, which does not conform to
>> exactly-once.
>>
>> [1]https://issues.apache.org/jira/browse/FLINK-19166
>> [2]
>>
>> https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit
>>
>> Best,
>> Jingsong
>>
>> On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <[hidden email]>
>> wrote:
>>
>> > Hi Zhu Zhu,
>> >
>> > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
>> >
>> > Will fix it soon.
>> >
>> > Best,
>> > Jingsong
>> >
>> > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <[hidden email]> wrote:
>> >
>> >> Hi All,
>> >>
>> >> Since there are still two 1.11.2 blockers items in progress, RC1
>> creation
>> >> will be postponed to tomorrow.
>> >>
>> >> Thanks,
>> >> Zhu
>> >>
>> >> Zhu Zhu <[hidden email]> 于2020年9月4日周五下午3:50写道：
>> >>
>> >> > @Dawid
>> >> > Thanks for the information and preparing a fix for FLINK-19133!
>> >> > I have made it a blocker for 1.11.2 and will keep tracking its
>> status.
>> >> >
>> >> > @Till
>> >> > Thanks for the updates and efforts for FLINK-18959!
>> >> >
>> >> > Thanks,
>> >> > Zhu
>> >> >
>> >> > Till Rohrmann <[hidden email]> 于2020年9月4日周五下午3:41写道：
>> >> >
>> >> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>> >> >>
>> >> >> Cheers,
>> >> >> Till
>> >> >>
>> >> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
>> >> [hidden email]>
>> >> >> wrote:
>> >> >>
>> >> >>> User has just reported another issue FLINK-19133 which I think
>> should
>> >> be
>> >> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as
>> soon as
>> >> >>> possible.
>> >> >>>
>> >> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> >> >>> > Thanks for the inputs!
>> >> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >> >>> >
>> >> >>> > And thanks a lot for offering help, zhijiang!
>> >> >>> >
>> >> >>> > Thanks,
>> >> >>> > Zhu
>> >> >>> >
>> >> >>> > Congxian Qiu <[hidden email]> 于2020年9月3日周四下午3:18写道：
>> >> >>> >
>> >> >>> >> Hi
>> >> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942
>> (this
>> >> >>> fixes a
>> >> >>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >> >>> >> Best,
>> >> >>> >> Congxian
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> Zhou, Brian <[hidden email]> 于2020年9月3日周四上午11:21写道：
>> >> >>> >>
>> >> >>> >>> Hi,
>> >> >>> >>>
>> >> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
>> >> blocker
>> >> >>> for
>> >> >>> >>> Pravega connector integration, hope we can have it included in
>> >> 1.11.2
>> >> >>> >>> release.
>> >> >>> >>>
>> >> >>> >>> Best Regards,
>> >> >>> >>> Brian
>> >> >>> >>>
>> >> >>> >>> -----Original Message-----
>> >> >>> >>> From: Becket Qin <[hidden email]>
>> >> >>> >>> Sent: Thursday, September 3, 2020 11:18
>> >> >>> >>> To: dev
>> >> >>> >>> Cc: [hidden email]; Till Rohrmann; david;
>> Jingsong
>> >> Li
>> >> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >> >>> >>>
>> >> >>> >>>
>> >> >>> >>> [EXTERNAL EMAIL]
>> >> >>> >>>
>> >> >>> >>> Hi Zhuzhu,
>> >> >>> >>>
>> >> >>> >>> Thanks for starting the discussion.
>> >> >>> >>>
>> >> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> >> >>> regression
>> >> >>> >>> from previous versions and is currently blocking the
>> development
>> >> of
>> >> >>> >> Pravega
>> >> >>> >>> source on top of FLIP-27.
>> >> >>> >>>
>> >> >>> >>> Thanks,
>> >> >>> >>>
>> >> >>> >>> Jiangjie (Becket) Qin
>> >> >>> >>>
>> >> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <[hidden email]>
>> >> wrote:
>> >> >>> >>>
>> >> >>> >>>> Thank you all for the inputs!
>> >> >>> >>>>
>> >> >>> >>>> I agree with Till that we should set a soft deadline first.
>> >> >>> >>>> I'd like to propose next Monday if no new blocker issue pops
>> up.
>> >> >>> >>>> But feel free to raise your concerns if you feel next Monday
>> as a
>> >> >>> >>>> deadline may not work for fixes which should be a blocker for
>> >> >>> 1.11.2.
>> >> >>> >>>>
>> >> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >> >>> HiveBulkWriterFactory
>> >> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
>> >> archiveExecutionGraph
>> >> >>> >>>> because job is not finished when dispatcher close
>> >> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in
>> connected
>> >> >>> >>>> stream
>> >> >>> >>>>
>> >> >>> >>>> Thanks,
>> >> >>> >>>> Zhu
>> >> >>> >>>>
>> >> >>> >>>> Konstantin Knauf <[hidden email]> 于2020年9月2日周三下午9:00写道：
>> >> >>> >>>>
>> >> >>> >>>>> I think it would be nice to include a fix for
>> >> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as
>> it
>> >> >>> >>>>> affects a highly requested feature of Flink 1.11 quite
>> severely.
>> >> >>> >>>>>
>> >> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
>> >> [hidden email]
>> >> >>> >
>> >> >>> >>>> wrote:
>> >> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >> >>> >>>>>> volunteering
>> >> >>> >>>> as
>> >> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug
>> fix
>> >> >>> >>>> release. I
>> >> >>> >>>>>> think we already collected quite a bit of fixes which our
>> users
>> >> >>> >>>>>> will benefit from.
>> >> >>> >>>>>>
>> >> >>> >>>>>> For the pending fixes, I would suggest setting a soft
>> deadline
>> >> >>> >>>>>> (maybe
>> >> >>> >>>>> until
>> >> >>> >>>>>> beginning of next week) and then starting to cut the release
>> >> >>> >>>>>> (given
>> >> >>> >>>> that
>> >> >>> >>>>> no
>> >> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve
>> the
>> >> >>> >>>>>> issues
>> >> >>> >>>>> even
>> >> >>> >>>>>> earlier which would allow us to cut the release also
>> earlier.
>> >> >>> >>>>>>
>> >> >>> >>>>>> From my side I would like to include FLINK-18959 in the
>> >> release.
>> >> >>> >>>>>> But it
>> >> >>> >>>>> is
>> >> >>> >>>>>> not a strict release blocker.
>> >> >>> >>>>>>
>> >> >>> >>>>>> Cheers,
>> >> >>> >>>>>> Till
>> >> >>> >>>>>>
>> >> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >> >>> >>>>>> <[hidden email]>
>> >> >>> >>>>>> wrote:
>> >> >>> >>>>>>
>> >> >>> >>>>>>> I think it's worth considering whether we can get this
>> bugfix
>> >> >>> >>>> included
>> >> >>> >>>>> in
>> >> >>> >>>>>>> 1.11.2:
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
>> >> Roman,
>> >> >>> >>>>>>> who
>> >> >>> >>>> has
>> >> >>> >>>>>>> been working on this.
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> Regards,
>> >> >>> >>>>>>> David
>> >> >>> >>>>>>>
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <[hidden email]>
>> >> >>> >> wrote:
>> >> >>> >>>>>>>> Hi All,
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
>> >> It's
>> >> >>> >>>>>>>> not
>> >> >>> >>>>> too
>> >> >>> >>>>>>> far
>> >> >>> >>>>>>>> but
>> >> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
>> >> the
>> >> >>> >>>>>>> release-1.11
>> >> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I
>> propose
>> >> >>> >>>>>>>> to
>> >> >>> >>>>> create
>> >> >>> >>>>>>> the
>> >> >>> >>>>>>>> next
>> >> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Most noticeable fixes are:
>> >> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >> >>> >>>>>>>> operations
>> >> >>> >>>> in
>> >> >>> >>>>>>>> per-job mode
>> >> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0
>> table
>> >> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >> >>> >>>>>>>> nullability
>> >> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >> >>> >>>>>>>> thread
>> >> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >> >>> >>>>>>>> incorrectly
>> >> >>> >>>>> in
>> >> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server
>> shutdown
>> >> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >> >>> >>>>>>>> previous
>> >> >>> >>>>>>>> jdk8u72(-b01)
>> >> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM
>> and
>> >> JM
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >> >>> >>>>>>>> merged
>> >> >>> >>>>>> before
>> >> >>> >>>>>>>> 1.11.2 release
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >> >>> >>>>> HiveBulkWriterFactory
>> >> >>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >> >>> >>>>>>>> release
>> >> >>> >>>>>>> process.
>> >> >>> >>>>>>>> What do you think?
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Please let me know if there are any concerns or any
>> missing
>> >> >>> >>>>>>>> blocker
>> >> >>> >>>>>>> issues
>> >> >>> >>>>>>>> need to be fixed in 1.11.2.
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Thanks,
>> >> >>> >>>>>>>> Zhu Zhu
>> >> >>> >>>>>>>>
>> >> >>> >>>>>
>> >> >>> >>>>> --
>> >> >>> >>>>>
>> >> >>> >>>>> Konstantin Knauf
>> >> >>> >>>>>
>> >> >>> >>>>> https://twitter.com/snntrable
>> >> >>> >>>>>
>> >> >>> >>>>> https://github.com/knaufk
>> >> >>> >>>>>
>> >> >>>
>> >> >>>
>> >>
>> >
>> >
>> > --
>> > Best, Jingsong Lee
>> >
>>
>>
>> --
>> Best, Jingsong Lee
>>
>